20081014 Poker Maths Paper Short

Kudorian Journal of Mathematical Analysis, Vol. 17, No. 4, 2008

A MAXIMUM LIKELIHOOD APPROACH TO ESTIMATING A POKER PLAYER’S STRENGTH, AGGRESSION AND TIGHTNESS USING PROFIT AND BUY-IN DATA Nicolas Paul Hare and Gerald Dribbler The Kudos Warrior Institute of Technology c/o The Rabbit’s Nest Schooner and Spade public house Charing Cross London [email protected]

Abstract. Modelling poker winnings as a Brownian motion, we use a maximum-likelihood approach to estimate a poker player’s overall strength, and degree of looseness / aggression, from data showing their profits per session and maximum in-session loss. Key words: poker, Kudos Warriors, Brownian motion, maximum likelihood

1. INTRODUCTION Poker players often describe players’ general styles of play in terms of several key characteristics. First, there is his strength as a player, which ultimately determines his overall profit rate. Second, there is his tightness, which roughly equates to how many hands he plays: a tight player will tend to enter few pots, but will generally have good cards when he does, whereas a loose player will enter many pots, and will have a correspondingly wider range of possible hands. Finally there is his aggression, which determines the degree to which he bets and raises: an aggressive player will tend to ‘lead the action’ while a passive one will tend to follow: calling, but not betting or raising. In this paper, we look at possible methods of defining these characteristics in terms of the effect they have on the time-path of a player’s profit or loss during a period of poker play. Modelling a player’s profit as a Brownian motion, we then derive probability distributions for both profit at the end of a session, and the lowest point reached during the session, for a given set of characteristics. We then use a maximum-likelihood approach to derive best estimates of the values of these characteristics, for a particular player, given a certain set of results.

2. MODELLING STYLES OF PLAY

417

Kudorian Journal of Mathematical Analysis, Vol. 17, No. 4, 2008

Over the course of a poker session, a poker player’s profit (i.e. his current level of chips, less the number of chips he has purchased during play) will fluctuate. A strong player’s profit will tend, on average, to move upwards. A tight player will enter fewer hands. Finally, an aggressive player will tend to find themselves in bigger pots, due to their tending to bet and raise, which will at least increase the variability of their winnings. This last point is not totally straightforward: obviously, in a particular poker hand, every player, aggressive and passive, contributes the same amount. But over a sufficiently long period of time, aggressive players will find themselves in pots with other passive players and aggressive players, whereas passive players will at least on occasion be in pots only with other passive players. This means that we should expect to see bigger swings, in the long run, in the fortunes of aggressive players compared to more passive players with the same strength (i.e. who have the same expected winnings per hand.) Although there are a number of ways we could describe the effects of these characteristics on a player’s profit, modelling profit as Brownian motion (BM) has a number of advantages. Being a Markov process, BM has no ‘memory’, which is similar to poker in that a player’s profits up to now do not affect their expected winnings in the next hand (or should not – the phenomenon of ‘tilt’ might mean that a player who has lost money will have lower expectation on the next hand due to bad play). If the number of hands played during a session is sufficiently large, the approximation to the normal distribution embodied in BM is justified. Finally, Brownian motion has relatively simple and well-understood properties compared to other stochastic data-generating processes (such as a discrete-time random walk). Of course in other ways a poker player’s profit is not like BM. For example, wins and losses come in discrete chunks, not a continuous trickle and during any one hand, a player’s losses are limited by the chips he has in front of him – there is no such constraint in unbounded BM. But these effects are dominated, over a sufficient amount of time, by the other features which make BM at least a worthwhile approximation. As an illustration, one of the following charts depicts real-life poker results over a number of games, and the other depicts random series generated using a BM-approximating process. They are difficult to distinguish from one another:

418

Kudorian Journal of Mathematical Analysis, Vol. 17, No. 4, 2008

(The bottom one actually shows the real-life results.) If we are modelling a player’s profit using BM, how should we attempt to model strength, aggression and tightness? A BM is characterised by just two variables: drift ( ) and variance ( 2). During a time t, the distribution of the amount a BM will have moved is normally distributed with mean t and variance 2t. In fact, these three variables (drift, variance and time) lend themselves quite readily to a poker interpretation. We should expect drift to be correlated with strength: that is, a good player will, over time, make a profit (this is true almost by definition). As discussed above, a player’s aggression is plausibly correlated with the variance of the BM describing his profit. Tightness is a little more complex, but if we think of a tight player as playing fewer hands, this is similar to running the process for less time. Consequently, we can think of it as the proportion of hands played, so that over ‘real’ time of 1, the BM describing that player’s profit-path will only move through p, where p is the proportion of hands played. (Actually, a higher p means more hands played, so we could define tightness as something like p-1). Consequently, we can describe this interpretation of a player’s playing style using a BM with drift equal to a player’s strength (in terms of profit per session) and variance of (aggression / tightness). Or, if preferred, if BM( , 2) describes the distribution of a player’s profit, we can say that his strength is approximated by , that his aggression is approximated by ( 2 x tightness), and that his tightness is approximated by (aggression / 2). That aggression and tightness are defined in terms of one another is not a surprise: a BM has only two defining variables, and we cannot preserve 419

Kudorian Journal of Mathematical Analysis, Vol. 17, No. 4, 2008

information from three characteristics within only two variables. This becomes a problem when we want to derive most likely values for strength, aggression and tightness, given a set of results, as those results only (by assumption) represent a twovariable process. So far, this interpretation is rather solipsistic in that it treats a player’s profit as a datagenerating process which involves only his own characteristics and thus ignores other players. To be more accurate, we should really interpret the characteristic BM of a player as his BM given certain other circumstances of the game: the strength and number of other players and so on. To make, as we do below, the assumption that these conditions are constant across games is not a very good approximation to real life, but it is an assumption that can be made to do useful work. There are a number of ways in which the model could be expanded to take account of other players’ strengths but these are beyond the scope of this particular paper. 3. ESTIMATING A PLAYER’S STYLE FROM OBSERVED RESULTS In our simple model as outlined above, we have defined strength, aggression and tightness in terms of the drift and variance of a BM which describes a player’s profit over time. Unfortunately, we do not have access to real-time data concerning a player’s profit over time, and so have to rely on two key pieces of information: a player’s profit levels at the end of every session, and their ‘buy-in’, which represents the lowest point their profit reached during the session1. The first stage in our estimation is to define the likelihood function of a particular result, given strength , aggression and tightness . Given how we have defined them above, a combination of these characteristics yields a BM with drift and variance -1, where time t is in units of identically-sized poker sessions. After t=1 (i.e. the end of a session), the player’s profit will therefore be normally distributed -1 with mean and variance . If the end-session profit is equal to then the probability density at that point is therefore given by the familiar:

which, given our definitions, is equal to:

1

In fact this is not quite so, at least in Kudorian games. Under current Kudorian practice, a player may buy in for either £5 or £10 as long as he would not have more than £12.50 after the buy-in. This means that, in fact, a player’s lowest point could be anywhere between (the negative of) his total buy-in, or his total buy-in less £12.50. However, we ignore this for the purposes of this section – it would be extremely easy to accommodate this consideration using e.g. high- and low-end estimates for a range of cases.

420

Kudorian Journal of Mathematical Analysis, Vol. 17, No. 4, 2008

The probability density function of the minimum of a BM is slightly less well-known. In this case, we need to know not the unconditional probability density of the minimum of a BM with the above characteristics, but the probability density of the minimum of such a BM given that it ends at t=1 on : in other words, a Brownian bridge between 0 at t=0 and at t=1. This can be computed from derivations in, among other places, Beghin and Orsingher (1999), who give the (reverse) cumulative probability function for the maximum of a Brownian bridge (with drift) of variance unity as follows:

As the authors note, the drift term does not feature in this formula because the stipulation that B (t)= effectively cancels it out. Taking account of the fact that the distribution of the maximum of a Brownian bridge ending on is identical to the distribution of the minimum of such a bridge ending on yields the following:

Because we will differentiate to find the density function, we then take its complement:

Next, we must take account of the fact that this is for a standardised BM with unity variance. Because B( , 2, t) = B( -2, 1, t 2) due to scale-invariance, we obtain:

We then substitute using our definitions of the poker characteristics, and letting t=1:

Finally, we differentiate to obtain the probability density function for , the observed buy-in (i.e. the maximum loss) as follows:

We now have the two components of our likelihood function – the probability that end-profit will equal and that during the same session, buy-in will equal :

To make differentiation easier, we then take the log of this equation:

421

Kudorian Journal of Mathematical Analysis, Vol. 17, No. 4, 2008

This is the log-likelihood of the results and being obtained in a single session for a player with playing characteristics , and . To obtain the log-likelihood for all the results obtained by that player, we simply take the sum of the log-likelihoods for every game. This gives us the following, where N is the total number of games played, and (for example) is the final profit in game i:

This gives us the following first-order conditions for the likelihood-maximising values of , and , given a player’s results ( :

The first of these solves straightforwardly to:

This shows that our best estimate of a player’s strength (defined as their expected profit per session, or the drift of the BM which describes their profit distribution) is their average profit per session, which is totally unsurprising.

422

Kudorian Journal of Mathematical Analysis, Vol. 17, No. 4, 2008

We can then substitute our estimate of into the second and third conditions. Rearranging either of these yields the same result:

(The reader can check for themselves that the second-order conditions are negative where the first-order conditions are jointly true, showing that these solutions are indeed maxima.) As discussed above, the interaction between the aggression and tightness terms cannot be unpicked. This leaves us with a single term, units of aggression per units of tightness, or variance per unit of time played. The reason we cannot solve the two terms separately is, mathematically, because they occur together in the determination of the various components of the log-likelihood function. Conceptually, it is because we cannot tell the difference between a high-variance, short BM and a low-variance, long BM. To put this in poker terms, if our model is accurate, we cannot tell the difference between an aggressive player who only plays a few hands, and a less aggressive player who plays lots of hands, at least not just by observing buy-ins and end-session profits. However, this does not mean that the metric is without descriptive value. What it tells us is the degree to which a player is loose, or aggressive, or both – we could call it the ‘L-Ag’, or looseness-aggression metric. We will leave it to others to evaluate its usefulness as a predictor of profit, but we suggest that it does at least have descriptive value in that it provides a way of quantifying a player’s activity level – we should expect to see passive players and rocks with a low L-Ag score, while maniacs and calling stations should be at the other end of the scale. A future paper will explore some of the descriptive applications of the L-Ag score.

423