A Brownian Motion Inspired ELO Model

Jan. 9, 2019, 10:15 p.m.



I've been meaning to document my ELO model for a while so here we go!

My model is ELO based with a few tweaks. We estimate teams' scores and then the margin using a combination of each teams' offensive strength, defensive strength and scoring accuracy.

Then turning the margin into a win probability involves considering the involved teams' historic variability in scoring to estimate the variance we expect in the predicted margin. We then combine the estimated margin and variance to estimate a probability using observations that AFL scoring tends to follow Brownian Motion which is discussed more here.

Before getting into the nuts and bolts we'll start with some observations and assumptions.

Assumptions

  1. Good teams generate more shots and give away fewer shots than poor teams. Good teams also tend to generate higher quality shots (closer to goal or at straighter angles or both). In the absense of expected score data I'll be using a teams rolling scoring accuracy over some previous time period as a proxy for the quality of shots a team generates.

  2. Mapping margin to win probability seems to be dynamic in that other modellers use different parameters in their margin -> probability mapping function for different eras of football. Instead, my algorithm will consider the uncertainty each team brings to the table separately and convert expected margin to win probability dynamically for each match.

    So in my algorithm we may predict the same margin for two matches, but we will be slightly more confident about a match between two lower variance teams than two higher variance teams, and so our win probability will be closer to 50% for the higher variance teams.

    This means my algorithm updates as scoring trends change without the need to tune different parameters to different eras of football.

    I will probably do an entire blog post on this topic and explore the validity of it in more detail. Variance is interesting!

With these assumptions aside, here are the nuts and bolts:

Ratings and Predicting

Teams have 4 ratings. These are:

  • Offence: Represents the number of shots a team can expect to produce (higher is better).
  • Defence: Represents the number of shots a team can expect to give away (lower is better).
  • Scoring Accuracy: \( \frac{\text{goals}}{\text{goals} + \text{behinds}} \) for the teams' last \(X\) matches.
  • Uncertainty: Represents the standard deviation in the error of my predicted score (in points) for the last \(Y\) matches.

    If I tend to me more wrong with respect to the score I predict for a particular team than other teams they'll have a higher uncertainty.

    Historically this ranges from around 18 to 22 points and for good teams, lower uncertainty is better, but for bad teams higher uncertainty is better.

    I.e. predictably good > unpredictably good > unpredictably bad > predictably bad.
\(X\) and \(Y\) are parameters we could tune, and ended up choosing \(X=22\) and \(Y=44\). They weren't optimised too much, they represent 1 and 2 full home and away seasons in the current era respectively.

I can then use these ratings to estimate two metrics for each match:
  • Margin prediction from the home teams perspective (\( \mu \)).
  • Margin prediction standard deviation (\( \sigma\)).
Team scores are calculated as: \[\text{homeScorePredict} = (\text{offenceHome} + \text{defenceAway} + \text{netHgaHome})\times(5\times\text{accuracyHome}+1) \] \[\text{awayScorePredict} = (\text{offenceAway} + \text{defenceHome} + \text{netHgaAway})\times(5\times\text{accuracyAway}+1) \] Where \(5\times\text{accuracy}+1\) represents the average number of points per scoring shot given a particular accuracy, and hga represents an adjustment for home ground advantage in number of scoring shots. I'll explain how that's calculated later.

Expected margin is then calculated as: \[\mu = \text{homeScorePredict} -\text{awayScorePredict}\] Standard deviation is calculated as: \[ \sigma = \sqrt{\text{uncertaintyHome}^2 + \text{uncertaintyAway}^2 - 2\rho\cdot\text{uncertaintyHome}\cdot\text{uncertaintyAway}} \] Essentially we expect variance in the final margin due to variance in the home teams score, variance in the away teams score and covariance between the home and away teams scores.

Here \(\rho\) represents the correlation between home and away scores from the (at most) 2 years prior to the current season (and is 0 for the first ever season). It turns out this standard deviation is around 35-40 for most games which is close to the historical standard deviation in margins. This provides me with some good confirmation bias ;).

I've plotted \(\rho\) by year below and with it the average total match score (home score + away score). In years of high scoring \(\rho\) tends to be around 0 or even positive. I.e. in these years if a particular team scored a lot, it had little effect on the scoring of their opponent. Or in the years with positive \(\rho\), a team was more likely to kick a higher score if their opponent also kicked a high score, strange!

This is in contrast to the current era where \(\rho\) is quite negative and so one team scoring highly means their opponent is more likely to score less. This all ties in with the discussion on scoring variance and is for another day.
Anyway, my estimate for the home teams win probability is then given by \(\Phi(\frac{\mu}{\sigma})\) where \(\Phi(x)\) is the cumulative normal distribution function. There are more details about why we use this equation and it's connection with Brownian Motion in this blog post.

Home Ground Advantage

Now, to determine home ground advantage we use a proxy of travel distance + historic venue performance to estimate the penalty or bonus each team receives due to venue effects, in terms of scoring shots.

The net travel penalty is calculated as: \[\text{netTravelPenalty} = a \times ( (\text{Home Travel Distance in km})^c - (\text{Away Travel Distance in km})^c )\] where \(a=-0.6, c=0.27\). I found this is a pretty good proxy for most of the home ground advantage effect but doesn't give benefit to teams like North Melbourne and Hawthorn playing in Tasmania. To make sure we give these teams an appropriate treatment I've also considered a dynamic venue performance measure.

The venue performance (\(\text{venuePerformance}\)) is calculated in a few steps:
  1. For the team of interest, what has been the historical effect of venue (call this \(\text{venueEffect}\)) at the venue of interest (over the last (max) 100 games) in terms of scoring shots, including adjustment for travel distance?
    \[\text{venueEffectHome} = \text{AVERAGE}(\text{shotsActualHome} - (\text{offenceHome} + \text{defenceAway} + \text{travelHome}))\]

  2. How many matches has the team played at the venue? Call this \(n\). We cap \(n\) at 100.

  3. Then the venue performance is given by \(\text{venuePerformance} = \text{venueEffect}\times f(n) \) where \(f(n)\) is a weight we apply to the historical venue performance. For few games played we don't place much weight on the historical performance but as more games are played we place more weight on past results.

    \(f(n)\) was chosen so that \(f(n) = 0 \) for \(n\le 3\), \(f(4) \approx 0.03\), \(f(50) \approx 0.31\) and \(f(100) \approx 0.52\). The function is exponential in nature and gets flatter with more games of information.

  4. Net venue performance for the match is then given by \[\text{netVenuePerformance} = \text{venuePerformanceHome} - \text{venuePerformanceAway} \]

Basically we take into account the teams historical performance and adjust our scoring shot expectation in line with this, and with more confidence when we have a larger sample of matches to base the calculation on.

So then the net HGA adjustment for the match is given by: \[\text{netHga} = \text{netTravelPenalty} + \text{netVenuePerformance}\] Then we adjust the home team and away team expected scoring shots according to: \[\text{netHgaHome} = 0.5\times \text{netHga}\] \[\text{netHgaAway} = -0.5\times \text{netHga}\]

Updating Ratings

In terms of updating the team rankings, after each match we calculate the error in scoring shots for each team: \[\begin{align} \text{errorHome} & = \text{shotsActualHome} - \text{shotsPredictHome} \\ & = \text{shotsActualHome} - (\text{offenceHome} + \text{defenceAway} + \text{hgaHome}) \end{align} \] \[\begin{align} \text{errorAway} & = \text{shotsActualAway} - \text{shotsPredictAway} \\ & = \text{shotsActualAway} - (\text{offenceAway} + \text{defenceHome} + \text{hgaAway}) \end{align} \] We then allocate half of this error to the team who scored the shots' offence and half to the other teams' defence: \[\text{errorHomeOffence} = \text{errorAwayDefence} = 0.5\times\text{errorHome}\] \[\text{errorAwayOffence} = \text{errorHomeDefence} = 0.5\times\text{errorAway}\] We then update the offence and defence rankings using the classic ELO method of: \(\text{New} = \text{Old} + k\times\text{Error}\) \[\text{offenceHome_new } = \text{offenceHome_old } + k\times\text{errorHomeOffence}\] \[\text{offenceAway_new } = \text{offenceAway_old } + k\times\text{errorAwayOffence}\] \[\text{defenceHome_new } = \text{defenceHome_old } + k\times\text{errorHomeDefence}\] \[\text{defenceAway_new } = \text{defenceAway_old } + k\times\text{errorAwayDefence}\] To update accuracy we just recalculate \( \frac{\text{goals}}{\text{goals} + \text{behinds}} \) for the latest 22 matches. And to update uncertainty we recalculate the standard deviation in the error in the teams expected score (not shots) over the last 44 matches.

Now the \(k\) factor we use to update the offence and defence ratings changes depending on how much information we have about a team in a given season. This is according to the following equation:
\[k(m) = \frac{k_0}{m^b}\] where \(m\) represents the number of games played (1 after the first match), \(k_0 = 0.30\) and \(b=0.18\). This means \(k\) decays from 0.30 after the teams' first match of the season to about 0.17 at the end of the home and away season.

Starting Values and Interseason Mean Reversion

In terms of starting values, we use an offence and defence of 10 for the very first season, accuracy of 0.4 and uncertainty of 15. We use uncertainty of 15 until there are enough games to calculate a standard deviation and we update accuracy after the teams' first match.

For seasons where there is a new team, we given them an offence, defence and accuracy equal to the worst in the league, and a uncertainty of 15.

At the end of each season we revert the offence and defence measures to the mean by 30%. We leave accuracy and uncertainty untouched as they are based on the teams' previous 22 and 44 matches.

Performance

That's basically it, how does it go you ask?

Well it was used in the Monash tipping comp this year and conveniently it gives me numbers for every variety of competition. It came 10th in the normal comp, 5th in the probabilistic one and 3rd in the gaussian one. The model was tweaked throughout the year and in it's final form it would have done slightly worse than is reported on the Monash site (but a bit better in other years).

The performance of this model by year is up here.

Final Thoughts

Predictions for the next season are up here too and they will be updated weekly following results as teams rankings change.

There are some extra steps in generating predictions for matches in future rounds. Essentially we add extra variance for each team to account for potential changes in their rankings over each match before the future match is played. This acts to bring matches closer to 50:50 the further they are away.

An example of this is the 2 matches between Richmond and Carlton. As of the start of season 2019 I have Richmond at 96% to beat Carlton in Round 1 but only 87% in Round 21, while my margin prediction is 47 points for each game.

This will be explained more in a future post.

Also to come:
  • Comparing teams over time, who does my model rank as the GOAT?
  • Ranking dashboard to explore team rankings by year.

Comments