Why use a ratings system for the AFL?

The quality of AFL teams generates much debate between pundits and fans alike. Everyone has their view on which teams are better and, consequently, who will win this week and go on to have success in September. This is all very subjective, so a ratings system aims to provide an objective view of team quality and inform predictions for upcoming games. It can help us better understand where each team sits, ignoring things like ladder position which aren’t always indicative of team strength.

How it works - overview

In a typical Elo model each team is assigned a rating, with the league average being held constant at 1500. These ratings are then used to generate an expected result for a given matchup of teams. After the teams play, the expected result is compared to the actual result and the ratings for each team are updated in a zero-sum fashion that maintains the average rating at 1500. Teams that exceed their expected result receive a ratings boost and the opposite occurs for worse than expected performance.

How it works - expected result

The expected result is between 0 and 1 and is generated using the following formula:

\[ \begin{aligned} \frac{1} {1 + 10^\frac{-(home\_rating - away\_rating + HGA)} {400}} \end{aligned} \]

Where \(home\_rating\) is the Elo rating of home team and \(away\_rating\) is the same for the away team. There is also a \(HGA\) parameter (home ground advantage) which applies in certain games. A value of above 0.5 means Elo is predicting a “win” for the home team.

Below is a plot showing how the ratings difference for the home team translate into expected results.

How it works - actual result

We also need an actual result on the interval \([0,1]\), and for this I’ve used team share of scoring shots (goals + behinds). The reason I’ve focused on scoring shots rather than total score is because total score is heavily impacted by accuracy and, despite what some might think, kicking accurately in one game is not predictive of doing the same the following week:

Basically, accuracy above or below league average is mostly luck.

In terms of opting for a result that doesn’t factor in margin, the reason for this is I think margin is flawed in not considering the total score of the game. Winning 140 to 100 means the winning team scored 40% than the loser, but if the scores were 40 and 80 then the winner scored 100% more than the loser. Margin sees these results the same and in my opinion this isn’t capturing the full context of the result - in the 2nd scenario the gap in performance between the teams is much greater. In this example I am using total score not scoring shots, but the point remains the same.

How it works - updating the ratings

After an expected result for a match has been generated the actual result has been recorded we can update the ratings using the following formula:

\[ \begin{aligned} rating\_home_{new} = rating\_home_{old} + K \times (A_H - E_H) \end{aligned} \]

Where \(A_H\) is the actual result for the home team, \(E_H\) is the expected result and \(K\) is a parameter that tells us how much to update each team’s rating after each match.

One thing about this Elo system is it will punish teams for poorer than expected performance, even if they win in terms of generating more scoring shots (of course the opposite is true for teams that lose but outperform expectations). This is because the system rewards teams for performances that are above expected results, and a heavily favoured team barely scraping over the line against a lesser opponent is hardly reason to view them more favourably.

How it works - K value

The first of three parameters that are part of this Elo system is the \(K\) value, which takes a value of ~77. This value indicates how responsive the system should be to match results. Higher \(K\) values will mean the team ratings bounce around as greater weight should be placed on recent performances and vice versa for low \(K\) values. This \(K\) value is fairly typical for a sport like the AFL.

How it works - home ground advantage & regression to the mean

Along with the \(K\) value, there are two other parameters in this Elo model; the first being home ground advantage (\(HGA\)). There are obviously many ways to approach home ground advantage and I have opted for a state-based approach. In this model, a home team will have \(HGA\) applied if they have played 2 more or home games in the state where the game is being played and the visiting team has not played 2 or more home games in that state (with Geelong being considered as separate from Victoria). The reason the 2 or more games filter is twofold. Firstly, it removes home ground advantage in cases where teams host 1 game in a certain state, like Melbourne hosting a game in the NT. Secondly, it means that if teams host 1 game in a state where other teams play, like St Kilda hosting a game in QLD, then if they have to travel back to that state for an away game home ground advantage will be applied.

The other parameter is regression to the mean (\(RTM\)) and this tells us how much the rating of each should regress back to mean of 1500 at the end of each season. This takes a value of ~0.47.

How it works - parameter optimisation

These three parameters, \(K\), \(HGA\) and \(RTM\), all have to be optimised for Elo to do a decent job. Again, I have kept this simple and looked to minimise the mean absolute error between the expected and actual results. In other words, the model is optimised to predict share of scoring shots as well as possible. The data used for this optimisation was from 2012 to 2023 inclusive, with initial ratings of 1500 given to each team for Round 1, 2012. 2020 and 2021 are excluded for \(HGA\) optimisation due to Covid wreaking havoc on the fixtures. 2023 is the only season that includes finals games as I originally created the fixture scripts as regular season only, for a reason I cannot now remember. I have not used more data as my optimisation process is quite slow, and I don’t want to kill my laptop.

That’s it