Is The EPL More Unpredictable Than Other Leagues? An (Early) Comparison of Models

Parental Discretion Advised: Over-generalizations from incredibly small sample sizes to follow.

Those of you who follow me on Twitter (which I assume is almost all of my readers, but if not you should follow me @Soccermetric) probably know I debuted prediction models for Serie A and the Bundesliga this weekend. They’re based on the TAM model (which needs a better nickname – I want to backronym NAGBE if possible), which is described in full here. The short version is it only looks at basically two variables: in-season results and goal differential. The math is a lot more complicated than that, but this is the basic idea. The TAM model hasn’t done especially well through its first two weeks in the EPL, predicting 8/20 matches correctly, a meager 40%, or barely better than the 33% we’d expect from just flipping a three-sided coin.

However, it did much better in Serie A and the Bundesliga this weekend. It predicted 6/9 correct this week in the Bundesliga, or 67%. Here are the first week’s predictions:

Week 24 (19) Bundesliga Predictions

The errors were Wolfsburg, Hamburg, and Hertha Berlin. I’m incredibly pleased with 6/9, and am equally pleased with the 6/10 in Serie A (especially because it failed to predict Milan’s win in the Derby).

Week 24 (22) Serie A TAM

In one week, the two new models each got 6 outcomes correct, which is almost as many as the same EPL model got correct in two weeks (MOTSON outperforms the simple model). This is obviously a small sample, and “correct” outcomes isn’t the right way to measure this, but it’s evidence that maybe the EPL is just really difficult to predict (especially this season). I’m not planning on putting together a model for La Liga, but that one might be even easier: you’d be hard-pressed to put together a bad model as long as you started with Barcelona > Real Madrid = Atletico.

The success of a model depends on the difficulty of the task, and if the same model performs much better in other leagues, then we may have evidence of a greater challenge in predicting EPL results rather than the other major European leagues and should adjust expectations of our models accordingly.






Week 23 EPL Model Comparisons (Plus Bonus Serie A Diagnostics)

Last week I posted my first set of comparisons between MOTSON (my predictive model calculated with pre-season data) and my TAM model (the model calculated using in-season results). I wanted to follow it up every week, so here are this week’s predictions and results:

GameMOTSONTAMActual
Norwich v. LiverpoolLiverpoolDrawLiverpool
Manchester United v. SouthamptonManchester UnitedManchester UnitedSouthampton
Leicester City v. Stoke CityLeicester CityLeicester CityLeicester City
Watford v. NewcastleNewcastle UnitedWatfordWatford
Crystal Palace v. Tottenham HotspurPalace/Draw (even)Tottenham HotspurTottenham Hotspur
Sunderland v. BournemouthDrawBournemouthDraw
West Brom v. Aston VillaDrawWest BromDraw
West Ham United v. Manchester CityDrawDrawDraw
Everton v. Swansea CityEvertonEvertonSwansea
Arsenal v. ChelseaArsenalArsenalChelsea

MOTSON got 5/10 games correct, predicting correct outcomes in the following games:1

  • Norwich v. Liverpool
  • Leicester v. Stoke
  • Sunderland v. Bournemouth
  • West Brom v. Villa
  • West Ham v. Man City

The TAM model only got 3/10 correct, predicting correct outcomes in:

  • Leicester v. Stoke
  • Watford v. Newcastle
  • Palace v. Spurs

I’m less than thrilled with 3/10, which is less than the 5/10 last week. Overall MOTSON is winning the prediction competition with 11 correct picks to 8 for the in-season form model. Two of the three TAM correct picks were ones MOTSON got wrong: Watford v. Newcastle and Palace v. Spurs. MOTSON has consistently underestimated Watford, so a home win from them isn’t entirely surprising. I’m not sure why MOTSON didn’t like Spurs more away against Palace – away fixtures are generally tough, but even based on last year’s form I would have thought Spurs would have fared better (although as I remember it their coefficient wasn’t *that* much above the mid-table pack at home last year so that may be what it’s thinking).

Quick note on the TAM’s Serie A predictions – it fared much better here, picking 5/10 correct. The 5 correct predictions were:

  • Juventus over Roma
  • Napoli over Sampdoria
  • Empoli drew Milan (bah)
  • Verona drew Genoa
  • Lazio over Chievo

The TAM is about as simple of an in-season model as one can build, but it’s interesting to see its comparisons vs. a model that knows nothing about 2015-2016, and how the (more sophisticated) pre-season model seems to be doing better so far.

  1. A quick note: Adding up the predicted probabilities for the most likely category had MOTSON getting 5.6 games correct this weekend, so predictions were pretty consistent with results.

Model Comparison: Pre-Season Predictions vs. In-Season Results Only

Warning: over-reading into a very small sample size ahead. I plan on re-visiting this topic over the coming weeks, but figured there was no reason not to start some quick discussion now. 

Regular readers will know that I calculated my predictions based on last season’s data, and have consciously not updated the predictions since then because I wanted to let it run for a season and see how it works. But as of a couple of weeks ago I decided I had enough data to at least start doing a predictive model using current season’s results, which you can read about here. I thought it was important to test the two models against each other, trying to learn about the strengths and weaknesses of each of them. And like everything I’ve done, I think doing it publicly is important. Here are MOTSON’s predictions for this week.1

Week 22 Predictions

Using the quick diagnostic method of “Did the category with the highest predicted percentage happen?” MOTSON predicted 5/9 games correctly: Arsenal v. Stoke ended in a draw, Man City beat Crystal Palace, Aston Villa drew against Leicester, Spurs beat Sunderland, and Southampton beat West Brom. This is actually pretty solid in a week where there weren’t a lot of overwhelming favorites (and one of them was Chelsea…..). Now let’s see how the TAM model performed.

Week 22 TAM

A quick note on reading this image because I think it’s a little less intuitive: the zones are based on how difficult the away fixture is, and the circle is how strong the home team is. If the circle is in the red area, it means the model predicts a home win. If it’s in the grey zone, that means it predicts a draw, and if it’s in the blue zone that means it predicts an away win. The probabilities are a little more complicated, but the quick explanation is that the deeper the circle is into the red zone the more likely a home win is while the deeper it is in the blue zone the more likely an away win is.

The TAM model also predicted 5/9 correct: Chelsea v. Everton, Stoke v. Arsenal, Man City v. Crystal Palace, Liverpool v. Man United, and Spurs v. Sunderland. The overlap between the two is Stoke v. Arsenal, City v. Palace, and Spurs v. Sunderland. The differences were that MOTSON got Villa v. Leicester and Southampton v. West Brom right, the TAM got Chelsea v. Everton and Liverpool v. United right.

Quick diagnostics: the TAM obviously has a better handle on how (not) good Chelsea is this season, unsurprising given the model inputs. Liverpool v. United isn’t much of a difference given that MOTSON had a 35% likelihood of a draw, only a few points lower than Liverpool’s 40% to win. Not a big miss, so I don’t think there’s a big advantage there.

MOTSON correctly predicting how strong Aston Villa would be at home against Leicester City was potentially a real coup. The table has Leicester as a huge favorite over Aston Villa, but the 1-1 draw *may* have even been a little generous to Leicester. That being said, I don’t want to read too much into a single result that may have been a fluke. Southampton v. West Brom is another tough one: I still think of Southampton as a good team, as does MOTSON but their form hasn’t really matched that. TAM recognizes this, having them drawing at home against West Brom, but MOTSON still thinks they’re a fairly good team. Southampton has started to come back up to expectations, only 5 points under MOTSON’s predictions at this point, so it may have a better appreciation for their quality than the TAM does.

Like I said in the beginning, these were just some quick thoughts on the two models. I think there’s a more important question here too: do underlying statistics predict better than simple wins/losses? I don’t include “recent form” in my model because it actually hurt the model’s predictions in training data last season. Beyond modeling, I think humans dramatically overestimate the value of recent form in predictions, and it’s nice to test that empirically.

It’s obviously a simple model, but I think it’s also important to think about what models using current season data add if they don’t predict any better than the pre-season predictions then what do we get from the ones that rely so heavily on in-season performance? If the pre-season models work, maybe they’re all we need and in-season is overvalued. These are all empirical questions that deserve a more systematic exploration, and I’m hoping to do some of that here. I’m going to keep looking at these things over the coming weeks, but this was just a start of the process. Plenty of soccer left to be played, plenty of analyses to do.

 

 

  1. A note: I’m writing this on Sunday night before the Swansea v. Watford game so we don’t have a result there yet. When I refer to the denominator for this week being 9 games, this is why.

Leicester City Looks Good for the Champions League: How Many Points Will It Take For 4th Place?

After the mid-week fixtures, I tweeted the following:

These optimistic forecasts rely on a set of probabilities that, despite doing fairly well, have still managed to underrate Leicester City’s quality so far this season. With their top 4 result seeming fairly secure, I had originally planned on writing a piece talking about this, but Mike Goodman beat me to it this morning. It’s a typically strong effort from him that you all should read, going through the points per game Leicester City would need to secure a Champions League spot given historical norms for 4th place. However, as we’ve seen, this season doesn’t fit any sort of historical patterns. So how does Leicester City need to do for the rest of the season if they want to finish in 4th place (or higher)?

To answer this, I took MOTSON’s 10,000 simulated seasons and removed Leicester City from each of the final tables. Then I sorted the table by number of points, and this gave me what the final table would look like without Leicester City. From this, I’m able to see what the “4th place” team (if Leicester City never existed) looks like, and determined that to earn a spot in the Champions League, Leicester City would need to earn this amount of points +1.  I show these point totals in the figure below.


Jan 15 Blog 1

The minimum number of points needed this season is likely going to be considerably lower than normal: probably somewhere around 59-62 points. It’s a weird year, with the expected points for the title being around 80, so it’s somewhat unsurprising that the number for 4th place is quite a bit lower than normal.

To replicate Mike’s analysis, and to determine the number of points per game “The Fighting Lesters”TM would need, I subtracted Leicester City’s current point total (43) from the number in the previous table. This tells me how many points Leicester City needs to come in 4th, and if I divide it by the number of remaining games (17), I get the points per game (PPG). Below is the CDF (cumulative distribution function) of how likely Leicester City is to qualify for the Champions League given a certain number of PPG.

Jan 15 Blog 2

Even if they only manage 1 PPG (approximately the number usually associated with barely avoiding relegation), they’re still about 50% to qualify for the Champions League. If they maintain anywhere near the ~2.1 PPG they’ve averaged so far (even slipping as low as 1.5), they’re basically guaranteed to finish in 4th place or better.

For my final analysis, I look at the previous analyses but do it for 3rd place (the first spot that directly qualifies into the UCL without a qualifier). I follow the same procedure as I did in the 4th place analysis, but instead I look at what it would take to finish 3rd instead of 4th. Here are the expected points needed for 3rd.

Jan 15 Blog 3

They’ll have to do better here, expected to need around 65 points to finish 3rd, which means 22 points out of their last 17 games.  This is still below their current pace, but allows for some regression in form. Finally, I present the CDF for 3rd place.

Jan 15 Blog 4

All Leicester needs to have a 50% chance of finishing third is 1.3 PPG, which is reasonable, and if they can keep up a 1.6 PPG pace they’ll be a virtual lock for 3rd and guaranteed qualification for the Champions League.

Even if Leicester City slips in the second half of the season, a simple fact remains: in such a low points season, they quite frankly don’t even have to be that good over the last 17 games to qualify for the Champions League.  I don’t want to jinx it, but if I’m a Leicester City fan I start getting ready to book my travel on Wednesday nights in the fall.






Regardless of Method, Prediction Models Basically Agree

A number of different blog posts comparing some of the different prediction models have been going around lately. If you haven’t seen them, you should check out Alexej Behnisch’s piece comparing various models (the post where I got data for this piece), and Constantinos Chappas’s work at StatsBomb doing a “poll of models.”

In his post, Alexej pointed out  how many similarities there were between the different model predictions, and highlighted some of the major differences in their predictions. However, something I’ve noticed in the past is that the middle of the table is basically one big tie, so predicting Southampton for 10th vs. 13th doesn’t necessarily mean there’s much of an actual difference in the models.my most recent heat map of probabilities might help illustrate what I mean:

Week 20-3 Heat Map

You can look at the brightness of the colors and almost see the clear boxes. The table has basically separated itself into 4 tiers: the top 2, the next 5 (contenders for Europe), the next 7 (mid-table malaise), the next 4 (partly safe with a chance of relegation), and the bottom 2 (getting ready for life in the Championship). One of the things Alexej’s article points out is differences in 3rd-5th place predictions, but if you look at the probabilities in my model they’re roughly tied. The mid-week results could easily see those three completely switch and switch again after the weekend’s fixtures. 1

So the question I’m interesting in is how similar the various models’ predictions are? In statistical terms, how well do the different models’ predicted points values correlate with each other? The answer is: incredibly highly. In fact, they correlate so highly I checked and re-checked my analysis against the original data several times because I didn’t believe it. Here’s a plot of the data to show you how highly they correlate.

Correlation Major Models

To see the correlation between two models, lineup the name on the bottom row and the name on the left side.  The lower diagonal shows a scatterplot of the two models specified on the bottom and left sides, and all of the scatterplots show basically a 1:1 relationship with almost no variance from that line. This indicates a high correlation, and the number for each pair is specified in the upper diagonal. The lowest correlation is between my model (Soccermetric) and Michael Caley’s (MC_of_A), and even that is 0.978 which is incredibly high. All of the major models here basically have the same predicted values for end of season points.

With so little variation, there’s not much else to be squeezed out of these data, but I wanted to present one more just because it’s a question that has been on my mind. There  are basically two types of models out there – the pre-season prediction models, and the in-season models.2 They use dramatically different data, so I’ve always been curious to see how the two types of models match up.  I highlighted the cells with models that “match” (both are either in-season or both are pre-season) in blue, while the cells with mis-matched models (one is pre-season, one is in-season) in red. The results are below.

View post on imgur.com

There’s certainly no statistically distinguishable relationship between the two groups, but mis-matched models (red cells) tend to have a little lower correlation than the matches (blue cells), but we’re looking at a comparison of 0.98 to 0.99 here so I don’t think it’s worth drawing any conclusions from. The similarity between all the models is striking to me, and at least by week 20 the in-season models seem to have roughly the same predictions as the pre-season models. This may be my bias as a pre-season modeler, but that speaks highly of the pre-season models in my mind.

The moral of the story is that despite using such different inputs, these models have roughly the same predictions, correlating at 0.97 or better. More importantly, they correlate with market expectations at a similar level, which either means stats conform to the wisdom of crowds or that gamblers are listening to our models and betting accordingly. At the end of the day, look at the model inputs, pick the model you’re most comfortable with, but it looks like whichever one you pick you’ll see roughly the same outcomes. It speaks well about the type of work that’s being done online, and it’s an exciting time to be following soccer analytics.





  1. This isn’t to criticize the piece – Alexej’s article points out an important point that there can be a huge difference in where a team finishes within these groups, and nowhere is this more true than the 3-5th place spots.
  2. I didn’t know how to classify the EuroClubIndex because it uses both historical and current season data so I omitted it from this analysis. I considered market data as in-season data because they update every week with new information.

Game Theory: Team Selection and The Magic of Properly Tanking the FA Cup

In previous posts, I’ve written why it’s rational for teams to field a sub-optimal lineup in the Champions League, despite the fact that it leads to a tragedy of the commons scenario. The short version is that the expected value of resting your top players in cup games and saving them for league fixtures is higher than playing them and trying to win the cup and meet whatever goals you have in the league. The expected value calculations are a little different between the UCL and the FA Cup, but the logic is the same. However, Arsenal’s strategy today made me think that there is one added strategic wrinkle in FA Cup roster selection, and one that Arsene Wenger may not have taken into consideration: the need to avoid a replay.

Arsenal’s lineup yesterday against Sunderland was: Cech, Bellerin, Gabriel, Koscielny, Gibbs, Oxlade-Chamberlain, Chambers, Iwobi, Walcott, Campbell, Giroud.

Wenger clearly chose not to tank the game, placing some value on a possible third straight FA Cup win. This is a strong lineup, with Walcott, Campbell, and Giroud up front, and Oxlade-Chamberlain, Koscielny, Bellerin, and Cech also in Arsenal’s strongest starting XI. However, he didn’t field his strongest lineup, choosing the rest some key players as well.  Notable omissions were Per Mertesacker, Mesut Ozil, and Nacho Monreal. Resting the player who is likely their top defender and their #10  who is one of the best playmakers in the league this season shows that he wasn’t overly concerned about winning either, as these players likely would have featured in a league game. Wenger chose not to tank, but also didn’t necessarily play to win, choosing a “third way” instead.

I’m going to start with a big assumption here: the maximum EV preferences are as follows.

  1. Loss
  2. Win
  3. Draw

Reasonable people can disagree on numbers 1 and 2 – maybe Arsenal gets more value out of winning a third round FA Cup match than I think they do, so maybe they really want to win. However, I think the draw, leading to the replay at The Stadium of Light, was Arsenal’s worst possible option here. The goal then would be to maximize the combined probability of winning and losing while minimizing the probability of a tie. At its extreme, this might look like a 4-2-4 with four defenders, two midfielders, and four strikers: a line-up that will score a lot of goals and concede a lot of goals.1 The odds of this lineup playing to a 0-0 or 1-1 draw are remote.

Wenger did something different: he played his top (available) attackers so maybe he planned on scoring a lot of goals, but he rested the team’s best provider which would potentially limit the number of opportunities they had. He also gave a 19 year old midfielder his first start in midfield, which is good for a lot of reasons but combined with resting Ozil really weakens Arsenal’s midfield.  Wenger didn’t play to win, but he didn’t choose a lineup of youngsters who would likely lose either. He risked a replay, which may have been the worst possible outcome for a team, stretched thin by injuries while trying to mount a title challenge.

Arsenal aside, the FA Cup adds an extra dimension into roster selection strategy that the Champions League doesn’t have. Avoiding a replay should be a priority for any team who hasn’t secured their league goals for the year, which means picking a team that is either going to win or lose, and will do either in convincing fashion. Adding another fixture to an already crowded list, is something all Premier League teams should want to avoid, and should select their teams accordingly.

 






  1. Like I said, this is extreme. I don’t actually think this would be a valid formation to choose, I only use it as an absurd example to illustrate the idea.

Is Petr Cech Worth 15 Points? A rough, back of the envelope calculation

A note before I start this post: all of these calculations are really rough, back of the envelope sorts of calculations. The method is sound, but there are some issues with data that I will identify in italics throughout. The end results don’t change the answer that Cech is likely not worth 15 points as John Terry now famously said, but this sets up what I think is a decent first cut at assigning an actual point value to goalkeepers. Hopefully others can fix the bad assumptions to tighten up the actual values a little.

John Terry famously said that Petr Cech would be worth 12-15 points for Arsenal, and we’ve seen several variations of this theme in the media with various goalkeepers (Lloris, De Gea, etc.) but no one really knows how to evaluate the actual value of a keeper to a team.1 Using data from Paul Riley (@footballfactman on Twitter) and my recent analysis of Michael Caley’s xG data against MOTSON’s predictions I think we can at least make an first cut here that makes sense and comes up with what I think are reasonable results.

Step #1: Regress MOTSON’s Expected Points against xGD

In my previous post, I showed that MOTSON predicts expected goals very well. So I did a quick, bivariate regression with Expected Points as my DV and xGD 2 as my IV. From this I get a model of:

ExpPoints = 26.73 +0.6723(xGD)

What this means is (as of today), each team starts with a baseline of 26.73 points, and then for every additional xGD you add 0.6723 to that value. Simple arithmetic can calculate each team’s expected points from here.

Quick diagnostics: The predictions from this model correlate with my actual expected points at 0.83, and correlate with teams’ actual earned points at 0.77. These are very high numbers, certainly high enough to continue with the analysis.

So each xGD is worth 0.6723 points in the league table. Next, I look at Paul Riley’s data showing all EPL keepers with more than 350 saves in the last 5 1/2 seasons. The data show Expected Goals Allowed (ExpGA) vs. Actual Goals Allowed, so I use this to calculate a goal differential for each goal keeper over the 5 1/2 time period, multiply this number by the regression coefficient calculated earlier ( 0.6723), and then divide by 5.5 to calculate a “per season” score.

There are two assumptions implicit here: first is that each goalkeeper played a full 5 1/2 seasons in the EPL, which is a bad assumption. I didn’t want to look at each keeper’s history for what is supposed to be a quick blog post, but if you did you could easily just change the last number to whatever the number of seasons someone played is.

The second is that goalkeepers are 100% responsible for the difference between ExpGA vs. Actual GA, and have 0% responsibility for scoring goals. This seems relatively reasonable, or at the very least the error terms are random between all the keepers in this model. There might be great counter-attack starting goalkeepers in the world like Manuel Neuer who deserve some credit for goals scored, but even in that cause I’d imagine the number of small, at most. 

Below is the a graph of all the goalkeepers and their per season point values, and the table with the raw data is located at the end of this postGoalkeepers. The top goalkeeper in the model is Adrian, and according to this method he’s worth about 2.4 points per year over a “neutral” goalkeeper, and a little more than 5 points per year over the lowest scoring keeper in the model. Petr Cech comes in 4th at 2.05 points per year, which is really good for a goalkeeper, but is well short of the 15 points John Terry asserted.

Like I said, this is a first cut at Expected Points for goalkeepers, but presents a way of quantifying their major contribution. Future work on the topic needs to look at error rates around xG calculations, sample size issues, and some other measurement issues with the model inputs, but overall I think it’s a really good first cut. Big thanks to Paul Riley for posting his data publicly and making this post possible and I’m looking forward to seeing how people can build on this.

NameExpected Goals AllowedActual Goals AllowedGoal DifferencePoints Over 5.5 Seasons
De Gea168.0914919.092.33
Hart187.0117017.012.08
Cech166.815016.82.05
Adrian108.638919.632.4
Begovic207.1419710.141.24
Foster225.8421114.841.18
Vorm128.261199.261.13
Schwarzer144.771395.770.70
Mignolet226.632215.630.69
Reina111.181074.180.51
Jaaskelainen169.611672.610.32
Lloris136.71360.70.09
Howard217.39219-1.61-0.20
Szczesny147.01150-2.99-0.37
Friedel112.88116-3.12-0.38
Hennessey119124-5-0.61
Al Habsi158.87165-6.13-0.75
Green146.87153-6.13-0.75
Ruddy161.94171-9.06-1.11
Guzan192.78207-14.22-1.74
Krul241.88264-22.12-2.70

 

  1. Goalimpact assigns values to keepers,  but they’re notoriously even more difficult to quantify than defenders.
  2. Expected Goal Difference

MOTSON Predicts The EPL Table Well, But Predicts xG Better

I’ve been focusing on diagnostics with MOTSON lately, but one thing I haven’t thought about is comparing the model’s predictions to some of the underlying statistics measures. The model has done well, but how much of the prediction error comes from, for lack of a better word, “error”, and how much of it comes from random variation. So I wanted to test this with the gold standard in soccer analytics: xG.

As a refresher, my most recent model’s expected points correlates with actual points at 0.61, which is pretty solid but not spectacular. Chelsea and Leicester City are pretty big swings and misses for the model right now, dragging the correlation down from about 0.8, which would be solid and spectacular.  Here’s the scatterplot of model fit with a regression line of b = 1.0 thrown in for good measure.

xg Predicted v Actual Points (xg is for ordering purposes)

I would imagine all my readers know this, but for those who don’t the quick version of xG is it’s a measure of shot quality. How many goals would you expect a team to have scored given the quality and number of their shots? But we know there isn’t a perfect correlation between the two measures, and variance causes teams either score more or less goals than the xG measure would predict.

I downloaded xG data from Michael Caley’s fancy stats site (@MC_of_A on Twitter), which included both expected goals for and against for each team in the EPL. I merged in my “expected points” data and the actual points each EPL team has earned, and created a new variable with the difference between each team’s xG and the actual number of non-penalty goals (NPG). This is what I used for my analysis.

Question #1: Is MOTSON successful in predicting the underlying stats? Specifically, how well do MOTSON’s predictions match up with xG?

I correlated the xGD1 measure with my expected points. We know Goal Difference is a great predictor of table position (correlating at 0.92 as of this post), so I want to see if MOTSON’s expected points correlate with Expected Goals. I ran a quick bivariate correlation, and got a Pearson’s r value of 0.83, very high by any accounts.

xGD v MOTSON Residuals

MOTSON passes this test, showing a high correlation between expected points and expected goal difference.

Question #2: How much of MOTSON’s error is explained by variation from underlying statistics? Specifically, do xG and MOTSON share the same misses?

For this, I correlated the xG residual measure (calculated as NPGD – xGD) with MOTSON’s residuals (calculated as Expected Points – Actual Points).2 If xG and MOTSON’s residuals are highly correlated, it could be evidence that the teams MOTSON over/underpredicts are over/underperforming compared to expectations and the underlying statistics the soccer analytics community looks at. The overall correlation here was almost as strong, with a Pearson’s r of 0.74.

xGD v MOTSON Residuals

MOTSON’s biggest outliers this season are Chelsea, Leicester City, and Aston Villa. The first two would have been almost impossible to pick pre-season, but do models using current season data do better on them? It turns out those three are fairly problematic for xG models as well. MOTSON over-estimates Chelsea’s points by 16, but their xGD is actually positive at 4.8. This xGD score compared to their NPGD of -9 shows that even using in-season statistics has trouble with Chelsea’s performance this year. Similarly, Aston Villa underperforms for MOTSON by 10.71 points, and underperforms xG by 7.4 goals. Leicester fares better using in season stats, overperforming by 13.28 points for MOTSON but only off by 3.5 xG.

MOTSON has done well for me this season, but it’s interesting that it does a better job predicting xG than it does actual points. This makes me more confident that the model is onto something, and figured a lot of it out in the pre-season without any current season data.

  1. Expected Goal Difference, or Expected Goals Scored – Expected Goals Allowed
  2. This isn’t the right statistical use of the word residual, but it sounds nicer than error so I went with it.

Why Rory Smith Hates Numbers Wizards, or How Can We Quantify Fun?

Let me preface this post by saying that I understand this type of question is exactly why pundits mock analytics. The idea of assigning a number to how fun a match was to watch seems silly and to be quite the opposite of fun. But I’m writing it anyway because I have a very different definition of fun than normal people do, or because I think this actually serves a real purpose that real people actually care about.

Imagine you’re in charge of sports programming at a major TV network, and your network just bought the rights to a new foreign soccer league or competition. For bonus points, imagine this is in a country without a big history with the sport (like America), and it’s your job to develop this contract into a major money maker for the network. How are you going to do that?

NBC does a great job with soccer here in America: they broadcast all EPL games either on one of their networks or on their live streaming app, which is far better than any other sport in America and from what I understand is better than what my English followers get.  If you want to watch a game, you can do it. But my only issue with their coverage 1 is the over-focus on a few teams. If Chelsea or Manchester United are playing, you can guarantee NBC will broadcast their games regardless of what else is going on, or if there are better games to watch. But there has to be a better way!2

So that leads me to the research design I’m going to propose here: the correlates of fun. Today’s Everton v. Tottenham Hotspur game was a lot of fun, especially the last 15-20 minutes or so where it was a seemingly never-ending series of counterattacks leading to goal-scoring opportunities. Even the commentators were visibly disappointed when it ended, hoping for just one more minute. The only reason this game aired on TV was because it had no competition in the 12:30 (Eastern Time) slot, and we could have easily missed it. Could NBC have predicted this, or more importantly, could an appropriately air-conditioned analyst have predicted it and advised executives to show this game over some other choices if it had competition?

I’m going to propose a series of measures that I know are available to folks with premium data subscriptions that I think correlate with what people think a “good” game is. This might not be exhaustive, and I’m open to disagreement, but I mean this more as a research design than a finalized answer to the question.

  1. Goals
    • More goals isn’t always a good thing, but on average a 2-1 game is going to be more interesting than a 0-0 draw.
  2. A competitive game
    • A close final score is more compelling because any moment could change the outcome of the game, and because it correlates with any number of other things that are interesting (a team with a 3-0 lead is probably more interested in killing off the game than scoring a 4th goal). Plus there’s the dramatic element of not knowing what the outcome will be.
  3. Fast average player speed
    • I saw this on the Valencia/Madrid game – they had real-time data on how fast each player ran during the game. A series of sprints probably means the end-to-end type of action that people like rather than aimless passing around the midfield.
  4. Large average distances run by players
    • This may be less direct than the previous one, but it seems like if players are covering more distance that means the ball is moving around more, which probably means there’s more action.
  5. Low numbers of fouls
    • Fouls can act as a heat check on a good game, slowing it down, ending interesting passages of play. Even free kicks from dangerous areas are unlikely to go in, but take seemingly forever to set up before someone kicks it over the goal and into the 20th row.

All of these things can be modeled/predicted ahead of time (there’s no reason one couldn’t apply @DoctorFootie’s method to these questions), and then use the model to pick which game is featured on television so fans have the best chance of an optimal viewing experience, increasing the likelihood of fans returning next week for another exciting game.

A second, probably more scientific (but more difficult) option would be to do a survey. There are a couple of options here. The first is that you could sit people down in a room, have them watch a game, and ask them to watch a game and then rate the game on an entertainment scale of 1-10. Then you mine stats from the games, correlate them with ratings, and figure out what people are looking for. In a big enough sample, you should find some key aspects of games that people find entertaining and use that to predict which game will be the best.

The second option along these lines would be like political focus groups do and dial-test things. Give people the dial where they turn it to the right if they see something they enjoy, left if they see something they don’t enjoy. That would increase your effective sample size because you’d have a large number of events in a given game rather than a single game as your level of analysis, and would also give you more precision in understanding exactly what sorts of things people like in games. More granular data is better in this case, and they already do this for new shows so why not for soccer? Then you’d find the events people enjoyed in individual games, and show games that are likely to have more of those types of events.

I’d like to see more games like Everton v. Spurs today, and fewer games where Man United drags the game to a crawl with a series of sideways and backwards passes and presumably the networks would want to show more games like that too. Better use of data could help networks make better decisions, both helping the on-air product and the league. This may not be as important for the EPL where ratings are high and NBC offers real-time access to all 380 individual games, but I can picture this being even more important for a league like MLS with a smaller fanbase, no real “big clubs”, and a need to improve the perception of the product. Data helps you make better decisions, and anything you can observe can be quantified as long as you look in the right places.

  1. I actually don’t like their “Breakaway” days where they switch from match to match whenever a team makes a good attacking move, but everyone else seems to like it and I’ll just switch to my Roku to pick my favorite game that day
  2. Obviously there are commercial appeals at play here: I know Man United has a huge fanbase here in America, and I’m assuming Chelsea must too because NBC has kept showing their games long after they stopped being competitive for the title. There are also “big game” considerations which I’m going to ignore for the purposes of this post. The Manchester Derby earlier this season was a colossal bore, but I get that it’s a game between two title contenders and a huge rivalry. That sort of thing trumps “fun” soccer for me and I’m willing to accept arguments that these types of big games need to be the main option regardless of how boring they often are, at least in the EPL.

EPL Power Rankings: 2015-2016 Performance Adjusted for Strength of Schedule

I’ve written previously about how MOTSON relies on last season’s results for its predictions, but I wanted to do something with this season’s data now that we’re at the halfway point.

The model is simple: it’s a generalized partial credit model (GPCM), which is basically the same model they use for standardized tests like the SAT or GRE. I’ve written in more detail about the approach in my initial post for my blog, but the basic idea is that I treat each team as a person taking a test, and each home game as a question on the test. If you win the game, you get full credit, if you draw you get partial credit, and if you lose you get no credit. GPCM models are good because they are agnostic as to history, payroll, Big ClubTM status, or any of the other things that confuse regular human brains. All they know are the results that have occurred this season between the teams that have played against each other.

These models also do well with missing data, so in a half-season each team has played approximately half the other teams at home. GPCM models fill in these gaps, adjusting each team’s strength against the difficulty of the fixture.

So based on home results, here is each team’s “Strength” coefficient.

Week 19 Power Rankings

Arsenal is head and shoulders above the rest of the league, dominating at home, well above #2 Leicester City. The odd results here are Crystal Palace, who is far under-performing at home, and Swansea, who is performing at home far above their position in the table. Their positions are basically reversed, which brings me to the second part of the equation: difficulty to beat on the road.

The strength coefficient in the previous graph shows how strong each team has been at home and how good they are at beating teams on the road. As stated earlier, this is the equivalent of the score a test-taker would earn. Now we turn to the difficulty of the question being asked, or the strength of teams on the road.

Week 19 Away Fixture Difficulty

The dark red point in this graph represents the strength coefficient needed for a 50% probability of the home team winning a game against the opposition (answering the question “correctly”). The blue point represents the minimum strength coefficient needed to secure a 50% probability of a draw (earning “partial credit”). Points to the left of the draw zone mean a higher likelihood of losing, points to the right of the draw zone mean a higher likelihood of winning based on results so far this season.

Interestingly, title favorites Arsenal and Manchester City are near the center of this table, seemingly weak on the road. However, when you compare their away difficulty coefficient to the home coefficients, only six teams (excluding themselves) would have a strong chance against them. This feels about right to me – the top 6 teams should have a good chance of beating title contenders at home, but beyond that it should be much more difficult.

On the other side, I’ve been skeptical until now, but Chelsea look to be at least a semi-legitimate relegation contender right now.  They are the 6th weakest home team right now (a far cry from the undefeated season at Stamford Bridge last year), and everyone except for Aston Villa would be favored to take at least a point off of them at home. I knew they weren’t as good as MOTSON says, but this model has them in serious trouble unless they things turn around. Maybe not relegation-level, but bottom 5 wouldn’t be surprising based on the eyeball test here.

Disclaimer: while these coefficients are accurate given results so far, we’re obviously at a very small sample size with a substantial amount of missing data that will be filled in over the next 19 weeks so these numbers could possibly change quite a bit. However, there’s a lot of logic in these numbers, and they match up at least somewhat well with my expected final table as of today. 

Also, one can calculate predicted probabilities for each outcome (and presumably expected points over the season). off of these models. I don’t know that I’m going to do that, but if there’s interest I can probably put it together in the next couple of weeks.