I tweeted about how the soccer analytics community doesn’t seem to spend enough time on diagnostics, so I’m doing an ongoing series of posts comparing my model to one of the more common (and more elegant) models out there: Club ELO. For more info on the measure and what I’m working on, check my opening post on the ELO challenge: http://soccer.chadmurphy.org/?p=173
First, some overall diagnostics: Week 1, the actual outcome was the most likely outcome 6.5/10 times (one had a tie between two choices so I give the SVM half credit there), week 2 I had 4/10 matches. Overall, the model’s better than 50% which is pretty strong in a complex model like a soccer game.
Time for this week’s update, and ELO won week 2 by about the same margin my SVM won week 1. It also won 6/10, while the SVM won 4/10. Last week, this week, and overall are in the graph below.
As you can see, we’re basically tied. The SVM won week 1, ELO won week 2, and overall we’re about even (a 0.26 point game means ELO is, on average, 1.3% better than the SVM). It’s really interesting how two very different measures come to such similar conclusions, both on many of the individual games and in the aggregate.
The SVM had a couple of REALLY bad misses this week: Everton and Leicester City both won in what my model considered significant upsets. I tweeted about my qualitative asssessment that Everton should have been higher probability this week, but Leicester could go either way. The SVM really likes their overall odds for the season (picking them to compete for a Europa League place), but it didn’t like the odds against West Ham for some reason. I’ll do some diagnostics at some point, but this week’s going to be about putting together a new interactive tool and getting Serie A data up (as well as class prep because school starts next week for me), but I definitely want to see why the model missed so badly on Everton and Leicester. My best guess is that the game stats diverged from the actual stats, but I have no way of knowing until I check. So far I’m happy that I’m doing about as well as the dominant model.