The Importance of Qualitative Scouting: A Response to the Analytics FC Response

@BobbyGardiner posted a response to my “Sell your overrated striker” post over at the Analytics FC blog, and it’s a good read for anyone interested in the topic. You should read the whole thing (and everything the @AnalyticsFC crew does), but his main point of disagreement with my post was the idea that Stephan el Shaarawy overachieved for Milan back in 2012. Bobby’s argument was that El Shaarawy wasn’t particularly spectacular that year, converting at a rate of 15% and scoring many of his goals from good positions. His shot chart from that season is below:

The alternative hypothesis is that el Shaarawy’s decline in production was based on a series of injuries rather than overachievement in one year and a regression to the mean in others.

I think these sorts of debates are interesting and important, and I appreciate Bobby’s response to my post. One of the things I was taught in graduate school was the highest compliment someone can pay is to engage with your work, and the worst is to ignore it. So I’m posting this response to engage further rather and maybe advance this discussion, not as some sort of attempt to “win.”

To that end, I have a couple of thoughts: one is sort of technical while the other is posting the next step of the argument. The first is that I picked il Faraone because, as a lifelong Milan supporter, his was the first name that came to mind. Milan’s biggest mistakes in recent years have been to sell low: Pato, Zlatan, and el Shaarawy have all been the subject of massive bids only to be sold a year or two later well below their original market value. He’s not necessarily the best case study, and Bobby’s choice of Alexandre Lacazette may very well be a better one. El Shaarawy also isn’t a good choice because I originally was aiming at talking about “selling” clubs, which Milan hasn’t historically been, and Lacazette may be a better option there as well. The case study wasn’t necessarily the point, the argument was that knowing when to sell overachieving players can make clubs a significant profit in the long-run.

More importantly, I think this points out the need for qualitative scouting to complement analytics approaches. Let’s use Lacazette as our example for a minute: he performed spectacularly last year, and Bobby’s argument is that he may very well have over-performed significantly. He also points out, correctly, that we don’t know this just from looking at stats. This is an area where Milan supposedly shined, but evidence seems to show that their methods were imperfect at best.

The vaunted Milan Lab describes itself as “a high tech interdisciplinary scientific research centre.” One of its big claims to fame back in the day was finding older players who could play at a world-class level far later into their careers than any other club, letting them buy 30+ year olds at a bargain price. This was demonstrably false when they let Andrea Pirlo go on a free transfer to rivals Juventus a few years back, and he put together some of the best years of his incredible career for perhaps their biggest rival. It’s other big claim to fame was maximizing training techniques to minimize injuries/to know which players were more susceptible to injuries. Pato proved this false on a regular basis, and more importantly to my point, they should have recognized el Shaarawy’s seemingly injury prone nature. That would be another inefficiency that could be exploited – knowing that el Sha would be likely to see significant injuries would be another way to get more money than his value from Manchester City.

Beyond this, regular qualitative scouting is important. I watched virtually all of Milan’s games that year (back when they were streaming on ESPN3’s app), and while he had an amazing season, el Shaarawy is a winger who was forced to play centrally by a combination of injuries and the sale of main striker Zlatan Ibrahimovic. He had a great season, but he was playing far too centrally far too often in the new Milan lineup. He was also a 20  year old kid being asked to lead the line for one of Europe’s biggest clubs, and there was plenty of talk at the time that this was too much pressure and too many minutes for him. I don’t watch enough of Lacazette to know for sure, but Lyon should be thinking whether he can repeat last year’s achievements or if they should max out now and sell him at his peak.

As with everything, the numbers can lead you where to look. El Shaarawy had an above-average season, just like Lacazette. Their performances were significantly more than one would expect from a young attacker, so a red flag should be going up from the analytics department. Then let the qualitative folks watch their performances: are they likely to improve, sustain, or decline? Is it worth selling now to cash in at the max rather than going through a couple drought years and then selling at a fraction of what you would have made originally? Numbers alone won’t get you there, they can only narrow down the range of possibilities you are expected to look at. Qualitative approaches are the perfect supplement for questions that numbers can’t answer alone.

Leave a Reply

Your email address will not be published. Required fields are marked *