Dellow's been doing a lot of work going through individual shifts of the Oilers, particularly before won and lost faceoffs in each zone, as a way of diagnosing why the Oilers' possession #s collapsed last year. One thing he noted (on twitter) was the following:
Nashville were the NHL's best team when they lost an NZ draw this year. Not a lot of places to make a play here: pic.twitter.com/x54Wim3XY0Alas, these tweets don't seem to be linkable in any active form, but for now google's cache still contains them here (http://webcache.googleusercontent.com/search?q=cache:CNZ02Yls2ogJ:https://twitter.com/mc79hockey/statuses/359712573161103360+&cd=7&hl=en&ct=clnk&gl=us&client=firefox-a).
Weird thing is that NSH was bad at this for years then good this year. Still better to win an NZ draw.
Dellow followed this up with a post about this topic - Nashville's success on neutral zone faceoff losses - and on the value of video analysis as an accompaniment to statistical analysis. To quote Dellow, so I'm not making any misrepresentations here:
I’m fooling around with some data, which I don’t propose to discuss in any detail in this post. I want to make a point about something though: the intersection of data and video when it comes to understanding how things go.LINK: http://www.mc79hockey.com/?p=6238
Nashville was excellent last year after they lost a neutral zone faceoff. Edmonton struggled when they won one. Simple thing to do? Look at them and see what’s going on. I pulled a collection of Nashville neutral zone faceoff losses against Edmonton to examine and it’s kind of amazing how easy it is for even someone like me, a non-expert in technical hockey matters to see what sort of a scheme the Preds are running.
[VIDEO]
...
Also of note: you can see what an advantage having the faceoff just outside the Predator blue is, as the puck can just be dumped in before the Preds can get their structure set. I love seeing stuff like this – for all the talk about systems and technical stuff in hockey, I’ve found it pretty easy to understand what a team is doing whenever you see a bunch of clips of a given situation lined up like this. The difficulty lies in the fact that you don’t get to see twenty identical situations played back to back in a game, which makes it harder to spot this stuff.
Now, here's the issue I have here. Dellow is seeing a statistical result: that Nashville is good when they lose Neutral Zone faceoffs. He's then going to video and seeing what they are doing. He's next IMPLYING that this strategy/system is the cause of the results. The problem here is that assuming causation here simply...doesn't work. In order to show causation we'd first need to see whether the result (Good at NZ Faceoff Losses) is a consistent result in the first place - which may be questionable given that Nashville didn't have that result in the past mind you. Then we'd have to have some way of comparing the results from Nashville with and without that strategy/system. And even then we'd have issues.
In short, there's nothing making such a video analysis different from the type of analysis talked about by Cam Charron in a post today:
It’s tough to blame any particular Bruin on that play. Nobody seems in good position but that could just be sheer fatigue. On some goals there’s a player on the team that was scored on that makes some grave error, one repeated maybe a dozen times by various players throughout the game but only noticed on an instance where it pops up as being evidenced in a replay on a goal against.
I’ve been a little curious as to “analysis by replay” and have had thoughts in the past to record which scoring chances for and against I tracked were shown again on replays. A goal or a not, a bad outlet pass that results in a two-on-one against is a bad outlet pass that results on a two-on-one against, and the more fans get to see a particular player’s mistakes, the more likely they are to be convinced that the defenceman is mistake-prone.
The beauty of hockey lies in its randomness, that marvellous things happen that we have no way of expecting. It’s the sort of pleasure you derive from the game when you actually watch it, and something you’re attempting to match when you’re catching up on junior camps throughout the summer because you miss the distinctive smell of hockey sweat mixed with artificial ice. But those hours of hanging around rinks aren’t going to make you any better at understanding the game. Non-hockey concepts can have value in hockey, and it’s worthwhile to occasionally step away from the sport, and rather than focusing on a specific random event, learn to modify our expectations for the improbable and unlikely by determining what’s random and what’s talent. That goes not only for hockey, but also life.
Again, even before we can test causation, we need to test that the result is real and not simply randomness. That's not done here. And yet the implication is made.
Now Dellow knows this is a problem - so he's denied this is the implication he's making on twitter. Fine, if you say so. But here's the issue with this - simply knowing the strategy/system a team is playing isn't particularly useful for analysis if we don't know the consistent results.
Think about it - we have results - Nashville good at neutral zone D faceoffs. We have what they did in those situations (Assuming for now this was the consistent strategy all year for Nashville). What does the video therefore tell us from an analytical standpoint? The answer is basically nothing - because as presented, you can't show from the video that certain systems/strategies/plays are what are causing the results.
I mean it's nice to know how a game will play out, and what the opposing styles of each team are is an interesting thing for a viewer to know. But from the perspective of analytics, it doesn't help us to simply know "Team X runs System A" or "Team Y runs System B" if we don't know what the results of those systems are going to be. And knowing the results of a sample doesn't necessarily tell us that "system" used in those samples has accomplished (or failed to accomplish)
---------------------------------------------------------------------------------------------------------
To explain in a better way, there is a clear place for video analysis as an addition to statistical analysis. If you can show that one team is consistently good at some form of play, then go to replay and see how they're executing it, and show that there is a pattern showing how certain execution of a system results in that great performance, well then you've made a case for one way for other teams to improve or what may be a better system than what other teams are running.
But arguably, this is the backwards method of analysis. You have a result and you're searching for an explanation, which means you're predisposed to find something in the video that may or may not actually be there. That's what Cam is talking about in the post above.
A better form of analysis would be to see what type of system teams are running and then try and find the results of those systems to find the benefits and cons of those systems. By not having a predisposed finding you don't have a bias that colors what you're seeing.
Mind you, this is difficult to do because it's unlikely that we're really going to be going into any video analysis blind.
------------------------------------------------------------------------------------------------------
Again, video analysis can be cool. I enjoy reading Justin Bourne's system analysis column - even outside the humorous text (which is hilarious btw) and you can easily learn about different types of systems teams use by doing such analyses.
But for actual analysis of performance - for finding out how teams can improve and how players fit in, this type of analysis is heavily limited unless you take certain steps, steps which Dellow in his post is failing to mention. And well, we're concerned with analysis, are we not?
At least I am.
No comments:
Post a Comment