The Current State of Hockey Analytics on Evaluating Defense

Steven Burtch of Pension Plan Puppets has been active on the internet, particularly twitter, in trying to figure out a method to evaluate the defensive performance of various hockey players, particularly defensemen.  He's come up with a statistic using shots against, called SDI, in order to attempt to quantify defensive performance. 

The problem with Burtch's methodology is that it uses shot attempts against as a measure of defensive ability.  In theory, this should be sound - shots aren't affected by goaltending, and we use shots all the time.  Surely those who are the most sound defensively prevent shots!

But the problem is that it's not quite that clear cut.  For one, certain teams, such as the New Jersey Devils, employ systems that quite clearly reduce the amount of shots for BOTH teams during each game - in other words, they play "low event hockey."  This isn't rink bias as it shows up on the road as well, and is notable as the Devils in 2013 were by far the leaders in neutral zone faceoffs (http://stats.hockeyanalysis.com/teamstats.php?disp=1&db=201213&sit=5v5&sort=NZPCT&sortdir=DESC).  Similarly, my zone tracking of 5 devils games, i found they had less 5 on 5 zone entries on average than most teams.  And the NZ faceoffs #s are consistently up near the top every year.  In short, the Devils keep the puck in the neutral zone for more than other teams, so their D #s by Burtch's metric look great (while their O #s presumably look lousy).  But that's not great defense! 

Of course there's a second problem with Burtch's methodology, and it's not unique to Burtch's "SDI" statistic.  Burtch, of course, isn't the first one to try and separate out defensive performance from the other parts of the game.  Hockey Prospectus' GVT uses Relative +/- to evaluate D performance (a stat I find garbage for the same reasons most of us don't use +/- anymore, but never mind that) and David Johnson's HARD statistic attempts to measure D using goals, shots, fenwick, or corsi against, again similar to Burtch. 

All of these metrics share the 2nd problem, which is this: the best offense is often the best defense for the obvious reason that possession in the offensive zone heavily reduces time in the defensive zone, thus reducing shots against.  So amongst your devils on the fenwick against leaderboard, you also see supposedly offensive-only Jake Muzzin of the Kings, Thomas Hickey of the Isles (alongside more known for his offense Lubomir Visnovsky), Anton Stralman of the Rangers, etc.  SDI and these other metrics will have these guys as great defensive players, when their teams would certainly not label such players as their first or second choice or maybe even 3rd or 4th choices in the last few minutes off a defensive zone faceoff. 

But here's the thing:  WHO CARES?!  As I just put it: The best D is often a good O, or to put it another way: the best way to avoid being scored on is to have POSSESSION of the puck.  We have metrics meant to measure possession of teams while a player is on the ice: in other words Corsi or Fenwick.  If a player's possession metrics are extremely positive, the end result is that the player is by DEFAULT a plus defensive player - after all, even if the player's doing it by way of keeping the puck in the offensive zone a lot, that's still preventing opponents from scoring.  And Possession Metrics, considering both shots against and shots for, essentially do away with the problem of low-event or high-event systems resulting in oddities like the NJ Devils or perhaps why Niklas Lidstrom's teammates allow less shots (but perform worse overall!) against without him on the ice, than with him. 

Now this isn't perfect obviously - if you're a GM or a coach, there's obviously still some value at determining which player is the best at limiting shots against if you're going into a defensive shell at late game, and possession metrics won't tell you that - and using shots against still runs into the two problems above.  There are however, solutions on the way: for one, zone entry tracking allows us to separate performance of players into offensive zone, neutral zone, and defensive zone performance, and to separate out the effects of system vs player.  There are some questions of whether teams or players can sustain better than average OZone or DZone performance, but despite Eric's original post, my findings suggest to an extent they can. 

But for now, until every team's neutral zone play is tracked, we really can't measure pure "defensive ability" of any player in the NHL.  And you know what?  That's still pretty damn OKAY!  If we built our team around only plus possession players, we'd come out ahead even if our team didn't have a so-called defensive specialist. 

Side Note: This blog used to be used for random thoughts and mainly for storing images for some posts on Lighthouse Hockey, but I think i'll add some hockey analytics thoughts here for thoughts that aren't big enough to be full time posts (or don't have the math).