Monday, February 23, 2015

What Statistics Actually Indicate Post-Season Success?

Recently I've been a doing little amateur research into what regular season performance metrics actually translate into postseason success.  I haven't studied statistics since college Econ classes, so really all I could do was a simple correlation comparison on a spreadsheet, but here is what I found.

Methodology

For this test, I ran a correlation against postseason success (defined by number of playoff games won), against regular season performance statistics (defined by league standing in particular category, ie 30 points for being first place in the category, 1 point for being last place in the category).

Looking at it now, there are two ways to improve this methodology.  One, would be to standardize the performance statistics such that, instead of giving one "point" for each spot in league standing, the statistic would be relative to the league mean.  This is because sometimes the difference between first and second is much more significant that the difference between 10 and 15th.  This I could probably do with some time.

Second, would be to develop an adjustment for win quality.  A win against the Kings in the first round is worth more than a win against Panthers.  This is probably beyond my current statistical kung fu, though.


Results


 Takeaways

-There is no magic bullet, as my sample size probably still isn't big enough to remove the noise.  Some years, you'll have a team like the 2010 Flyers who simply weren't very good in the regular season, transform themselves in the postseason and no regular season statistic captures their real quality.  Other years, you'll have two top notch teams play each other in the first round, so one of them has to be eliminated and can't notch more than 3 playoff wins.  Still, some patterns begin to form.

-Over a 7 year window, plain old goal differential proves to be by far the most accurate predictor of winning playoff games.  Over the last 3 seasons however, penalty kill and team save percentage have been shockingly accurate in predicting playoff success.

-The last few seasons, teams that relied on the PP and a high shooting percentage in the regular season have gotten killed.  As you increase the sample size to 7 years, those numbers regress to an insignificant correlation either way.

-For all the talk of advanced statistics, looking at Corsi or Fenwick at 5v5 isn't telling you anything that more basic statistics are already capturing.  Over a 7 year window, plain old 5v5 goal differential proves a better predictor, and vanilla shot differential may be the most robust predictor of all.  My theory?  The real value of Corsi and Fenwick are that they generate more statistical events when evaluating a team over a short period of time, or an individual who is only seeing a fraction of the teams overall ice time.  In the long run however, advanced statistical events like Corsi or Fenwick will eventually translate into shots, shots will eventually translate into chances, and chances will eventually translate into goals, and goals translate into wins.  Over an 82 game window for all team events, the value of using Corsi or Fenwick statistic is minimized, and a more direct measurement of quality like goals gives a better prediction.

Conclusion

So in my highly simplified, surely flawed first analysis, I would turn to total goal differential, shot differential, penalty kill, and team save percentage as key performance indicators when you want to guess at which team is primed for playoff hockey.