Comparing Team-Win Projections
I love preseason projections: the fact that so many smart people put some much work into it, the promise of the season to come, thinking about my own ideas for the season, and comparing across projections. I hope you will indulge my impulse to do this last one. Here I am going to compare the projected win totals -- but it would be very cool to do the same for player projections -- across six different projections systems. First the projected win totals based on the FAN projections as fangraphs; BPro's PECOTA; THT's OLIVER; Rally's CHONE; RLYW's CAIRO and, though it is not a projection per se, the Vegas over/under lines. Here are the RMSE between each of the six projections systems. Another way of analyzing this is to use principal component analysis (PCA). Picture each projection system as a 30-valued vector. You could plot each of the six systems in 30-space and see how close they are to each other, but, unfortunately, I cannot display 30-space on the computer screen. PCA is a tool to reduce the dimensionality of a data set. As an example if all the systems projection projected the same number of wins for all teams expect the Yankees and Red Sox, we could just look at their projections for the Yankees and Red Sox and get all of the information of the variation between the systems. In this case it is not as neat, but we can still find the teams which account for the most variation between the systems. By reducing dimensionality you lose some information, but the hope is the information lost is largely correlated (redundant) and much of the variation can be reduced to a handful of dimensions. Here you can see the FANS and CHONE clustering out relatively closely, with Vegas and PECOTA not that far off. Then THT and CAIRO falling out far away. CAIRO because of its love of the Reds, Twins and Mariners, while THT for its love of the Braves and Rangers, and to a lesser extent the Yankees. Again it would be very cool to do this for player projections and see whether the principal components to fall out as particular player types. Finally I wanted to see which teams had the most disagreement or consensus. Here is the average pair-wise disagreement for each team. |