Last week over at Sports Illustrated, I wrote an article on the biases of modern Hall of Fame voters. In it I highlighted five ways that Hall of Fame voters either overrated or underrated candidates. While, I provided mostly anecdotal evidence at SI, here I'll use a statistical approach to analyze whether or not my hypotheses were true (and what else I may have missed).

My goal here is to determine if voters are underrating or overrating certain types of players. But to do so, first I need to determine how to define the true "value" of each player. For example, if I say a particular player is overrated, what is the gold standard which defines how voters should consider a player?

Here I choose to use career Wins Above Replacement (WAR), taken from Rally's WAR database. Rally's WAR considers all aspects of a player's performance, including hitting, defense, baserunning and pitching, and by all accounts gives a pretty accurate picture of a player's contributions.

If the Hall of Fame voters are completely unbiased, they will simply use WAR and only WAR to consider a player's credentials for the Hall. My goal here is to determine empirically, what factors apart from WAR contribute to a player's Hall credentials - in other words how are Hall of Fame voters biased?

To set up the problem, I took all players eligible for the Hall of Fame going back to 1986. I then put them into five categories:
1. HoF on first ballot,
2. HoF in 2-4 years,
3. HoF in 5-15 years,
4. >5% of vote, but did not make HoF
5. <5% of vote on first ballot

I then used a multiple logistic regression to model the players' chances of falling into each of these categories. Obviously, the model included a player's WAR. However, the goal was to see if the model had any other significant variables. Significant variables besides WAR, would show a Hall of Fame voter bias.

Hitters

For hitters, a reduced model broke down as the following (here shown for the probability of making the Hall of Fame at all):

Probability of HoF = exp(a)/(1+exp(a))
where a is equal to:
(-38.5 + .172*WAR - 75*BB_RATE + 94.4*BAV + .0032*PA + 57.9*HR_RATE)

Here we see that, obviously, the more WAR, the better. However, what we also see is a positive bias for batting average, homerun rate, and for the total number of plate appearances. This indicates that Hall of Fame voters are biased towards guys with high batting averages who hit a lot of homeruns. In other words, voters overvalue these statistics in their evaluations. However, we see a strong negative bias towards a player's walk rate. This indicates, that players who walk a lot are being unfairly punished by Hall voters. These findings pretty much confirm what is expected. Hall voters have the same biases that most mainstream media do, in undervaluing walks and overvaluing batting average.

RBI rate, while not significant in this model, is significant if homeruns are removed (the two variables are fairly correlated). As one would expect, RBIs also are overvalued by Hall of Fame voters.

Interestingly, Hall of Fame voters are also biased towards players who have long careers. I had expected voters to possibly have a bias toward high peak performance, but instead the voters seem to have the opposite bias. They overvalue a player's longevity, rewarding mediocre and bad years from players and undervaluing peak performance. In other words, Hall Voters set the bar for replacement level too low.

Pitchers

For starting pitchers, we get an entirely different model of course:

Probability of HoF = exp(a)/(1+exp(a))
where a is equal to:
(-39.8 + .037*WAR + 40.2*WPCT + .053*Wins)

In this case, the model boils down to three key variables. As you'll notice, WAR is not a very important factor in the model. In fact, with a p-value of .37 it is not even significant! Highly significant however (p-value <.003) are a pitcher's winning percentage and career wins. In fact, these seem to be the only two variables necessary to predict Hall of Fame induction. ERA, strikeout rate, and other factors are not necessary (at least with this dataset, though it may be noted that we haven't had a short career Koufax-type pitcher inducted in the last 25 years). Obviously the message is clear, a pitcher's wins and losses are vastly overrated by Hall of Fame voters. Again, Hall of Fame voters overvalue a long career (wins is a more significant proxy for innings pitched). Surprisingly, a voters are not only biased towards wins and losses, but these statistics almost totally replace the pitcher's true WAR value as predictors.

Relievers

As for relievers, the dataset was fairly small, however, here are the results:

Probability of HoF = exp(a)/(1+exp(a))
where a is equal to:
(868.4 + .181*WAR-.446*First_Year + .022*Saves)

For relievers, I added a term for the year in which a player started his career. This year variable had a p-value of .06, indicating that voters may have given early relievers an advantage for "pioneering" the role of short reliever. Saves were only marginally significant with a p-value of .11, however, the effect appears to be positive. Since there are relatively few relievers enshrined or even considered for the Hall of Fame there is not a lot of power to figure out what's going on.

Hitters vs. Starters vs. Relievers

Another effect, not seen in the above models is the bias between hitters, starters, and relievers. Doing another model including only WAR and a dummy variable indicating whether the player was either a position player, starter, or reliever, shows strong differences between the three groups.

The results? In order to have a 50% chance of making the Hall, a reliever has to only amass 43 WAR. For a position player, he has to amass 59 WAR, while a starting pitcher has to amass 72 WAR. Here we see a big difference in the standards set up for each of the three roles. Although only four modern relievers currently occupy the Hall of Fame, voters have been giving relievers a break. Meanwhile, starting pitchers have been getting the shaft. For starters to make the Hall, they must provide more value to their teams than a position player does.

A similar analysis of just position players broken down by type of position, shows that those in "fielding positions" (2B, 3B, SS, C) have it tougher than those in "hitting positions" (OF, 1B, DH). This seems to agree with the common perception that outfielders are overrepresented in the Hall of Fame, while players such as third basemen have a tough time.

Conclusions

In all, the empirical analysis shows the following:
1) HoF voters undervalue walks (p-value .001)
2) HoF voters overvalue batting average (p-value .001)
3) HoF voters overvalue longer careers (p-value .001)
4) HoF voters undervalue starting pitchers (p-value .001)
5) HoF voters overvalue relief pitchers (p-value .001) though this bias seems to be decreasing
6) HoF voters overvalue Wins and Losses for pitchers (p-value .003)
7) HoF voters undervalue players at defensive positions (p-value .005)
8) HoF voters overvalue homeruns/RBI (p-value .06)

Fun Stuff

While, it's not the point of the exercise, you may be wondering about the values predicted for each player. To satisfy your curiosity here are the following breakdowns in order of likelihood of making the Hall of Fame:

Likely (80% or higher)
Henderson
Schmidt
Eckersley
Ripken
Carew
Brett
Jackson
Boggs
Molitor
Rose
Seaver
Yount
Gwynn
Carlton
Murray
Morgan
Dawson
Winfield
Sutton
Palmer
Gossage
Neikro
Ryan
Fingers

Probable (50-80%)
Fisk
Alomar
E. Martinez
Perez
Puckett
Sandberg
L. Smith
Rice
McGriff
W. Clark
Blyleven
Larkin

Unlikely (10-50%)
John
Dw. Evans
Trammell
Parker
Burks
Whitaker
McGraw
Raines
Carter
M. Williams
Morris
B. Bell
K. Hernandez
Oliver
Garvey
Baines
Sutter (8%)

*Ozzie Smith was removed from modeling, as he is the only player ever inducted (or really even given consideration) solely for his defensive efforts.

As Rich alluded to yesterday, some of these biases may (and hopefully will!) disappear as voters become more savvy about how to properly evaluate players. It will be interesting to see what happens during the next 25 years, and how voting will have changed after the "sabermetric revolution".