Behind the Scoreboard January 12, 2010
Biases of Hall of Fame Voters

Last week over at Sports Illustrated, I wrote an article on the biases of modern Hall of Fame voters. In it I highlighted five ways that Hall of Fame voters either overrated or underrated candidates. While, I provided mostly anecdotal evidence at SI, here I'll use a statistical approach to analyze whether or not my hypotheses were true (and what else I may have missed).

My goal here is to determine if voters are underrating or overrating certain types of players. But to do so, first I need to determine how to define the true "value" of each player. For example, if I say a particular player is overrated, what is the gold standard which defines how voters should consider a player?

Here I choose to use career Wins Above Replacement (WAR), taken from Rally's WAR database. Rally's WAR considers all aspects of a player's performance, including hitting, defense, baserunning and pitching, and by all accounts gives a pretty accurate picture of a player's contributions.

If the Hall of Fame voters are completely unbiased, they will simply use WAR and only WAR to consider a player's credentials for the Hall. My goal here is to determine empirically, what factors apart from WAR contribute to a player's Hall credentials - in other words how are Hall of Fame voters biased?

To set up the problem, I took all players eligible for the Hall of Fame going back to 1986. I then put them into five categories:
1. HoF on first ballot,
2. HoF in 2-4 years,
3. HoF in 5-15 years,
4. >5% of vote, but did not make HoF
5. <5% of vote on first ballot

I then used a multiple logistic regression to model the players' chances of falling into each of these categories. Obviously, the model included a player's WAR. However, the goal was to see if the model had any other significant variables. Significant variables besides WAR, would show a Hall of Fame voter bias.

Hitters

For hitters, a reduced model broke down as the following (here shown for the probability of making the Hall of Fame at all):

Probability of HoF = exp(a)/(1+exp(a))
where a is equal to:
(-38.5 + .172*WAR - 75*BB_RATE + 94.4*BAV + .0032*PA + 57.9*HR_RATE)

Here we see that, obviously, the more WAR, the better. However, what we also see is a positive bias for batting average, homerun rate, and for the total number of plate appearances. This indicates that Hall of Fame voters are biased towards guys with high batting averages who hit a lot of homeruns. In other words, voters overvalue these statistics in their evaluations. However, we see a strong negative bias towards a player's walk rate. This indicates, that players who walk a lot are being unfairly punished by Hall voters. These findings pretty much confirm what is expected. Hall voters have the same biases that most mainstream media do, in undervaluing walks and overvaluing batting average.

RBI rate, while not significant in this model, is significant if homeruns are removed (the two variables are fairly correlated). As one would expect, RBIs also are overvalued by Hall of Fame voters.

Interestingly, Hall of Fame voters are also biased towards players who have long careers. I had expected voters to possibly have a bias toward high peak performance, but instead the voters seem to have the opposite bias. They overvalue a player's longevity, rewarding mediocre and bad years from players and undervaluing peak performance. In other words, Hall Voters set the bar for replacement level too low.

Pitchers

For starting pitchers, we get an entirely different model of course:

Probability of HoF = exp(a)/(1+exp(a))
where a is equal to:
(-39.8 + .037*WAR + 40.2*WPCT + .053*Wins)

In this case, the model boils down to three key variables. As you'll notice, WAR is not a very important factor in the model. In fact, with a p-value of .37 it is not even significant! Highly significant however (p-value <.003) are a pitcher's winning percentage and career wins. In fact, these seem to be the only two variables necessary to predict Hall of Fame induction. ERA, strikeout rate, and other factors are not necessary (at least with this dataset, though it may be noted that we haven't had a short career Koufax-type pitcher inducted in the last 25 years). Obviously the message is clear, a pitcher's wins and losses are vastly overrated by Hall of Fame voters. Again, Hall of Fame voters overvalue a long career (wins is a more significant proxy for innings pitched). Surprisingly, a voters are not only biased towards wins and losses, but these statistics almost totally replace the pitcher's true WAR value as predictors.

Relievers

As for relievers, the dataset was fairly small, however, here are the results:

Probability of HoF = exp(a)/(1+exp(a))
where a is equal to:
(868.4 + .181*WAR-.446*First_Year + .022*Saves)

For relievers, I added a term for the year in which a player started his career. This year variable had a p-value of .06, indicating that voters may have given early relievers an advantage for "pioneering" the role of short reliever. Saves were only marginally significant with a p-value of .11, however, the effect appears to be positive. Since there are relatively few relievers enshrined or even considered for the Hall of Fame there is not a lot of power to figure out what's going on.

Hitters vs. Starters vs. Relievers

Another effect, not seen in the above models is the bias between hitters, starters, and relievers. Doing another model including only WAR and a dummy variable indicating whether the player was either a position player, starter, or reliever, shows strong differences between the three groups.

The results? In order to have a 50% chance of making the Hall, a reliever has to only amass 43 WAR. For a position player, he has to amass 59 WAR, while a starting pitcher has to amass 72 WAR. Here we see a big difference in the standards set up for each of the three roles. Although only four modern relievers currently occupy the Hall of Fame, voters have been giving relievers a break. Meanwhile, starting pitchers have been getting the shaft. For starters to make the Hall, they must provide more value to their teams than a position player does.

A similar analysis of just position players broken down by type of position, shows that those in "fielding positions" (2B, 3B, SS, C) have it tougher than those in "hitting positions" (OF, 1B, DH). This seems to agree with the common perception that outfielders are overrepresented in the Hall of Fame, while players such as third basemen have a tough time.

Conclusions

In all, the empirical analysis shows the following:
1) HoF voters undervalue walks (p-value .001)
2) HoF voters overvalue batting average (p-value .001)
3) HoF voters overvalue longer careers (p-value .001)
4) HoF voters undervalue starting pitchers (p-value .001)
5) HoF voters overvalue relief pitchers (p-value .001) though this bias seems to be decreasing
6) HoF voters overvalue Wins and Losses for pitchers (p-value .003)
7) HoF voters undervalue players at defensive positions (p-value .005)
8) HoF voters overvalue homeruns/RBI (p-value .06)

Fun Stuff

While, it's not the point of the exercise, you may be wondering about the values predicted for each player. To satisfy your curiosity here are the following breakdowns in order of likelihood of making the Hall of Fame:

Likely (80% or higher)
Henderson
Schmidt
Eckersley
Ripken
Carew
Brett
Jackson
Boggs
Molitor
Rose
Seaver
Yount
Gwynn
Carlton
Murray
Morgan
Dawson
Winfield
Sutton
Palmer
Gossage
Neikro
Ryan
Fingers

Probable (50-80%)
Fisk
Alomar
E. Martinez
Perez
Puckett
Sandberg
L. Smith
Rice
McGriff
W. Clark
Blyleven
Larkin

Unlikely (10-50%)
John
Dw. Evans
Trammell
Parker
Burks
Whitaker
McGraw
Raines
Carter
M. Williams
Morris
B. Bell
K. Hernandez
Oliver
Garvey
Baines
Sutter (8%)

*Ozzie Smith was removed from modeling, as he is the only player ever inducted (or really even given consideration) solely for his defensive efforts.

As Rich alluded to yesterday, some of these biases may (and hopefully will!) disappear as voters become more savvy about how to properly evaluate players. It will be interesting to see what happens during the next 25 years, and how voting will have changed after the "sabermetric revolution".

Nice job with the modeling, Sky.

Re your asterisked comment about Ozzie Smith being "the only player ever inducted (or really even given consideration) solely for his defensive efforts," Rabbit Maranville and Bill Mazeroski would fit into this category. Maranville was elected by the BBWAA and Mazeroski by the Veterans Committee. Smith was a more productive offensive player (hitting and baserunning) than Rabbit and Maz. There may be others as well, but these two former players jumped out at me.

Rich, You make a good point. I should amend my comments to say the only player in the data (last 25 years of eligibility).

OK. A re-read of your fine article tells me you meant the last 25 years as it relates to Ozzie. I just didn't pick up on that the first time through.

Hall of Fame voters seem to value MVP, Cy Young, and Gold Glove Awards plus magic milestones like 3,000 hits, 500 home runs, and 300 wins. Would it make sense to try and model for these items?

Where does Dale Murphy fit in your probabilities? I know his induction isn't likely but his career OPS+ is 2 points higher than Dawson's so I have to hold out some hope.

I agree with your conclusion, which seems to give credence to what you would have expected.

Just curious from a modeling perspective--how did you develop the regression model you used? eg, did you regress all possible hitting coefficients and then keep the statistically significant ones? I could imagine a scenario in which random regression running yielded a regression equation that showed SB and HBP as meaningful coefficients...

Jason, Murphy comes in at 6% probability of eventual enshrinement. Given the results so far, I'd put his actual chances at less than that.

Dave, I didn't use all possible variables in the original model - just the 5-6 factors I thought might have an effect. I did try stolen bases (no effect) but I didn't try HBP. I didn't want to go crazy with tons of covariates for the reasons you cite.

Thanks, Sky. Is there enough evidence to create a model to determine whether someone will get voted in by the Veterans' Committee? I wonder if their biases are any different than the writers'.

Excellent work, Sky.

As a Cubs fan, I can't help but hope the growth of Sabermetric knowledge leads to the likes of Ron Santo getting elected into the Hall. As a third baseman (and a diabetic throughout his career), he seems to be almost forgotten among voters.

You noted that "Hall voters have the same biases that most mainstream media do, in undervaluing walks and overvaluing batting average." This made me laugh, because almost by definition, Hall voters are the mainstream media.

Actually, I think Ozzie got some votes for his offense. For a small guy with no power, his offensive #s were pretty good the last 60% or so of his career. He had a 9 year stretch with an OPS+ of 95 or better in 8 of those years. Although I haven't looked it up, I bet his offensive WAR was probably near his defensive WAR. He was also just shy of 2500 hits, had over 400 doubles, and 580 SB. Ozzie also struck out only once every 19 PA. He probably would have made the hall with lesser offensive #s, but would not have been an easy first ballot choice if he had not improved his hitting over that of his first 6 years.

Another player inducted almost solely on defensive merit was Phil Rizzuto. Well, defense, and being on 9 pennant winners and winning 7 WS.

Forgive me... my math fu is weak, it's been a long time since high school. The term "exp(a)" is confusing me. If (for kicks and grins) I'm calculating the HoF chances of a current player, and calculate a=20 (just to pull a number out of thin air), what am I doing to that raw number to get this probability?

This is really good work.

It seems that the BBWAA are less likely to vote in a guy like Ozzie Smith where defense was a large part of his story. B. Robinson is closest guy I could think of.

It seems the Veteran's group voted in a lot more of these type of players, Maz, Schoendienst, Rizzuto, Reese, I think George Kelley only case was that he was a very good defensive first basemen.

nightfly,

exp(a) is another way of writing e^a (or e to the a power) where e is just a mathematical constant. Most newer calculators will have a button for this, but using e = 2.71828 should get you plenty close.

http://en.wikipedia.org/wiki/E_%28mathematical_constant%29

This is nitpicking, but with one prominent exception every pitcher in the Hall of Fame was inducted for his defense.

Interesting stuff Sky.

"I think George Kelley only case was that he was a very good defensive first basemen."

Kelly's case like numerous others- Hafey, Youngs, Bancroft, Marquard and Haines- was based primarily on the fact that he was a teammate of Frankie Frisch who was on the Veterans Committee from 1967 and 1973 and succeeded in greatly watering down the Hall of Fame by pushing through his buddies.

There are two guys who were put in the Hall solely for defense, with absolutely no other qualifications: Rabbit Maranville and Ray Schalk. Maranville at least was considered one of the very top level players in the game, even though he couldn't hit a lick.

I'm curious why you choose to use a multiple logistic, rather than regress HoF percentage from each election on player variables. Seems like a lot of data was thrown out of the modeling by that approach, rather than by having multiple observations for each player where y=percent of votes in that year.

Ryne - much thanks!

Where would Jim Edmonds and Larry Walker fit in the prediction model? I would assume Edmonds would do poorly, but I'm curiuos.