Change-UpJanuary 12, 2011
Why Assume WAR Overemphasizes Defense?
By Patrick Sullivan

Recently Dave Cameron took to ESPN Insider to pen a column about Andruw Jones, how he stacks up against Derek Jeter, and what his Hall of Fame prospects might look like. Dave's a great writer, as you know, and he's at his best when handling provocative topics. It's a compelling read since, according to WAR, Jones stacks up nicely next to the Yankee legend.

I really only have one issue with it. At the end, he starts to back off. I can respect that on the one hand, because there is so much we don't know about defense and how it might impact Wins Above Replacement totals. On the other, he leaves no room for the possibility that Jones's defense could make him even better than WAR shows him to be.

While no one can deny the number of base hits that Jeter has accumulated, the idea of Andruw Jones being in the defensive company of Hall of Famers Brooks Robinson and Ozzie Smith could certainly be a contentious claim. Data should be used to inform our discussions, but we should not be slaves to the numbers, and there is a reasonable discussion that can be had about the scale of credit that should be given to players for their defensive abilities.

Certainly, Jones should get a significant boost for his defensive chops as he was widely seen as the game's best center fielder during his prime. He won 10 consecutive Gold Gloves, after all, so it is not only the numbers that see him as a historically elite defender. However, there are enough legitimate questions about defensive metrics, especially those from before this century, that we should be careful with equating defensive specialists with those whose value was created in more traditional ways.

I like that we're all going to stop short of assigning too much value to WAR. One number should not tell us everything. But with regard to defense, I don't like that our default assumption is that WAR overrates players who derive relatively more value from their defense. If we're throwing our hands up and saying "I don't know" then let's not then turn around and say "but I know if anything his defense is overvalued." Maybe when it's all said and done, when we really have a great sense for how to evaluate and then contextualize defense, players like Jones and Mike Cameron and Smith will have been sold short by WAR.


I don't think the point is assume that WAR overemphasizes defense (I don't have an Insider account, so I could be missing what Mr. Cameron is arguing). The point would be to question it at it's extremes. Which is more likely, that Andruw Jones is a good fielder who somehow through the statistical analysis comes off as a great fielder, or Andruw Jones is an otherworldly defender that gets knocked down by that same analysis? There's way more good fielders than otherworldly, so the former is more likely. The same is true on the other end of the spectrum with poor and horrible defenders. This isn't to say Jones isn't underrated, but it's more likely to be overrated and therefore we should be conservative when making claims about extreme defensive values.

I think MGL himself makes this point. This thread helps:


I don't see where Cameron did anything to indicate a "default assumption ... that WAR overrates players who derive relatively more value from their defense." I think the issue is that we have a much more limited agreement on how to measure defense and whether defensive metrics accurately measure what we think they do. To take the flip side, Dave Winfield has terrible defensive WAR stats. He is said to be worth something like -90 runs over his career. Winfield was generally considered a very good defender, won multiple gold gloves, had a cannon for an arm, was a great athlete and had good speed. It is possible that perception was wrong, but it doesn't mean it is. By contrast, Larry Walker has a similar profile and is shown as being plus 80 runs in the outfield.

How likely is it that Larry Walker was 170 runs better defensively over his career than Dave Winfield? I think even if you ask the most sabr friendly observer who saw them, they would be uncomfortable with that result.

By contrast, we can calculate offense and offensive value fairly well and make adjustments for context fairly well because the record is much more reliable.

I would say that defense is overemphasized in both WAR simply because it is given equal weight to offense when we have (or should have) less confidence in our assessment of defense than we do in our assessment of offense.

I am all for resisting the urge to assign too much significance to defensive metrics either way. But the whole tone it seems (and it's not just Cameron) is to temper our enthusiasm over defensive players rather than to maintain that neutral stance of simply not knowing as much as we do about offense.

DB, if Walker had much better instincts than Winfield, Winfield's speed wouldn't help too much.

It reminds me of Herb Washington, the track star signed by Oakland in the 1970s? to be a pinch runner. Washington was picked off his first time as a pinch runner and was a mediocre base stealer (like 70% or so success rate) even though he had world-class speed.

I agree that it's quite possible the current versions of WAR understate Jones' value. UZR and TZ are powerful in smaller samples, given the granularity of the data. However, over the last decade, Jones has played behind mainly groundball pitchers, limiting his opportunities, and common defensive metrics don't account for that.

Fangraphs' version of WAR evaluates the defense using that year's UZR. It doesn't use Defensive Runs Saved, which is probably a better system, or blend the two. It doesn't regress the measurement to league average.

I think for those reasons, it's fair to believe that WAR overstates the extremes, whether that is for players credited with exceptionally good or poor defense.

The range component of fielding stats is better than some of the other defensive data because it does not depend much on the choices of the opposition, such as not running on the strong-armed catcher or OF. No one ever thought of changing their swing to hit to RF because Ozzie Smith was at SS, whereas in football a team might run or throw to a side based on where the elite defender is, so their totals of tackles and interceptions might be misleadingly low.

I don't like that our default assumption is that WAR overrates players who derive relatively more value from their defense. If we're throwing our hands up and saying "I don't know" then let's not then turn around and say "but I know if anything his defense is overvalued."

I think Cameron' main concern here isn't whether WAR overrates defense. It's that Andruw Jones is an outlier in a method that we don't have as much confidence in as our methods to value hitting contributions.

I know "regression to the mean" is overused, but if the metrics used to calculate dWAR are wrong or flawed in some way, the far likelier scenario in the case of an outlier like Jones is that the current, flawed system is overrating him.

A few years ago +/- pegged Chase Utley as +47, and Cameron made the same point. Is it likelier that Utley's true defensive value that year was +42 or +52. I think the answer is obvious.