Command PostFebruary 15, 2008
Weighing In
By Joe P. Sheehan

Johan Santana's changeup has been on my mind for the past week. Ever since I learned that if right-handed hitters make contact with the pitch, which doesn't happen very often, they tend to drive it, I haven't been able to stop thinking about it. Santana's changeup is said to be one of the best pitches in baseball, so I thought that in addition to creating a lot of swings and misses, this pitch wouldn't be beaten like a mule when it was put in play. I wasn't sure how the relationship between the swings and misses he got and the hits he allowed impacted the perception of the pitch but some comments on the article offered different ways to look at the changeup. One suggestion was to find the run value of every pitch to see which pitches are most beneficial, so thanks to Renè's idea, I did just that.

Finding the run value of a pitch is not as hard as I initially thought it might be. Using Tango's linear weights generator I found the run value of a single, double, triple, home-run and out. Using those values, I was easily able to find the value of each pitch for balls that were put in play, but I also needed to account for pitches that weren't put into play. To find the value of an average ball and strike, I converted the wOBA for each count into runs for that count, and then found out how much adding one ball changed those values for every count. I did the same thing for strikes, with the end result being that a ball is worth about .097 runs and a strike is worth about -.124 runs. There's a huge difference in the value of a ball or strike depending on what the count is, but I used these average values for my analysis because I didn't want to slice my already somewhat small sample of pitches into 12 smalled samples. As I continue to sift through this topic, I'm going to have to account for the different counts.

Below are the 10 pitches that saved the most runs in the 2007 season. In addition to the run value of each pitch, the Sw% (swings and misses/total swings) and SLGBIP (includes home runs) are also shown. I broke the pitches up by batter hand to give a more accurate portrayal of exactly who is impacted by a pitch.

Name            Pitch   N       Batter  LWTS    Sw%     SLGBIP
Brandon Webb    FB      460     R       -13.28  0.12    0.270
Jake Peavy      FB      456     R       -9.16   0.22    0.288
Chris Young     FB      363     R       -7.91   0.22    0.328
Kason Gabbard   CH      147     R       -7.72   0.36    0.182
Roy Halladay    FB      224     L       -7.36   0.07    0.250
Felix Hernandez CH      124     L       -7.27   0.23    0.069
Greg Maddux     FB      443     R       -6.89   0.05    0.430
Brian Bannister FB      289     R       -6.86   0.14    0.333
Dan Haren       CB      264     R       -6.81   0.26    0.309
Cole Hamels     CH      176     R       -6.70   0.37    0.308

Brandon Webb's sinker was most valuable pitch in terms of preventing runs last year, coming in at 13 runs saved vs. a league-average pitch. Other stud pitches fill this list, which was actually made up of more fastballs than I would have anticipated. However, since this is just total runs saved and fastballs are thrown so frequently, the results really aren't surprising. Finding the raw number of runs saved is going to highlight quality pitches, but it also is impacted by the number of times the pitch is thrown. If I want to look at the quality of a pitch, independent of how often it's thrown, LWTS per pitch is going to be much more informative. Here is a list of the best pitches by LWTS/pitch, for pitches that were thrown a minimum of 50 times.

Name            Pitch   N       Batter  LWTS    LWTS/pitch
Matt Herges     CH      67      L       -5.95   -0.09
David Weathers  SL      50      R       -3.95   -0.08
Jon Rauch       FB      52      L       -3.78   -0.07
Ruddy Lugo      CB      59      L       -4.03   -0.07
Matt Capps      FB      68      R       -4.67   -0.07
Brandon Webb    CH      68      R       -4.23   -0.06
Felix Hernandez CH      124     L       -7.27   -0.06
Kason Gabbard   CH      147     R       -7.72   -0.05
J.C. Romero     CH      71      R       -3.36   -0.05
Brett Myers     FB      71      L       -3.11   -0.04

This list has some crossover from the first list, and the new list confirms that King Felix has a great changeup (vs. LHH), especially compared to other changeups thrown by right-handed pitchers to left-handed hitters. Kason Gabbard's changeup (vs. RHH) also makes an repeat appearance on the list, which is a bit of a surprise because I had no idea his changeup was that good. Changeups thrown to an opposite handed batter generally cost a pitcher .01 runs per pitch, but Gabbard, Hernandez and Matt Herges were all able to buck that trend last year. Webb is also on this list, but for his changeup, not his fastball. Webb actually has a higher ground ball percentage on his changeup than on his fastball, which helps to explain the inclusion of his changeup on this list, but it's interesting that while Webb's sinker is considered his money pitch, his changeup might actually be a more effective pitch.

Looking a little closer at Webb's pitch repertoire you can see the effectiveness of each of his pitches. He's tougher on right-handed hitters overall, although lefties have a tough time hitting his curveball. Against righties, his changeup is twice as effective as his sinker, although that could be because he throws it infrequently relative to the sinker.

Pitch     N     Batter  LWTS    LWTS/pitch
FB      460     R      -13.28  -0.03
FB      517     L       2.15    0.00
CH       68     R      -4.23   -0.06
CH       89     L       0.90    0.01
CB       77     R       0.28    0.00
CB      112     L      -2.17   -0.02
CT       67     R      -1.32   -0.02
CT       97     L      -0.77   -0.01
Total  1487     -      -18.42  -0.01

One thing that piqued my curiosity when looking at this list of pitches was if the 18 runs that Webb's pitches prevented could be something larger. Was Webb 2 wins above average in the starts that he made in Gameday parks? Could those wins be directly attributed to his pitches? Webb's pitches prevented 18 runs over what a set of average pitches would have done, so his pitches could be said to be responsible for 1.8 wins more than an average pitcher. Counting the playoffs, Webb made 16 starts in stadiums with the pitch f/x system in place, pitching 113 innings and posting an ERA of 2.55. 113 innings with a 2.55 ERA in the NL makes a pitcher 5 wins above average in his starts at enhanced parks. Perhaps fielding made up the 3 win difference over this time period, or perhaps Webb leveraged his pitches effectively, throwing strikes when it was important and throwing outside the strikezone when it wouldn't hurt him too much. Exploring this topic in more detail probably deserves a whole column at some point.

Getting back to all pitchers, I wasn't very happy with the list of LWTS/pitch that I showed earlier. There were a lot pitches that had great rates but had only been thrown a handful of times, making me wonder if the pitcher had just gotten lucky throwing them. I'm sure Rudy Lugo has a great curveball, but he's only thrown it 59 times. I could have raised the minimum number of pitches, but that would eliminate the interesting pitches. The solution in this case is to regress the LWTS/pitch values toward the mean. Using the average value of every subset of pitch (fastballs thrown by LHP to LHH and fastballs thrown by LHP to RHH are examples of subsets) I did a rough regression which gave results that matched the general perception of pitches.

Name            Pitch   N       Batter  LWTS/pitch (regressed)
Kason Gabbard     CH    147     R       -0.04
Roy Halladay      FB    224     L       -0.04
Felix Hernandez   CH    124     L       -0.04
Matt Herges       CH     67     L       -0.04
Cole Hamels       CH    176     R       -0.04
Scott Kazmir      FB    288     R       -0.03
Aaron Laffey      CT    226     R       -0.03
Bobby Jenks       FB    107     R       -0.03
Jonathan Papelbon FB    148     L       -0.03
Jonathan Papelbon FB    127     R       -0.03
Mariano Rivera    FB    187     L       -0.03

This list makes much more sense. Gabbard's changeup (vs. RHH) remains at the top, which is something that bears watching in 2008. The rest of the list is filled with most of the usual suspects, Cole Hamels' changeup (vs. RHH) lives up to the hype, Kazmir's fastball is up where you would expect it and Jonathan Papelbon's fastball is amazing. It's equally effective against both lefties and righties, which is impressive by itself, but its even more amazing that it's so effective against both types of hitters. The last pitch on this list is Mariano Rivera's cutter (vs. LHH), which is another pitch that has been on my mind recently. This pitch showing up is no surprise, and I wish we could have seen where it ranked when Rivera was on the top of his game. If you're wondering, Jared Burton's cutter, the closest thing Rivera's pitch has to a modern-dayclone, was the 12th most effective pitch in baseball, falling just outside of this list. He's someone else to to watch in 2008. Also, after doing the regression, Webb's sinker (vs. RHH) is slightly more effective than his changeup (vs. RHH).

So where does all this leave us with Santana's changeup against right-handed hitters? Compared to other left-handed changeups thrown to right-handed hitters, Santana's changeup is exactly average, with a regressed LWTS/pitch of 0. Last year, the swings and misses the pitch created were counterbalanced by the pounding the ball took when it was put in play. Against righties the pitch Santana was most effective with was his fastball, which was worth -.03 runs every time he threw it (it also fell just outside the top-10). There are a ton of factors that impact how effective a pitch is, and maybe right-handed batters have started to sit on Santana's changeup more at the expense of hitting his fastball, but for last year at least, his changeup was pedestrian while his fastball was tremendous.


I know that Rich Lederer commented in a previous article that, "The first question should be "why?" Joe mentioned "deception, arm angle and pitch selection" in his article. I would add count and pitch location as important variables."

One of the things that could also be considered and that you use roughly is the difference of two related pitches in terms of movement and speed compared to the league average (such as a fastball and a changeup). I mean, maybe the reason that Brandon Webb's changeup appears to be a better pitch is because his changeup plays so well off of his fastball compared to league average.

For example, Webb throws his sinker ~70% of the time, yet he only throws his fastball ~6% of the time and his changeup ~11% of the time. In this, Webb's changeup has a very similar z movement as his sinker. That in and of itself is incredibly different versus the league average for that pitch (~ -3.6"). Furthermore, his changeup plays so well off of his fastball likely because his difference in the z plane between his changeup and fastball is also very different than the league average difference (~ -2.3"). I'd like to think that this is why Webb is more effective, but I don't have any real evidence to this point.

Well, this is good stuff. I think it is a pretty exciting time to be a baseball analyst.

I think you'll find it interesting to use specific values for strikes and balls by count, because you'll see strikeout pitches come up a lot in the list, and you'll be able to separate 2-strike foul balls from other foul balls (which are essentially as good as strikes). Pitchers do leverage their pitches so in the end it is important to check against their use.
For instance, with Josh Kalk's general dataset, I had Papelbon saving 11.5 runs with his fastball, 0.5 with his splitter, 0.4 with his slider and 0.8 with his curveball, by using average values. When I checked every single count, I found that his fastball saved 8.9 runs, while his splitter saved 0.7, his slider 0.4 and his curveball 0.6.
The run value of his splitter looks ridiculously low as opposed to the perceived effectiveness (and to Sw% as you call it - I found that 88% of the times he throws it, it isn't put into play!), but I found that early in the count (basically, when hitters have the luxury of "taking" the pitch) it has actually hurt Papelbon because it is often taken for a ball, while with 2 strikes it has saved as much as his fastball, even though he has thrown the splitter less than one third of the time in those situations (which speaks volumes about his splitter's effectiveness when hitters "have" to protect the zone).
I'm looking forward to see what the result is when we take a look at every single pitch, count by count. Unfortunately I haven't had the chance to work will the whole data and have had to extract "manually" from Josh Kalk's database count data for individual pitchers (and have therefore calculated run values for them) but I haven't worked with all the pitchers and pitches, so I'm very curious to see what you will come up with.
Not to offer a plug, but if you click on my name you should be able to see a table with the results I found regarding Papelbon's pitches. I'm not linking my actual blog with my work because it's in a different language (even though google translator is pretty good).

Joe: Excellent work. One minor issue: I think you were comparing Webb's pitch linear weights above AVERAGE to his actual runs above REPLACEMENT. His ERA makes him about 22 runs above average, quite close to your linear weights estimate.

It seems to me that sinkerball pitchers are often mistaken in their pitch assignments by scorers. Isn't there a tendency for the sinkerball fastball to be erroneously categorized as a changeup instead of an actual sinking fastball?

How is that accounted for when a pitcher has a regular fastball and a sinker? Do scorers attribute each as a FB or do they call the sinker a CH to create a distinction? You don't see any SK's on scoring sheets, but you should.

Wow, great analysis. One comment that I don't see made in your analysis, but that should be made is that sometimes a pitch that might not have a very high effectiveness rating might be necessary to make the pitcher's other pitches effective. Trevor Hoffman's fastball would be the typical example of a pitch that is only used to setup his change up, but is necessary to make the changeup effective.


I did compare him to replacement level, but doing it again (with average as the baseline) I get 28 runs above.

A 2.55 ERA and 4.40 ERA combine for a 6.95 earned runs/9. I divided that by .92 to get total runs, which is 7.55 runs/9.

I used PythagenPat to get his exponent (7.55^.28=1.76)

Webb's winning % is .723-.500=.223 above average.

.233*(113/9)=2.79 wins above average.

I'm relatively inexperienced at calculating these values, so followed the example that Tango did here.

I probably goofed somewhere though, and if you could show me where, that would be really helpful.

Joe: the easier calculation is to forget about wins and just do (LgAvg - Webb) * IP/9, which is (4.40-2.55)*113/9 = 23.2 ER above average. But Webb also gives up a lot of unearned runs (often true of extreme GB pitchers), so R = 1.15*ER. Don't know if that's true in your sample, of course, but if so then Webb is (4.71 - 2.93)*113/9 = 22.3 runs above average. The four run difference could be good defense, GDP, "clutch" pitching, or -- most likely -- random noise.

Actually, if I'm following your method, fielding couldn't contribute to the difference since your linear weights already incorporate the actual hit/out outcomes. Is that right? In which case, the linear weights are not defense-independent (or park independent, which may partially account for Maddux appearing on your first list.)

It also occurs to me that good pitchers may consistently overperform your linear weights estimate, because the value of a Webb strike is higher than average and the harm done by a Webb ball is lower than average. Might be worth checking to see if you find such a pattern.

Guy: I think it's mostly that he's worked with the average value of strikes and balls, so there's the issue of 2-strike foul balls, plus GIDP which aren't really accounted for. But he said he's going to work with actual counts, so we should see how that works out.

A couple things...

Mike Dreyfus,

The pitch labeling is done by me with a cluster analysis based on the pitch speed and pfx/pfz values. It's not perfect, but it does well for most pitchers. As far as the differentiation between fastballs, sinkers and changeups, the way I label pitches is that the fastest pitch a pitcher throws is his fastball, so I don't explicitly label anything a sinker. Using the movement data, I didn't feel comfortable drawing a line and labeling a certain set of pitches as 'sinkers', so if I use sinkers as a group in an article, I explain how I grouped them.


Thanks for walking me through that calculation.

The linear weights I used in the article were the average values for each event, across all counts. I just finished doing it so that every event (hits, outs, balls, strikes and fouls) had a different value, based on the count it occurred in (a 2-1 single hurts a pitcher less than a single in an 0-2 count), and the sum of Webb's pitches was now 25.4 runs better than average, compared to 22-23 runs above average using his ERA.

I'm doing this right now for Peavy. I have some data for every start he made after 7/5, which is 111 innings with a 2.99 ERA. (4.4-2.99)*(111/9)=17.39 ER or around 19 R. The sum of his pitches is -17.47.

Derek Lowe. I've got him from 7/2 and after, 84 innings, ERA of 5.06, which gives 4-6 runs worse than average. Sum of his pitches is 0.02, which is a similar difference for Webb.

One more, Jamie Moyer. Pitch f/x data from most games from 7/16 and after. ERA of 5.78 over 86 innings during then, which is 14-15 runs worse than average. Sum of his pitches is 16.91, which is close.

The one thing I didn't factor into my lwts calculations that could be skewing these results are double plays. I treated all outs as the same, but obviously if double plays are happening, they need to be counted twice.

The next step with this is to find personalized lwts for every pitcher, in every count and for every event, because as Guy points out, a Webb strike is better than an average strike.