Touching BasesSeptember 09, 2009
On That Stuff
By Jeremy Greenhouse

Justin Verlander. A.J. Burnett. Ubaldo Jimenez. These are the names that make scouts salivate. Why? One word. Stuff.

Two components determine how nasty a pitcher’s stuff truly is: velocity and movement. We’ve had radar guns to track the league’s hardest throwers for some time (that would be Joel Zumaya, of course) But now, with the help of pitchf/x data and a local regression technique picked up from Dave Allen, we can come pretty close to quantifying a pitcher’s stuff. We can assign every single pitch an expected run value given its physical characteristics—be it velocity, movement, location, release point, or any other data point given by the pitchf/x data. For the purposes of measuring expected run value based on stuff (StuffRV), I used velocity, horizontal movement, and vertical movement as my three independent variables, and restricted my sample to only righties who released the ball from at least five feet off the ground, with a minimum of 1,000 pitches over the last three years. To the leaderboards.

Name StuffRV
A.J. Burnett -46
Felix Hernandez -31
Zack Greinke -26
Edwin Jackson -26
Ubaldo Jimenez -26
Chad Billingsley -23
Brian Wilson -22
Brandon Morrow -21
Roy Halladay -21
Matt Garza -20
Dave Bush 15
Jeff Suppan 17
Braden Looper 19
Livan Hernandez 20
Greg Maddux 23


Really no surprises on this list, which at least gives some validity to this method for evaluating stuff. I probably could have guessed 10 of these names correctly given 20 tries. One pitcher with lightning stuff who often goes overlooked is Gil Meche, who registered twelfth. One pitcher with deplorable stuff who does not go overlooked is Livan Hernandez.

Justin Verlander showed up a bit lower down in the rankings than I would’ve thought, sandwiched between Brett Myers and Jeremy Guthrie, and good for 30th out of 250 pitchers. But actually, Verlander's worse-than-expected ranking just goes to demonstrate that last year's off-putting performance was more than just random fluctuation. His average fastball velocity dipped below 94 miles per hour for the first time in his career last year, which was reflected in the actual run value of his fastball. However, his reputation as a flamethrower is well-deserved. Limiting the sample to only the top quartile of a pitcher’s stuff knocks Ubaldo Jimenez out of the top five in favor of Verlander, who, at his best, might be the league's best. Verlander has a very large difference between his best stuff and his worst stuff. By the numbers, it looks like Scot Shields owns the widest disparity between best and worst stuff. The difference in Verlander's fastball last year, which clocked in at 94 MPH with eight inches of run and ten inches of rise, and his fastball this year, which has the same average movement (to the nearest inch), but has picked up two miles per hour in velocity, is .4 runs per 100 pitches in expected value. The actual difference in Verlander's fastball, according to fangraphs, is even greater than that, at just over a full run per 100 pitches. That uptick in velocity has made a big difference.

How about the best stuff on a per-100-pitch basis?

Name StuffRV/100
Brian Wilson -1.08
Matt Lindstrom -0.99
Jonathan Broxton -0.97
Joel Zumaya -0.96
Brandon Morrow -0.87
Heath Bell -0.85
Dennis Sarfate -0.83
Grant Balfour -0.78
Mariano Rivera -0.78
A.J. Burnett -0.73
Josh Geer 0.41
Edwar Ramirez 0.42
Livan Hernandez 0.48
Greg Maddux 0.53
Joe Nelson 0.56


Max-effort relievers and a couple of Yankees top the pitch-for-pitch list. Also, Dennis Sarfate has apparently followed the route of Daniel Cabrera—an Oriole with eyebrow-raising stuff who lost his velo and thus, likely his place in the Majors.

Greg Maddux really survived the latter part of his career on his pitching moxie. Even when I restrict the sample to the top 25% of his pitches, he continues to show below-average stuff. Nevertheless, he accumulated five WAR over the last couple years.

Come to think of it, setting a minimum of 1,000 pitches for this analysis might have been a mistake. Given the precision and granularity of the data, this technique could be used to assess a pitcher's stuff given only a handful of pitchers. For example, I had never heard of Carlos Rosa before conducting this analysis, but now, from a sample of just 50 pitches, I can’t stop wondering why he’s not in the Majors. Great stuff. Decent control. The only evident knocks against him are his 2-8 Win-Loss record in AAA and 4.56 ERA. Maybe Dayton Moore knows something I don’t, or perhaps Rosa brought it just for his brief appearance in the Majors, or it’s possible GMDM is undervaluing a young talent who can get Major League hitters out. Actually, all three of these scenarios have probably taken place.

Who has made the most of his stuff? For this ranking, I subtracted the actual run value that each pitcher has been worth from the expected run value based on his stuff.

Name StuffRV-RV
Roy Halladay 61
Brandon Webb 60
Derek Lowe 54
Dan Haren 51
Greg Maddux 46
Jake Peavy 45
Jonathan Papelbon 38
Justin Duchscherer 37
Joakim Soria 35
Brad Penny -28
Carlos Silva -30
Adam Eaton -32
Bronson Arroyo -32
Miguel Batista -36


This group consists of pitchers who can succeed without the strikeout. While Webb and Lowe both have heavy sinkers that are able to generate grounders like clockwork, the location of these pitches down in the zone has likely had more to do with their success than the tremendous movement on their pitches.

And at the bottom, there are two types of pitchers. The Batistas and Pennys are granted roster spots because their passable stuff at least gives them upside, but the Eatons and Silvas of the world, who have crummy stuff and can’t even pitch up to their already limited abilities, well, those are the guys to whom you don’t give multi-year deals.

A spreadsheet containing most of the data used in this article.

Pitchf/x data from wantlinux.net via MLBAM. Thanks to Dave Allen and all others who helped me with the code used in this analysis.

Comments

LOVELY ANALYSIS!!!

BTW - TOLAXOR FINDS IT FASCINATING THAT (AMONG OTHERS) CARPENTER & LINCECUM DON'T SHOW UP...

IT WOULD BE INTERESTING TO COMBINE YOUR METRICS WITH METRICS THAT SUGGEST "CONTROL" MEASUREMENTS FOR AN AGGREGATE "ABILITY" MEASUREMENT FOR ANY GIVEN PITCH...

Posted by: TOLAXOR at September 9, 2009 7:37 AM

Tolaxor,

Carpenter didn't show up because he only threw 300 pitches in 2007-2008, and my dataset does not include most of this year, when he's been outstanding. Lincecum not showing up is a clear flaw in my method. I don't have most of his pitches, and I would think that the reason is because he throws very much from over the top, and not from the normal arm angle that most overhand righties throw.

Control/location is certainly another component that could be added.

Posted by: Jeremy Greenhouse at September 9, 2009 8:52 AM

OHHHHH...

ALSO, IT WOULD BE FUN TO SEE IF STUFFRV EXHIBITS SIGNIFICANT CHANGE IN A PARTICULAR GAME, OR OVER THE COURSE OF A SEASON (HOW BADLY DOES "STUFF" DEGRADE, AND IF CERTAIN PITCHERS DEGRADE MORE RAPIDLY THAN OTHERS)...

AGAIN, LOVELY ANALYSIS!!!

Posted by: TOLAXOR at September 9, 2009 9:32 AM

Carlos Rosa is a September call-up for the Royals.

The story with him is he was originally expected to be a starter. Then he developed some arm issues last year and wound up on the DL in June after two major league appearances. He only pitched something like two innings in the minors the rest of the year. Durability doubts have always been there, so the Royals have now opted to make him a reliever. They left him in AAA so he could make the adjustment to the pen. It didn't go well at first, so they didn't hurry bringing him up. He did appear to finally settle in to the new role as the season wore on. I don't think they are undervaluing him at all, and I would expect that barring further health issues he will be in the majors from here on out.

Posted by: ecp at September 9, 2009 9:34 AM

It's certainly an interesting analysis. How far back does pitch f/x data go (that is, 'when did they have the cameras active in every ballpark, for every single game?'), and is it freely available?

I don't believe for a second that one can assign a run value to a pitch based entirely on its flight path. Filth and/or velocity aren't everything in the major leagues, control and keeping the batter on his toes are a necessity.

A high-90s fastball loses much of its effectiveness if it isn't mixed with good off-speed stuff, and the nastiest of hooks will be taken for a ball -- or taken for a ride -- if the batter is sitting on it.

Posted by: Jim D at September 9, 2009 10:00 AM

Tolaxor,

I did look at that for a couple pitchers I was interested in to see whether stuff degrades on a start by start basis or inning by inning basis, but I didn't see much worth noting on.

ecp, thanks a lot for the info. Good stuff.

Jim, they've had the cameras active in every ballpark, for every single game since 2008, and it is freely available.

You should definitely check out some of Dave Allen's and Josh Kalk's work to see the importance of pitch sequencing and mixing pitches.

Posted by: Jeremy Greenhouse at September 9, 2009 10:42 AM

What about Dave Robertson?

Posted by: Al at September 9, 2009 1:33 PM

Al, what an obscure pitcher. He must be your favorite player or something? Robertson comes in at about half a run better than average per 100 pitches. By my numbers, an average fastball of 92 MPH with 0 horizontal movement and 11 inches of vertical movement is the equivalent to a 96 MPH fastball with 9 inches of horizontal movement and 11 inches of vertical movement.

Posted by: Jeremy Greenhouse at September 9, 2009 1:55 PM

Good stuff Jeremy. Can you point me towards a description about how the "stuff" RV are calculated?

Posted by: Nick at September 9, 2009 2:19 PM

Nick, take a look at Dave Allen's presentation at the pitchf/x summit here: http://baseball.sportvision.com/summit/2009 for a background in loess regressions.

Posted by: Jeremy Greenhouse at September 9, 2009 2:27 PM

Couldn't you combine this with a "control" measurement and speed? Strike frequency or pitches in the black or something. Pitching is speed plus movement + location right? If you measure all three you should get an accurate ERA estimator or an accurate overall rank of the pitcher. You could figure out how much weight to put on each of those three components so that they add up to the most accurate total ability for a pitcher. As you get career pitch f/x lines you could also figure out which of them develops in young pitchers over time. If a guy already has minimal speed and ok stuff and great control, can he be expected to improve at all given that it is likely hard to improve speed and stuff?

Posted by: KY at September 10, 2009 11:56 AM

Very interesting, but as the first commenter noted, the lack of Lincecum on the list is pretty curious.

I was also going to throw in Matt Cain but I see from your spreadsheet (thanks, BTW) that he is up there, just not on top, with -17 StuffRV vs. -21 actual RV.

About Stuff but with limited time in the majors, I ran across a former Giants farmhand, Billy Sadler. He is pretty up there in StuffRV/100 (-0.57) but the reason he never stuck in the majors is that despite his stuff, he could never get the ball within the strikezone consistently enough for the Giants to keep him up here, he has had horrible walks rates, both in the majors as well as in the minors. They gave him numerous tries up here and he never stuck.

I could say the same about another former Giants farmhand, David Aardsma, the human asterisk (he's now the first player alphabetically in the all time baseball directory). Always had a lot of stuff, just couldn't get it over the plate a lot. Even this year, his best season ever, he has walked a lot more guys than a pitcher normally should, except that he can strike out a ton too and finally got his walks down enough where he's valuable.

So that would be my observation for this study. Assuming this captures stuff adequately, stuff does not equate to success in the majors because if you cannot locate well enough, you will be always wild and walking too many. Looking at the top pitchers, walks had (and still are for some) always been their achilles heel, Turnbow, Brian Wilson, Zumaya, Jenks.

It would be interesting if you could devise a corresponding measure for accuracy/location, which when linked with StuffRV, separates the wheat from the chaff.

Posted by: obsessivegiantscompulsive at September 10, 2009 4:08 PM

On the last list (with Roy on top) I was wondering where Mariano Rivera fit in.
I know he's a great pitcher, but maybe from stuff it's not much.
Thanks, great post.

Posted by: Thomas Robust at September 12, 2009 3:48 PM