So, what does Kyle Farnsworth do to lower his FIP in high-leverage situations? Because since joining the Yankees he's lost 9 games and has blown four saves, as a setup man. Yankees fans are pretty fed up with him, yet this suggests that all his walks and homers are but an illusion...

Does this study take into account whether a pitcher's own performance impacts the leverage of the situation. For example, if Farnsworth loads the bases and then proceeds to get out of the jam, would he get credit for being "clutch"? Similarly, if Rivera mows down the side, does that lessen the leverage of his performance? Unlike hitters, pitchers can increase their leverage by failing. Without taking that into account, this study seems seriously flawed.

Also, it's a little silly to call Johan "unclutch" based solely on his own high standards in "low leverage" situations. After all, when a dominant starter like Johan is on his game, very few hitters reach base, so the leverage is necessarily lower. That pretty much ensures he'll perform "worse" in high leverage, even though that "worse" performance is still better than the "clutch" pitchers identified.

mehmattski: I'm not saying Farnsworth has been great for the Yankees, but he has performed better in high-leverage situations than he did in "average or low" leverage situations. His WPA over the past couple years with the Yankees has been positive so he hasn't been totally useless.

Will: This study does not take into account whether or not a pitcher got himself into the jam in the first place. Starting pitchers can pretty much only get themselves into high leverage situations and I think it's fair to ask if a pitcher gets himself into a jam, is he "clutch" enough to get himself out of it?

Also, just because there are runners on base does not necessarily mean Johan will perform worse in those situations. There are lots of starting pitchers who perform better in those situations. Johan performs very well in both situations, but over the past 6 years he's definitely been worse.

FIP doesn't take runs into account, so we're just looking at strikeouts, walks, hbp, and home runs.

I think in general our definition of "clutch" may differ. When I think of "clutch", I ask myself if that player has raised the level of his game from where he usually is, not if that player has performed well compared to everyone else in high leverage situations. So it's the difference between a self comparison and a mlb/league wide comparison.

This is not so much a comment on your analysis as it is on the use of FIP as a measure of pitcher performance. I understand that pitchers have little control over what happens to balls in play. But does it make sense to assign full responsibility to the pitcher for every HR he gives up but absolve him of all blame for long flys or line drives that bounce off the fence for doubles or triples? Some of those hits would certainly have been HR's under different conditions: park, wind, temperature, etc. Conversely, some of the HR's could have landed on the warning track or even been caught. For that matter, if Torii Hunter "brings one back" should we pretend it never happened as far as the pitcher is concerned? Wouldn't it make more sense to use the pitcher's mix of results on balls in play (GB, IF, OF, LD) times the league average results for each to determine his FIP?

James, I think what you're looking for is something like xFIP or batted ball DIPS

What exactly do you mean by ERA doesn't work well for starting pitchers? Were the results just not what you expected. I'm not sure FIP is the most reliable statistic here. What about WHIP, K/9 or such?

Even going by FIP the general assumptions of what is being considered clutch are a bit preposterous. Farnsworth's best FIP (in high leverage situations) is still worse than Papelbon's everything else.

It's hard to call someone clutch if they're only delivering mediocre quality. By using difference as the final judge, you're not filtering out guys who still stunk even when at their best.

Jeff, this is where you get into the differences in what you consider clutch.

I wrote this in a previous comment: "When I think of "clutch", I ask myself if that player has raised the level of his game from where he usually is, not if that player has performed well compared to everyone else in high leverage situations. So it's the difference between a self comparison and a mlb/league wide comparison."

I'm not really sure how else to put it, it's just a difference in what we consider clutch. If I were to look at your definition, you'd most likely have the best pitchers in general at the top of the list and the worst pitchers in general at the bottom of the list. Of course Papelbon is going to pitch better in clutch situations than Farnsworth, he's a better pitcher.

The reason ERA doesn't really work for starting pitchers is that typically they are pitching in lower leverage situations and many of the situations they can give up runs in are the high ones. It inflates their ERA in the high leverage situations considerably and that hardly seemed like a fair comparison.

I'm curious why you think FIP doesn't work here. It's basically saying, here are the pitcher's K's, BB's, HR's, and HBP. He should have had this ERA. It's more or less a skill measurement and much more so than WHIP, or just K/9 or BB/9 on their own.

Dave,

Now that you've explained it I see why you think ERA would not being appropriate given the way you've structured your analysis.

Regarding clutch definitions, performance in clutch situations is the defining factor in my opinion, regardless of how well they do normally. When we talk about clutch performances, to me it means which guy do you want in when the game is on the line. Who will perform best when the game is on the line, who can be counted on in pressure situations. Being able to elevate your game to a below league average level is of little use to a manager in game 7 of the World Series.

I don't care for FIP because it includes a few too many things (without including base hits) given various weights and manipulated to look like ERA. WHIP is pretty simplistic and gives a less complicated look at two fundamental factors, walks and hits. (K rate would be interesting to look at for clutch performances, as the strikeout is something many consider to be a sign of a pitcher dominating a hitter, although it most certainly doesn't tell the whole story).

Since I have an entirely different definition about what clutch means, I'd personally like to look at things like runs allowed in high leverage situations to determine who was a clutch pitcher in my book.

Jeff, if you haven't seen it, you might be interested in Tangotiger's current clutch study, which is being tracked on FanGraphs.

http://www.tangotiger.net/clutch/
http://www.fangraphs.com/clutch.aspx

It's about who the fans think should be at bat when the game is on the line, versus who Tangotiger's Marcels project is the best player on the team.

David - Interesting links, thanks. I've bookmarked them and will follow.

First, I like the fact that you are examining the question of "clutch performance" as it might apply to pitchers. If "clutch" ability exists among hitters, then one must also assume that it exists for pitchers. In fact, I assume it is easier for a pitcher "to reach back for something extra" in the clutch situation than it is for a hitter to increase his physical skills in the situation. And, that makes me question the validity of conclusions which have been reached about clutch hitting. In effect, only one side of the match up is considered, and the studies implicitly assume that only the hitter might be capable of clutch ability.

Second, I'm not comfortable with using FIP as the measurement of performance in clutch situations. I understand why you didn't use an ERA comparison, and I can't say that I have a good suggestion for such a comparison right now. (One possibility comes to mind: ERA with runners on base when the game score is close vs. ERA with runners on base when the game score is not close.) It seems to me that the most successful outcome for a pitcher in a clutch match up is situaton-specific, e.g., which bases are occupied and the number of outs, and what is the potential effect on the game score. Some pitchers might use different strategies in clutch situations, such as trying to induce a double play, inducing a groundball (to avoid a sac fly), and pitching around stronger hitters (particularly if a base is available). Adopting these strategies might lead to more successful results in clutch situations but may actually increase their FIP measure, relative to their normal pitching approach.

It might be interesting to look at other measures such as Batting Average Against, SLG% Against, and Groundball rate in clutch vs. non-clutch situations.

Interesting that 4 of the 10 "clutch" starters were members of the 2004 and 2005 Cardinals rotations. Another feather in Dave Duncan's cap?

Jeff K,

Sure, in Game 7 of the WS you want a player who you can depend on to produce. However, succeeding when the game is on the line may have little to do with "clutch" but rather simply with the fact that the player is "good." You know why Billy Wagner gets the ball in high leverage situations and Farnsworth doesn't? Because Wagner is GOOD and Farnsworth isn't! The above study simply measures a player's ability to perform above or below his own standards when the pressure is on.

Rather than FIP, I'd suggest a measure of batter effectiveness against that pitcher in "clutch" situations, such as OPS+ or some other measure that better accounts for on base percentage. If clutch pitching exists, I want to know if my pitcher is going to shut down the opposing batter in clutch situations.

It is also worth mentioning the limitations of using Leverage Index to measure clutch situations for pitchers v. hitters. Since LI is based exclusively on base, runs, innings situations, it does not take into account the quality of opposition. Whereas, hitters are likely to be facing at least a fresh, same-handed reliever, if not a teams best reliever, in many, if not most, high LI circumstances, a pitcher is just as likely to be facing the bottom third of the order (or the best bench bat), as one of the opposing teams more fearsome hitters. Due to the small sample sizes involved, this could be a substantial confounding factor in a study of this nature.

Jason Isringhausen is the master of the white-knuckle save. If you use only his stats, he looks awful--but he often walks a couple of batters and then gets the side out. Put him in with a 3-run lead, he'll give up two. But he gets the save a lot more often than he blows it.

This is really a question of semantics I suppose, but just looking at the tables laid out, I notice that the mean low LI FIP for the top clutch pitchers is probably about a point higher than for the not-clutch pitchers....That seems to indicate that a low FIP pitcher is more likely to be "un-clutch" than a high FIP pitcher.

Awesome piece. I've been thinking about this concept for the past couple of weeks but have been too lazy to do anything about it.
I've also been wondering about whether clutch fielding and clutch baserunning/basestealing exist, and if they do, how to quantify them. Anyone seen any studies on this?