The Mazzone Effect Revisited
Back in the early days of the off-season, like many of my fellow Braves fans, I began to ponder the Braves pitching situation for the upcoming season. The team was about to lose three members of the rotation along with several effective members of the bullpen. It looked like the Braves would be rebuilding a pitching staff with spare parts once again.
With the exception of the "Big-3" of the 1990s (Glavine, Maddux, and Smoltz), the Braves pitching staff is always in motion. This is not much different from many other teams, but few clubs with such shuffling have had the sustained pitching success of the Braves. And much of the success has come from players not deemed worthy of roster spots on other big league clubs. The Braves have made a habit of turning cast-off non-roster invites into big-name free agents.
Clearly, the Braves are doing something right, and much of the credit has gone to pitching coach Leo Mazzone. The table below lists the mean rank and the mean of each team's ERA since 1991, Mazzone's first full year has the Braves pitching coach.
Team Mean MLB Mean Team Mean MLB Mean
ERA Rank ERA ERA Rank ERA
ATL 2.18 3.50 PIT 15.04 4.37
LAD 4.57 3.75 OAK 16.14 4.49
NYM 8.04 4.01 ANA 16.39 4.46
STL 9.36 4.12 SEA 16.75 4.49
HOU 9.50 4.02 CHW 16.96 4.45
MON 10.04 4.09 TOR 17.21 4.48
ARI 10.79 4.20 CLE 17.54 4.50
SFG 10.89 4.15 MIL 18.21 4.56
SDP 12.00 4.21 BAL 18.57 4.56
BOS 12.57 4.25 MIN 19.61 4.70
FLA 13.42 4.35 KCR 20.96 4.72
CHC 13.50 4.28 TBD 22.29 4.89
NYY 13.50 4.26 TEX 23.50 4.92
CIN 13.86 4.32 DET 24.57 5.04
PHI 14.61 4.32 COL 27.46 5.32
There's no doubt that the Braves have had the best pitching staff in baseball since Mazzone became the organization's big league coach. But, just as I was becoming complacently confident in the Braves ability to succeed with a new group of pitchers, I began to have doubts. Is Leo Mazzone as good as we think he is? We all remember Jaret Wright, John Burkett, and Damian Moss; but what about Albie Lopez, Odalis Perez, and Jason Schmidt? Maybe, we're just remembering the success stories. Maybe some of the guys who were successful with the Braves just got a bit lucky during their tenure. Even if the success under Mazzone is for real, maybe John Schuerholz's keen ability to nab good pitchers or Bobby Cox's clubhouse management are the real secrets to success. I needed to know. (Little did I know that the Braves would soon make some bold moves by adding Tim Hudson and John Smoltz to the rotation, and acquiring Danny Kolb for the pen.)
With this in mind, I set out on a quest to analyze how much better pitchers have been with Mazzone than without. I looked at every pitcher who had pitched at least one full year for Mazzone, and compared their seasonal ERAs with and without Mazzone as their pitching coach. In December of last year I posted some rough results on my weblog, Sabernomics. The verdict: having Leo Mazzone as a pitching coach lowered a pitcher's ERA by a little more than half a run. This estimate controlled for many potential biasing factors. To my surprise the results set off a chain-reaction of follow-up studies at Baseball Think Factory, which largely confirmed the robustness of my initial findings.
Since I first crunched the numbers, I haven't had much time to work with the data in greater depth. Thanks to a kind invitation from Rich and Bryan, I have a reason to revisit the issue. In this article I discuss an expansion of my previous approach looking at the difference in Mazzone's effectiveness on starters and relievers. One of the supposed keys to Mazzone's success is an off-day throwing program for starters. If this is his "secret" to success, the Mazzone effect should be more pronounced for starters.
To begin, I used The Lahman Baseball Archive to identify all pitchers who pitched at least one full season for Leo Mazzone over their careers. I used a sample of pitchers who pitched for both Mazzone and a different pitching coach, and I only looked at player-seasons in which the player stayed on the same team for a full season. Using this data, I compared the seasons in which the pitchers pitched for Mazzone to the seasons without his oversight. To control for other influences, I used multiple regression analysis -- including control variables for age, the run environment of the league, the career quality of the pitcher, and the defense behind the pitcher.
Defining a pitcher as a starter or a reliever is a bit tricky, because some pitchers do a little of both over the course of the season. After some toying around with the data I settled on the following definitions. For this study, a starter was a pitcher who pitched at least 100 innings and started 75% of the games played in that season. A reliever must have pitched at least 30 innings and started in only 25% or less of the games in which he played that season. I tried several other starter/reliever measures, but they yielded results that were not meaningfully different from the ones I present below.
Here are the regression-estimated impacts of Leo Mazzone on park-adjusted ERAs for all pitcher-seasons in the sample (a minimum 30 innings pitched for a pitcher-season) and separated into starters and relievers. For simplicity, I only report the coefficient estimates (which are statistically significant) on Mazzone's presence as the pitching coach, the number of pitchers, and number of pitcher-seasons in the sample. You can view the full regression results and technical notes here.
Pitcher Impact Pitchers Pitcher
Classification on ERA in Sample Seasons
All -0.625 98 694
Starters -0.412 22 152
Relievers -0.676 56 248
When looking at all of the pitchers in the sample, Leo Mazzone's presence lowered a pitcher's ERA by about 0.63 ERA points. To put the effect in perspective, for the average 2004 National League pitcher (4.31 ERA) Leo's impact on earned runs was about the same as Coors Field in the opposite direction. Note to Dan O'Dowd: take the balls out of the humidor and hire Leo Mazzone.
When separating pitchers into their defined roles, relievers appeared to benefit more from their time under Mazzone than starters did, though not by much. For starters, having Leo Mazzone as a pitching coach was worth about 0.41 earned runs per 9 innings or 1 earned run per 22 innings. For relievers, Mazzone was good for about a 0.68 reduction in earned runs per 9 innings, or 1 run per 13 innings. It's pretty clear that he helps both classes of pitchers quite a bit, but he seems to do a little bit more for relievers than starters.
What about the ability of players to retain what they have learned from Leo? Should we expect success under Mazzone to continue? And what if the success under Mazzone is the result of John Schuerholz identifying good pitchers before they become Leo's responsibility? One of the results from my earlier study was that players tended to pitch worse both before and after pitching for Leo. So, I ran a second set of regressions with indicator variables for seasons pitched before and after working under Leo.
Pitcher ERA ERA
Classification Before Leo After Leo
All 0.625 0.624
Starters 0.367 0.461
Relievers 0.747 0.559
Starters and relievers pitched worse both before and after playing for Mazzone. Something good was clearly happening when pitchers played for the Braves. One noticeable difference between starters and relievers was that the Before Leo impact for starters was smaller in magnitude. Why might this be? I considered the possibility that the long run of Glavine, Smoltz, and Maddux in the Braves rotation, all of whom had some pitching success before Leo arrived, might be the culprit. When I dropped the Big-3 from the analysis, the estimated impact was the same, though the before Leo variable was just barely statistically insignificant. The smaller impact on ERA from seasons before Leo may reflect John Schuerholz's ability to target good starters that make Mazzone's job easier as well as some of the impact of the Big-3.
But, there is no doubt that pitchers of both classifications suffered when they left Atlanta. If Leo is just spotting problems and fixing them, pitchers should retain the advantage they received when they join the team. It turns out that whatever inspiration Leo provides, you can't take it with you. Whether it's a lack of fine-tuning by Mazzone or Braves management just knows when it's time to cut a guy loose, I can't say. No matter what the cause, these numbers do not bode well for the Angels, Diamondbacks, and Yankees who acquired Paul Byrd, Russ Ortiz, and Jaret Wright in the off-season. While all of these players may provide solid pitching performances for their new clubs, I don't expect their performances to be as good as they were for the Braves.
The fact that pitchers seem to lose the Leo magic when they leave the Braves indicates to me that part of the Mazzone method involves handling pitchers during the game. The larger ERA gains for relievers over starters are consistent with this hypothesis. If this is the case, then manager Bobby Cox deserves some credit as well. It would be interesting to study the in-game use of pitchers by the Braves, and how their strategy differs from other organizations. I bet Studes could figure out a way to do this with Win Probability Added.
One real problem with determining exactly who is responsible for the success of Braves pitching is that the team has had stable management over this era. John Schuerholz and Bobby Cox have overseen the Braves pitchers along with Mazzone, so I can't isolate their individual influences. However, I don't think this really matters. Leo is in charge of the pitchers, and the rest of the management over him has stuck by him; therefore, I feel comfortable attributing much of this success to Mazzone. But, I want to acknowledge the potential contributions of Cox and Schuerholz to the Braves pitching consistency.
So where does this leave us? Well, Mazzone is a pretty darn good pitching coach. I think we already knew that, but now we really know it. Leo's influence seems to extend beyond a simple off-field training strategy for his pitchers, and both starters and relievers benefit from his oversight. If the powers that be ever decide to open the Hall doors to pitching coaches, Leo Mazzone has a very strong case.
J.C. Bradbury is an economist who runs the weblog Sabernomics: Economic Thinking About Baseball.
[Additional reader comments and retorts at Baseball Primer.]