Designated HitterMarch 17, 2005
The Mazzone Effect Revisited
By J.C. Bradbury

Back in the early days of the off-season, like many of my fellow Braves fans, I began to ponder the Braves pitching situation for the upcoming season. The team was about to lose three members of the rotation along with several effective members of the bullpen. It looked like the Braves would be rebuilding a pitching staff with spare parts once again.

With the exception of the "Big-3" of the 1990s (Glavine, Maddux, and Smoltz), the Braves pitching staff is always in motion. This is not much different from many other teams, but few clubs with such shuffling have had the sustained pitching success of the Braves. And much of the success has come from players not deemed worthy of roster spots on other big league clubs. The Braves have made a habit of turning cast-off non-roster invites into big-name free agents.

Clearly, the Braves are doing something right, and much of the credit has gone to pitching coach Leo Mazzone. The table below lists the mean rank and the mean of each team's ERA since 1991, Mazzone's first full year has the Braves pitching coach.

Team	Mean MLB     Mean	  Team	  Mean MLB    Mean
	ERA Rank     ERA	           ERA Rank    ERA
-----+-----------+----------+--------+-----------+--------
ATL	2.18	    3.50	  PIT	  15.04	     4.37
LAD	4.57	    3.75	  OAK	  16.14	     4.49
NYM	8.04	    4.01	  ANA	  16.39	     4.46
STL	9.36	    4.12	  SEA	  16.75	     4.49
HOU	9.50	    4.02	  CHW	  16.96	     4.45
MON	10.04	    4.09	  TOR	  17.21	     4.48
ARI	10.79	    4.20	  CLE	  17.54	     4.50
SFG	10.89	    4.15	  MIL	  18.21	     4.56
SDP	12.00	    4.21	  BAL	  18.57	     4.56
BOS	12.57	    4.25	  MIN	  19.61	     4.70
FLA	13.42	    4.35	  KCR	  20.96	     4.72
CHC	13.50	    4.28	  TBD	  22.29	     4.89
NYY	13.50	    4.26	  TEX	  23.50	     4.92
CIN	13.86	    4.32	  DET	  24.57	     5.04
PHI	14.61	    4.32	  COL	  27.46	     5.32
			  League		     4.38

There's no doubt that the Braves have had the best pitching staff in baseball since Mazzone became the organization's big league coach. But, just as I was becoming complacently confident in the Braves ability to succeed with a new group of pitchers, I began to have doubts. Is Leo Mazzone as good as we think he is? We all remember Jaret Wright, John Burkett, and Damian Moss; but what about Albie Lopez, Odalis Perez, and Jason Schmidt? Maybe, we're just remembering the success stories. Maybe some of the guys who were successful with the Braves just got a bit lucky during their tenure. Even if the success under Mazzone is for real, maybe John Schuerholz's keen ability to nab good pitchers or Bobby Cox's clubhouse management are the real secrets to success. I needed to know. (Little did I know that the Braves would soon make some bold moves by adding Tim Hudson and John Smoltz to the rotation, and acquiring Danny Kolb for the pen.)

With this in mind, I set out on a quest to analyze how much better pitchers have been with Mazzone than without. I looked at every pitcher who had pitched at least one full year for Mazzone, and compared their seasonal ERAs with and without Mazzone as their pitching coach. In December of last year I posted some rough results on my weblog, Sabernomics. The verdict: having Leo Mazzone as a pitching coach lowered a pitcher's ERA by a little more than half a run. This estimate controlled for many potential biasing factors. To my surprise the results set off a chain-reaction of follow-up studies at Baseball Think Factory, which largely confirmed the robustness of my initial findings.

Since I first crunched the numbers, I haven't had much time to work with the data in greater depth. Thanks to a kind invitation from Rich and Bryan, I have a reason to revisit the issue. In this article I discuss an expansion of my previous approach looking at the difference in Mazzone's effectiveness on starters and relievers. One of the supposed keys to Mazzone's success is an off-day throwing program for starters. If this is his "secret" to success, the Mazzone effect should be more pronounced for starters.

To begin, I used The Lahman Baseball Archive to identify all pitchers who pitched at least one full season for Leo Mazzone over their careers. I used a sample of pitchers who pitched for both Mazzone and a different pitching coach, and I only looked at player-seasons in which the player stayed on the same team for a full season. Using this data, I compared the seasons in which the pitchers pitched for Mazzone to the seasons without his oversight. To control for other influences, I used multiple regression analysis -- including control variables for age, the run environment of the league, the career quality of the pitcher, and the defense behind the pitcher.

Defining a pitcher as a starter or a reliever is a bit tricky, because some pitchers do a little of both over the course of the season. After some toying around with the data I settled on the following definitions. For this study, a starter was a pitcher who pitched at least 100 innings and started 75% of the games played in that season. A reliever must have pitched at least 30 innings and started in only 25% or less of the games in which he played that season. I tried several other starter/reliever measures, but they yielded results that were not meaningfully different from the ones I present below.

Here are the regression-estimated impacts of Leo Mazzone on park-adjusted ERAs for all pitcher-seasons in the sample (a minimum 30 innings pitched for a pitcher-season) and separated into starters and relievers. For simplicity, I only report the coefficient estimates (which are statistically significant) on Mazzone's presence as the pitching coach, the number of pitchers, and number of pitcher-seasons in the sample. You can view the full regression results and technical notes here.

Pitcher	        Impact	 Pitchers   Pitcher
Classification   on ERA	 in Sample  Seasons
--------------+------------+----------+----------
All              -0.625	  98	   694
Starters         -0.412	  22	   152
Relievers        -0.676	  56	   248

When looking at all of the pitchers in the sample, Leo Mazzone's presence lowered a pitcher's ERA by about 0.63 ERA points. To put the effect in perspective, for the average 2004 National League pitcher (4.31 ERA) Leo's impact on earned runs was about the same as Coors Field in the opposite direction. Note to Dan O'Dowd: take the balls out of the humidor and hire Leo Mazzone.

When separating pitchers into their defined roles, relievers appeared to benefit more from their time under Mazzone than starters did, though not by much. For starters, having Leo Mazzone as a pitching coach was worth about 0.41 earned runs per 9 innings or 1 earned run per 22 innings. For relievers, Mazzone was good for about a 0.68 reduction in earned runs per 9 innings, or 1 run per 13 innings. It's pretty clear that he helps both classes of pitchers quite a bit, but he seems to do a little bit more for relievers than starters.

What about the ability of players to retain what they have learned from Leo? Should we expect success under Mazzone to continue? And what if the success under Mazzone is the result of John Schuerholz identifying good pitchers before they become Leo's responsibility? One of the results from my earlier study was that players tended to pitch worse both before and after pitching for Leo. So, I ran a second set of regressions with indicator variables for seasons pitched before and after working under Leo.

 
Pitcher	        ERA	     ERA
Classification   Before Leo     After Leo
--------------+------------+-------------
All              0.625	     0.624
Starters         0.367	     0.461
Relievers        0.747	     0.559

Starters and relievers pitched worse both before and after playing for Mazzone. Something good was clearly happening when pitchers played for the Braves. One noticeable difference between starters and relievers was that the Before Leo impact for starters was smaller in magnitude. Why might this be? I considered the possibility that the long run of Glavine, Smoltz, and Maddux in the Braves rotation, all of whom had some pitching success before Leo arrived, might be the culprit. When I dropped the Big-3 from the analysis, the estimated impact was the same, though the before Leo variable was just barely statistically insignificant. The smaller impact on ERA from seasons before Leo may reflect John Schuerholz's ability to target good starters that make Mazzone's job easier as well as some of the impact of the Big-3.

But, there is no doubt that pitchers of both classifications suffered when they left Atlanta. If Leo is just spotting problems and fixing them, pitchers should retain the advantage they received when they join the team. It turns out that whatever inspiration Leo provides, you can't take it with you. Whether it's a lack of fine-tuning by Mazzone or Braves management just knows when it's time to cut a guy loose, I can't say. No matter what the cause, these numbers do not bode well for the Angels, Diamondbacks, and Yankees who acquired Paul Byrd, Russ Ortiz, and Jaret Wright in the off-season. While all of these players may provide solid pitching performances for their new clubs, I don't expect their performances to be as good as they were for the Braves.

The fact that pitchers seem to lose the Leo magic when they leave the Braves indicates to me that part of the Mazzone method involves handling pitchers during the game. The larger ERA gains for relievers over starters are consistent with this hypothesis. If this is the case, then manager Bobby Cox deserves some credit as well. It would be interesting to study the in-game use of pitchers by the Braves, and how their strategy differs from other organizations. I bet Studes could figure out a way to do this with Win Probability Added.

One real problem with determining exactly who is responsible for the success of Braves pitching is that the team has had stable management over this era. John Schuerholz and Bobby Cox have overseen the Braves pitchers along with Mazzone, so I can't isolate their individual influences. However, I don't think this really matters. Leo is in charge of the pitchers, and the rest of the management over him has stuck by him; therefore, I feel comfortable attributing much of this success to Mazzone. But, I want to acknowledge the potential contributions of Cox and Schuerholz to the Braves pitching consistency.

So where does this leave us? Well, Mazzone is a pretty darn good pitching coach. I think we already knew that, but now we really know it. Leo's influence seems to extend beyond a simple off-field training strategy for his pitchers, and both starters and relievers benefit from his oversight. If the powers that be ever decide to open the Hall doors to pitching coaches, Leo Mazzone has a very strong case.

J.C. Bradbury is an economist who runs the weblog Sabernomics: Economic Thinking About Baseball.

[Additional reader comments and retorts at Baseball Primer.]

Comments

As a long-time Braves fan, I would like to make a few comments.

Though people talk about Mazzone's throwing program, I don't think that's the reason for his success. He has a comprehensive pitching philosophy and I think that's the reason for success. From what I've been able to gather over the years from listening to interviews with Mazzone and reading articles about him, here are some of the keys to his pitching philosophy.

One, he wants his pitchers to own the lower outside corner. He's come out and said that. It's a sound point. That's the place that's hardest for the umpire to see and, if the pitcher can hit the catcher's glove down there, even if the pitch is off the plate, he often gets a strike call. Tom Glavine has made a career of that lower outside corner.

Two, Mazzone feels that the fastball is a pitcher's most effective pitch and the first thing he wants a pitcher to be able to do is throw it for strikes. If that means backing off of the velocity a bit, then so be it. Mazzone seems to believe that, if a pitcher cannot consistently throw his fastball for strikes, then it doesn't matter what else he throws.

Three, if a pitcher throws a lot of different pitches, Mazzone insists that he eliminate all but his three or four best pitches.

Mazzone made a success out of Jaret Wright by convincing him to take a few mph off of his fastball and throw it for strikes on the outside corner. Last year, the Braves' announcers made that point in almost every one of Wright's appearances.

As for Mazzone's failures, a good many of them are pitchers who refuse to listen to him. Albie Lopez was one of those. He came to Atlanta throwing lots of different pitches and he liked to challenge the batter, even though he didn't have the stuff to do that successfully with any great frequency. He made a half-hearted attempt to adopt Mazzone's approach then, in mid-season, announced that he was more comfortable pitching the way he used to pitch. He was then relegated to mop-up duty and was out of Atlanta at season's end. Another failure was Jason Marquis. He is apparently something of a stubborn bonehead who wouldn't listen and I think that's why they gave up on him. My opinion was confirmed for me when I saw a Fox telecast of a Cardinals game last year in which they talked about Marquis having difficulties with Dave Duncan because he wouldn't listen to what Duncan told him.

I wonder how much of the drop in ERA--what small amount that is--might be attributable to Cox's use patterns, both in terms of pitch counts and roles. Cox sure seems to know when to make a starter into a reliever for instance.

It seems worthwhile for someone to look at whether the difference is in early innings, late innings or across the board (at least for starters). Late innings improvement could indicate better judgement at when to pull a pitcher. A 1/2 run in ERA could be pulling the starter 1/3 inning sooner every fourth game, before he allows a 2 run dinger. That could be attributable to ome combo of Cox and Mazzone. Early innings would be better preparation and more directly attributable to Mazzone. The numbers would be small for any individual, but combined it should be sufficient across the decade+

One of the things that the Braves do religiously is limit the innings of their starters in blowout games. Cox and Mazzone have a fairly set pattern of using the guys in the back of the bullpen for an inning at a time from the seventh inning on when they have a lead of four or more runs. This serves a lot of purposes - the starters stay fresher, the back end of the bullpen guys get enough work to keep themsevles sharp, and the front end of the bullpen isn't overworked.

Is this really enough, though? How many of these guys moved to lame hitter's parks? I suppose a half-run is enough to prove the case broadly, but I'd be interested in seeing dERA for these guys before and after the Mazzone treatment.

Thanks for the comments guys. I'll respond to a few thoughts.

I think that usage pattens by Mazzone and Cox are a major factor in both in the short-run and over the season. Mike makes a good point that in blowout games the starter comes out. And the Braves always seem to keep a long-man on the roster just for mopping up.

I would love to look at in-game data to see exactly what it is the Braves are doing well. The early/late inning idea is right on. Unfortunately, this data just isn't available for free. I normally watch most every Braves game, so maybe I'll try and record pitcher usage this year.

Pitcher ERA is corrected for park effects, so it's not just that some players are coming from and moving to more hitter-friendly parks. I did run some models using Tango Tiger's FIP ERA, and the results were pretty similar. In the Primer thread for the original article, dks reports the results of several estimates using FIP ERA. The results there are quite similar.

Thanks again for the comments! And thanks to Rich and Bryan for the invite.

Thanks, JC, for sharing your findings with us. Your article was very informative and enlightening. Bryan and I hope you will choose to contribute to Baseball Analysts again in the future.

excellent stuuf - would that I were a stat geek so I could take your approach and try it out on other pitching coaches with good reps (and bad).

Perhaps I can talk you into seeing what the impact of working with other coaches is?

According to my calculations, if Leo Mazzone had been the pitching coach of the Baltimore Orioles in 2004, they would have won 80 games instead of 74.

I used team splits between starting/relieving and the "Mazzone Effect" numbers above to determine a new ER number then added the old number of unearned runs to get a new RA total and then applied the Pythagorean Formula to get 80 wins.

Of course, last year the formula predicts the Orioles to get 77 wins.

So Leo Mazzone's worth 3 wins (above replacement?) to the Orioles.

Players with 29-31 WS? David Ortiz, Carlos Delgado, Jeff Kent, Miguel Cabrera, Morgan Ensberg

Players with 30 WS(Above Baseline)? None. Pujols has 25 WSAB.

Well, I think we're about to see next season just how much of an impact Leo has made. I can't blame him for accepting a job to manage with the guy who was Best Man at his wedding. But I cannot accept that the Atlanta brass didn't think enough of the situation to break their long-standing rule of not offering multi-year deals to coaches. If anyone deserved to be an exception, it's Leo. Mel Stottlemyre appears to be a leading candidate, but after so many years, it won't be the same without Coach Mazzone.