The Value of a Good Farm System
Baseball America's farm system rankings are one of the most respected rankings of a club's minor league talent around. Since 1984, they've been rating and ranking minor league systems in terms of their potential for major league impact. In this post, I try to determine just how much of an impact a team's farm system has on future performance.
Recently, the Baseball America came out with its December farm system rankings. Baseball America had the Houston Astros dead last, while the Rangers were ranked #1. If you're a Rangers fan, you might be smiling ear to ear, believing that the Rangers, who were also ranked #1 in 2009, would be poised for a long-term dynasty. Meanwhile Astros fans might despair, knowing that good young talent is not on the way.
But really, how predictive are these rankings? Does a good ranking actually lead to future success? If so, just how much?
To test this, I obtained Baseball America's organizational rankings from 1984-2010. I first transformed the rankings into a ratings, assuming that teams' minor league talent was normally distributed. This reflected the likely reality that the difference between having the #13 and #17 farm system is pretty small, but the difference between the #1 and #5 farm system is quite large. Transforming the ratings into normally distributed scores (which range from about -2.1 to 2.1) reflects this nicely.
I then used statistical regression to find the relationship between Baseball America ratings and team winning percentage. Doing a simple, single-term linear regression, it appears that the Baseball America rankings have predictive power for many years forward. One year's Baseball America ranking has a statistically significant effect on winning percentage for each of the next 8 years. As you would expect, those with higher rankings will tend to do better. If the only information you have is a team's 2010 Baseball America ranking, you would predict that a team with good rankings now will have an advantage come 2018.
But of course, we have more information than that. To really get at the heart of the matter, we need to take into account potential confounding variables. We can take these into account by using a multiple regression. To predict the next year's WPCT, significant important factors were:
a) WPCT from last year
Now, to test the effect of farm systems, we can add in the Baseball America rankings data. When we do, we get an interesting, yet difficult to interpret model, the results being the following:
Clearly the salary and previous winning percentage variables are the main predictors of a team's success in a season, with market size close to significant. Less clear is the Baseball America rankings, which don't have a clear pattern. The years with most predictive power are the rankings from the previous season and from four seasons ago. Rankings from two years ago and from seven years ago show some predictive power, but not a lot. Meanwhile the other years show very little predictive power, with the effect being negative in some years.
The reason for this volatility of course is that the sample size is fairly small, so the estimates are not all that accurate. While using these weights would give the best fit, it doesn't seem to make sense that a BA ranking from one or four years ago would have much more predictive value that the BA ranking from two or three years ago. What does appear clear however, is rankings from the previous four years combined have a pretty strong correlation with WPCT, while rankings from after that time, on the whole, don't really a strong much effect.
My imperfect solution, then is to put the average of the previous four years of BA rankings into the model. When I do this, I get the following result.
Overall, the values of the other terms are relatively unchanged, but we get a nice, highly significant, result for the Baseball America rankings. What does it all mean? Those ranked as the #1 farm system for the previous four years would get the maximum Baseball America score of 2.1. Multiplying 2.1 by .0155 gives means that it would be expected to add about .033 points to its WPCT in the next season. That translates to about 5.3 wins. Now five and a half wins is nothing to sneeze at, but it’s also not an enormous factor. Teams with weak farm systems do take a hit in future production, but it's certainly not insurmountable. The Astros, ranked last now for three consecutive years, figure to take a hit of 3.3 wins in 2010 and 4.4 wins in 2011. While that's certainly not desirable, there's no reason they still can't compete in the coming years, despite a poor farm system.
The model can be extended to predict values further into the future as well. Using only known, WPCT's, salaries, market size, and Baseball America rankings, we can build models for years down the road. For instance, using only known 2010 variables, how many wins does the #1 farm system provide in 2015? The models show that being the best farm system in 2010 correlates to about 4 extra wins in 2015.
The Rangers should feel good, but not get too overconfident, despite having the #1 system in both 2009 and 2010. The Rangers, who were ranked #1 in '09 and '10, were ranked #27 in 2008 and #15 in 2007. What do the models show the Rangers farm system producing over the next several years? The models predict the following boost in wins:
Since the Rangers' system was rated #27 as recently as 2008, the expected farm impact in 2010 is small. However, the impact increases dramatically starting in 2012. Overall, over the next 9 years, the Rangers farm system will likely net them 31 extra wins, meaning that while their system won't have a huge effect in any one particular year, it's likely to have a strong impact on the Rangers franchise over the next decade.
How about for their Texas counterpart, the Houston Astros? For them, the following 9-year outlook looks as follows:
For the Astros, it's nearly the opposite situation. Their farm system projects to cause them to lose over 36 games over the next ten years. So, is the difference between the Rangers and Astros farm systems really 67 wins over the next nine years? It would appear that way, although there are some caveats. For one, the year-to-year farm system rankings are correlated with one another, so the fact that the Rangers have a good farm system now is also indicative that they will have a good system in the future. That undoubtedly accounts for some of the large difference in wins. While the Rangers may not be still reaping fruit from their 2010 farm system in the year 2018, the fact that they have a good farm team now bodes well for their future farm teams, and hence their future major league teams.
Another factor to consider is how teams go about team-building. The fact that the Rangers have a good farm system means that they may be in strong contention in the next few years. With the team blossoming, this may spur the front-office to go out and sign free agents to supplement the team. Thus, the wins the future free agents provide are also correlated with the Rangers having a good farm team. While the Rangers may win more because of the free agents, this boost (reflected in these numbers) is not necessarily a direct product of having a good farm system in 2010.
For these reasons, I would hesitate to put a dollar value on having the #1 farm system in baseball vs. the #30 farm system in baseball - at least using this analysis. There are too many potential confounding variables here such as the ones I mentioned above. Still, if you are a fan, it matters little where your team's wins are coming from. Rangers fans really do have a reason to be smiling. While a handful of wins each year may not have a major impact, 30 wins over the next 9 season is a significant force. Whether the Rangers can parlay those wins into championships remains to be seen.
The following graph shows some trajectories for some of the more extreme teams in the league:
The results also are a testament to the accuracy and relevance of the Baseball America organizational rankings. While obviously a #1 ranking doesn't guarantee championships, the ranking is significant predictor of major league wins far into the future. Kudos to Baseball America for doing these rankings. Their well-respected reputation is well-deserved.