Behind the Scoreboard March 28, 2009
Championship Leverage Index: How Meaningful Is This Game?

Opening day is right around the corner and soon your favorite team will be taking the diamond for its very first game. Hope springs eternal and the beauty of opening day is that every team starts at 0-0. As the season wears on, the games either become more or less meaningful depending on the standings. As a Cubs fan growing up in the 80's and 90's, I remember many a year when opening day was the most meaningful game of the year, with the rest of the season a slow march into irrelevance. In a lucky few years, the games took on more importance as the year progressed as the Cubs fought for contention. It's easy to tell which games are big and which games are meaningless, but this article attempts to put a quantitative number on the relative meaning of each game of the season.

Tom Tango's Leverage Index is a great tool for measuring the impact of a particular in-game situation. A Leverage Index of greater than 1.0 indicates the at-bat is more meaningful than an average play, and an LI of less than 1.0 indicates the at-bat is less meaningful, with LI's ranging from nearly 0 up to more than 5.

Taking this to the next level, we can create the same type of metric, except instead of producing it at a game level, we can produce it at a season level, with a value of 1.0 indicating an average regular season game's impact on a team's chances of winning the World Series. LI's larger than 1.0 will indicate the game has additional meaning, and LI's less than 1.0 indicate the game is less meaningful than an average regular season game. Dave Studeman touched on this subject at Hardball Times, but his index and mine, which I'll call "Championship Leverage Index" give quite different results.

Each team's Champ LI for a particular game is calculated by first getting the current probability of winning the World Series. Then we calculate this probability again, this time assuming that the team wins the game. The difference between the two is then found and this difference is the potential impact of the game. Tango's regular Leverage Index has to deal with multiple potential events, and thus has to calculate the standard deviation of the impact of winning depending on several outcomes, however in this case, because there are only two potential events in a game (win or loss), taking the difference in probability between the pre-game and post-game is sufficient.

For instance, in 2008, after 81 games, the Cubs probability of winning the World Series was 10.22% (81.8% to make the playoffs). A win in the 82nd game would up the probability of winning to 10.54% (84.3% to make the playoffs). This difference of 0.32% is the basis of the calculation of Champ LI. The difference is then indexed to the increase championship win probability of an average regular season game.

This average game, is also, not coincidentally, the same as opening day. Because nobody knows what the rest of the season will hold, the opening day game is, by definition, the average regular season game - depending on what happens sometimes it will be much less meaningful than other games, and sometimes much more. This increase in championship probability due to winning this average game is 0.28% (the increase in probability of making the playoffs is 2.25%). Using the example from above, 0.32/0.28 gives a Champ LI of 1.14, meaning the 82nd game (played with a 49-32 record and a four game lead over the Cardinals) was slightly more meaningful to the Cubs championship hopes than the average regular season game.

As you can imagine, the work that goes into this requires a lot of simulation. With simulations come assumptions, and here I assumed that all teams were of equal strength. This assumption is certainly not true, but it's acceptable because actual team strength is largely unknown, especially early in the season, and there is a nice symmetry to placing teams on equal footing. This is analogous to Tango's leverage index assuming opposing teams are of equal strength within an individual game. My current simulation also does not take into account the schedule of the teams, though that would be possible, changing the results very slightly.

Below are a few graphs to illustrate the Championship Leverage Index. First, are simply three graphs of each NL team's chance of making the playoffs in 2008 (to get the probability of winning the World Series, simply divide by 8).

Now let's look at the same graphs for each team's Champ LI. How much do the standings affect the importance of each game? As I mentioned before, each of the teams start opening day with an LI of 1.0.

To illustrate the Championship Leverage Index, let's focus in on the NL Central, which has a variety of teams that illustrate various scenarios nicely.

There are several interesting things to point out. As you'd expect, right off the bat, the teams that start poorly see their Champ LI decrease, while teams that do well see their games grow in importance. By late season, those teams that were out of the race, Pittsburgh and Cincinnati, had a Champ LI of essentially zero.

Similarly, the Champ LI also decreases dramatically when a team becomes too far ahead. After the Cubs 100th game, with a 1 game division lead and a two-game lead in the wild card, the Cubs games had a Champ LI of 1.70. But after they went on a tear and built up a 5 game lead three weeks later, their games' importance dropped dramatically, with the Cubs' Champ LI reduced to only 0.50. Because the playoffs seemed so likely, their games took on less importance. A few weeks later, coasting with a large lead, their Champ LI was reduced to essentially zero because the playoffs were assured.

We also see that the Champ LI of teams who remain in contention (but not too far ahead), grows as the season goes on. Furthermore, as long as a team is in contention, the game's meaning doesn't change much whether the team's prospects for the playoffs are on the high side or the low side. By the 125th game, the Cardinals and Brewers were both in contention, but had vastly different probabilities for the postseason (Brewers at 65% and the Cardinals at about 30%), however their Champ LI was about the same at around 2.0.

Another finding is, not surprisingly, all things being equal, late season games mean more. Eleven games into the season the Astros were struggling at 3-8, their playoff probability had dropped to 11%, and their Champ LI was down to 0.65, far less than an average game. However, fast forward to game #147 and the Astros, three games out of the wild card, had a playoff probability that was also about 11%. However, now the Champ LI was at 1.67, far more than an average game and certainly far more than their mid-April games when they had the same probability of making the playoffs. All things being equal, September games mean more than April games.

Furthermore, as the season draws to a close, if a team is still fighting for a playoff spot, their Champ LI grows exponentially. The Brewers' Champ LI was so high by the last games of the season (when they were fighting for a wild card spot with the Mets and Phillies), that their Champ LI is off the chart. By the last game of the season, which they went into tied with New York, their Champ LI was 11.1, meaning that the final game was 11 times more important than the average game (this is the maximum Champ LI for a regular season game, unless Milwaukee and New York had been playing each other, in which case the Champ LI would have doubled to 22.2).

Of course, the Champ LI applies in the postseason as well. You can see from the following chart below, the Championship Leverage Index of each possible postseason game, depending on the status of the series.

As you can see, every postseason game takes on vastly more importance than an average regular season game. The maximum Champ LI is of course, the 7th game of the World Series, with the game taking on 178 times as much meaning as an average regular season game.

Like Tango's individual game Leverage Index, the Championship Leverage Index doesn't exactly tell you anything new, but just quantifies a game's importance into a useful number. It can be useful in analyzing players' performance in "big games" as well as looking at things like attendance or TV ratings. It's also fun just to realize in quantitative terms exactly how much each game matters.

Another handy feature is that to figure out the importance of an individual at-bat within an individual game, you can simply multiply Tango's Leverage Index with the Championship Leverage Index. For instance, can you name the most important at-bat of the season last year?

It was Game 7 of the ALCS (Champ LI of 88.9) when JD Drew came to bat with the bases loaded, two outs, in the bottom of the 8th inning of a 3-1 game (game Leverage Index of 5.19). The total Championship Leverage Index of the at-bat is 461.4 (5.19 x 88.9), meaning that the at-bat was 461.4 times more important than an average regular season at-bat.

As Sox fans recall, Drew struck out, ending the inning. In one at-bat as big as some players entire seasons, he blew it. So what proportion of a championship did Drew lose by striking out? For that you'll have to wait until next week, when I introduce Championship Leverage Index's sister stat, Championship Win Probability Added.

his study seemed only to consider the time of year the game was played while this index, which I'll call "Championship Leverage Index" is far more comprehensive.

Not true, and I don't know how you could make that mistake. My study calculated each team's probability of making the postseason each day based on its place in the standings. I'm not sure that your approach adds anything to the regular season analysis I did, though I might be missing something (obviously!).

Studes,
Looking back at your article, all of your examples and graphs do only take into account the time of year, not the standings. You also say you index to the 112th game of the season, which I don't think is the proper approach. Additionally the results are different - under your method, the last game of the season has a meaning of less than 5 times that of an average game, whereas the real value is more than 11 times if the race is tied (which seems to be your assumption). I see now at the very end how you mention potentially re-doing this based on the standings, although you don't say exactly how you would do so.

While your article does approach the topic of creating a leverage index for a season (which I give you credit for), this is the first time, to my knowledge, that it's been actually put into practice using proper methodology with examples.

That's ludicrous. A system that only looks at the time of the year would give every team the same credit at each point of the season. How do you account for the fact that the White Sox and Nationals had completely different indices in their 161st game? I have no idea what you're reading.

Also, what does it matter where you set your baseline for an index? It doesn't. That's why it's an index.

Sorry, perhaps I am not aware of your full work - I don't see any mention of the White Sox or Nationals in the article I was referencing. If you have written more on this can you link to it?

Ah, that explains it. The article you linked to was just the intro to a three-parts.

Here's part two:

http://www.hardballtimes.com/main/article/the-drama-index/

Here's part three:

Sky,

I think that this is the article Studes is referencing:

http://www.hardballtimes.com/main/article/the-drama-index/

Ah yes, that does explain it - I had never seen the other two parts until now and I see that your index does indeed take into the account the standings - I'll change my intro to correct that.

Interestingly we both took different approaches (binomial vs. simulations) and got actually dramatically different results. By late-May the Nats were in last place and 8 games out of first - you have the game at about 3 times MORE important than their opening day game, while I have the game at about 3 times LESS than their opening day game.

I prefer the simulation approach and think it yields a better and more intuitive result - certainly there are major differences. Now both approaches are out there and readers can decide.

Sounds good, Sky. It's not apparent to me that your Nats' example proves that one system is better than another, because I'm not sure what the "intuitive" answer is. Perhaps when you're done publishing your results we can directly compare them to show other distinctions. Having two systems is certainly better than one.

Reading over your article again, a couple of questions occur to me:

1. My system compares the difference between a win to a loss to compute the index. You only look at the impact of a win vs. the current probability, right? Why did you approach it this way?

2. What assumptions did you make about the opposition? If you looked at the impact of a win, what did you assume the opposition would on the same day?

And did you do anything special when two contending teams played each other, when a win for one would result in a loss for another (so the opposition outcome would be known?).

So Bucky Dent is therefore the MVP of the 1978 AL?

Studes, I think comparing our systems would be interesting - to answer your questions:

1. If we assume a 50% chance of victory, comparing a win vs. current probability would give the same results as a win vs. loss. If you start using values different than 50%, then you'd have to take a different approach like the one you mentioned.

2. No assumptions were made for the opposition when computing the new probability - the games were still yet to be played and had to be simulated.

3. The graphs don't reflect the opponent in each game, but I have some output that does. I plan to talk about this in a later post.

Thanks, Sky. That's interesting. I assumed that the opposition would play .500 ball, and I did look at the difference between a win and loss for the team.

I was considering this topic just yesterday, trying to figure out the relative importance of a game depending on standings position and playoff chances. I separated the playoffs and worked from a starting point that a one-game playoff -- the Bucky Dent game -- would have a value of 1.000, or 1,000, because if you win you're in and if you lose you're out. I didn't get very far, but early-season games seemed to have a value of about 30.

Also, shouldn't there be a minimum game value? A team that's out of the playoffs is, presumably, working for future years, evaluating players and such. There's some value there.

Is this similar to what Sports Club Stats and a few others sites do during the year?

http://www.sportsclubstats.com/MLB/American.html

vr, Xei

"1. If we assume a 50% chance of victory, comparing a win vs. current probability would give the same results as a win vs. loss."

Does that have to be true? I would think that in some cases the impact of a loss would be greater (in absolute terms) than the impact of a win. And could that explain the different assessments of the Nats in late May? If further losses meant a relatively big decline in WS probability, then the win-vs-loss spread would be large, while win-vs.-current would be smaller.