(This article is presented as it was written in 2004 or 2005. BJOL opens for business in 2008, so it is a few years out of date. Somebody remind me, and I will update the data and add an end-note about the changes. Thanks. . ..Bill)
I. Introduction
The goal of this research, this paper, is to provide definitive or as near as possible to definitive answers to two questions:
- Do teams with outstanding bullpens tend to do well in close games?
- Do teams with outstanding bullpens tend to do well in post-season play?
These are questions, of course, to which the average sports broadcaster already has quite definitive answers: Of course they do. We are involved in the familiar and treacherous process of documenting what seems obvious to those who are wise enough to be less skeptical. Also, there may be. . ..I believe there is. . .previously published research in this area by other writers, but I don’t know because I don’t know how to find that stuff.
To figure out whether teams with good bullpens do well in close games, we first have to establish by some systematic process whether a particular team does or does not have a good bullpen. In order to address that question, we have to back up to a yet more elementary question: How good is this reliever? How good is Keith Foulke, 2004, compared to Dick Radatz, 1964, or compared to Bill Campbell in 1977, or, for that matter, compared to Dick Drago in 1978 or Luis Aponte in 1982?
Let me stress this up front: I am not here to debate those issues. I am not here to debate who was the greatest relief pitcher of all time, or how good was Dennis Eckersley in 1991 compared to Goose Gossage in 1977. What I am really trying to do is not to debate those issues, but to get past the debate on those issues, so that I can engage a couple of questions that lie further down the battlefield.
However, neither can we afford, if we are to develop convincing answers to the target questions, to dodge these earlier questions. We have to deal with the entry-level questions in a satisfactory manner, so that we are then in a position to deal with the subsequent questions which are our real purpose. It is my view that it is impossible to give completely convincing or satisfying answers to these entry-level questions about specific pitchers in specific seasons. No matter what values we place on Dennis Eckersley in 1991 and Goose Gossage in 1977, somebody is going to be unsatisfied, and he or she will always have a valid point to make in support of his or her position. Our answers will not be perfect, because no answers to those questions are perfect.
What our answers to those questions must be is consistently reasonable, more or less without exception. We thus need to begin by assigning consistently reasonable values to every relief pitcher in baseball, in every season in baseball history.
II. Method
The method I have chosen here is essentially to ask, “How many standards of performance does this relief pitcher meet?” What we’re really doing is asking ten yes-or-no questions about each reliever’s season, although we are asking those questions through mathematical equations. If the pitcher has a “yes” answer to all 10 questions, that’s a 10-point season; if he has yes answers to none of them, that’s a 0-point or “non-contributing” season.
These standards are not set so that every pitcher is automatically included. You have to do something sort of marginally note-worthy to make the list, even if it is only show up all season. If a pitcher pitches 40 games in relief, league-average ERA, 3-5 wins and 3-5 saves. . .that’s a zero-point season. He would be near the top of the non-contributing range, but he’d still be in it. 50 games, league-average ERA, five wins and five saves—that’s a three-point season. But if a pitcher’s season is so unremarkable that you would assume every team has some guys like that. . .he could be a zero.
The first two standards are easy to count:
- If the pitcher pitches 50 games in relief, 1 point.
- If the pitcher pitches 65 games in relief, 1 additional point.
The next three standards depend on a combination of wins and saves, with each win counting two points, each save one point.
- If a pitcher has 15 saves (with each win counting as two saves), add one point.
- If a pitcher has 30 saves (with each win counting as two saves), add an additional point.
- If a pitcher has 45 saves (with each win counting as two saves), add an additional point.
We added wins to saves in an effort to make the scores of the earlier pitchers balance better with those of pitchers today. When I first laid out the system I had points for saves, then wins + saves, but the system created too-low values for relievers before Bruce Sutter. Rollie Fingers had only two seasons in his career with 30 saves. Hoyt Wilhelm had none—but they won more games than modern relievers, since they were often brought into the game with the score tied. Fingers won 11 games in 1972, 10 in ’75, 13 in ’76. Double-counting wins puts Fingers and Wilhelm (and Perranoski and Konstanty and others) on a more equal footing with modern relievers.
Standards 6 through 9 are based on the pitcher’s Games in Relief, multiplied by 1.30 minus his relative ERA. . ..let’s call it a gorilla number (GIR-ERA). Suppose a pitcher has an ERA of 3.00, and the league ERA is 4.00. That’s a relative ERA of 0.75. Subtract that from 1.30, to get a measure of how much better he is than a useless pitcher, (a pitcher 30% worse-than-league.) If he has pitched in 40 games with a 0.75 relative ERA, that makes a gorilla number of 22 [(1.30 – 0.75) * 40 = 22.] If he pitches 50 games (in relief), that’s a gorilla number of 27.50; 60 games, a gorilla number of 33.00. The highest gorilla number in the history of baseball is 79.4, by Paul Quantrill in 2003 (89 games pitched with an ERA of 1.75. Remarkably enough, a teammate of Quantrill’s in the same season, Eric Gagne, also posted the third-highest gorilla number in history, 78.6.)
- Add one point if the pitcher has a gorilla number above 13.
- Add an additional point if the pitcher has a gorilla number above 26.
- Add an additional point if the pitcher has a gorilla number above 39.
- Add an additional point if the pitcher has a gorilla number above 52.
The final point is based on strikeout/walk ratio. . .actually, not EXACTLY strikeout/walk ratio. Strikeout/walk ratio, but
- not counting intentional walks, and
- counting hit batsmen as walks.
Call it “adjusted KW”.
- Add one point if a pitcher has an adjusted KW rate of 3-1 or better with 10 or more strikeouts and 10 or more games in relief.
That’s the system. . ..very simple, very straightforward. Essentially, what we’re saying is that a pitcher is a contributing member of the bullpen if he stays healthy and pitches all year, if he piles up saves and/or wins, if he pitches a good number of games with a good ERA, and, to a small extent, if he has a good strikeout/walk ratio.
It is a not a precise system. It is a solid, reliable system. It reliably places each relief pitcher in a group with other relievers of essentially the same accomplishment, so that the overall credentials of a group of relievers can be compared to another group. This is all we’re asking it to do.
III. Evolution of Scores over Time
In all of 19th-century baseball, only one pitcher receives a point for his contributions to his team as a reliever. Monte Ward in 1879 pitched 10 times in relief, and had a strikeout/walk ratio (for the season) better than three to one. . .that’s a point. Otherwise, the 19th century was shut out.
The first twentieth-century pitcher to earn a point was Doc White in 1907, then three pitchers contributed in 1908 (Christy Mathewson, Ed Walsh, and Jake Boultes. This is, by the way, the first time in 98 years that Jake Boultes has been placed in a group with Christy Mathewson and Ed Walsh.) Doc Crandall in 1911 earned two points, and two points (by Crandall and others) remained the record until 1925.
In 1925 and again in 1926, Firpo Marberry is credited with a five-point season by our simple method. . .a breakthrough season for a reliever, as is generally known. A long series of other pitchers then had fives, and Ace Adams finally pushed the record to six in 1945. Joe Page recorded a seven in 1949, Jim Konstanty an eight in 1950, and Ellis Kinder recorded a nine in 1953. Sevens, eights and nines became common in the late 1950s, and finally, in 1964, Dick Radatz and Hoyt Wilhelm recorded the first ten-point seasons for relief aces.
Along the same time-line, the standard for a team in a season wriggled slowly up from zero. In 1900, of course, the average total for a team in a season was zero. By 1920 the average was 0.13; by 1940 it was 1.00, and by 1950 still only 1.56. As late as 1950, seven of the 16 major league teams were still at zero. Essentially, those teams simply did not have bullpens. They had starters, and they had spot starters, and they had extra pitchers, but they had no pitcher who met any significant standard of quality or quantity as a reliever.
The point was finally reached at which every team had some bullpen of some kind in 1957. Excluding the strike-fractured 1981 season, the last team in major league history which had essentially no bullpen, no points going to the bullpen, was the California Angels of 1974. The Angels had two terrific starting pitchers, Nolan Ryan and Frank Tanana; as long as Bill Singer was healthy they had three terrific starting pitchers. The team leader in saves was the 40-year-old Orlando Pena, who pitched in only four games for the Angels at the end of the season, saving three of them. The leader in relief appearances was Skip Lockwood, who appeared in relief 35 times, but apparently almost always when the team was losing; he finished 23 of his 35 games, but had only one save and two wins. With an ERA far worse than the league norm, he was nowhere near qualifying for a point in our system. No one else on the team appeared in relief more than 20 times, and all of the leaders in relief appearances had bad ERAs.
There is no indication that the ’74 Angels ever tried to develop a bullpen. It doesn’t appear that they were worried about it. The team’s manager at the start of the year was Bobby Winkles, who had been a highly successful college manager. I hope I’m not reading too much into it, but college baseball in many respects trails the evolution of major league ball. Winkles may have simply figured that with Ryan, Tanana, Singer and Chuck Dobson, he could win games with his starters and use his bullpen to mop up, as had been done in the major leagues up until 1956. Singer and Dobson went out early with injuries, and he found himself trying to stagger through games with Andy Hassler and Dick Lange. They finished last. Winkles was fired in late June.
With the exceptions of the Angels, the ’62 Mets and two teams from 1959 and two from 1981 (which involves a data and standards problem), every major league team since 1957 has had at least some bullpen, although often, until the mid-1970s, it was a very limited bullpen. In 1948 major league teams had an average of one contributing reliever apiece. In 1957 the average reached 2.00, and in 1964 it reached 3.00.
Bullpens reached a peak of usage in 1965 and then declined for almost ten years. We have two measures here—the number of contributing relievers per team, and the average “bullpen score” for a team. In 1965 the average major league team had 3.25 relievers and a bullpen score of 9.50. Both of these numbers went down almost every year from 1965 to 1974, reaching lows of 2.13 relievers per team (1974) and a bullpen score of 6.33 (1972). That, of course, was the era of abundant twenty-game winners. In the early 1970s there were historic numbers of twenty-game winners, and of starting pitchers pitching huge numbers of innings, up to 376 innings in a season by Wilbur Wood and Mickey Lolich. As starting pitchers worked harder and harder, the bullpens, for a few years, were asked to do less.
Beginning in 1975, however, the trend lines went back in their normal direction—fewer innings for starters, fewer complete games, deeper and deeper bullpens. The average number of contributing relievers per team was back up to 2.85 in 1980, back over 3.00 in 1982, over 4.00 in 1987, and over 5.00 in 1996. The trend toward higher and higher numbers probably continues to this day. In 2004 the average major league team had 5.70 contributing relievers, tying the record set in 2001, and the bullpens scores of the last two years have been the highest of all time.
Through 2005 there have been 48 perfect or ten-point seasons by relief pitchers— four each by Mariano Rivera and Robb Nen, three each by Billy Wagner and Eric Gagne, two each by Trevor Hoffman and Dan Quisenberry, and one apiece by Dennis Eckersley, Keith Foulke, Tom Gordon, Goose Gossage, Willie Hernandez, Bryan Harvey, Al Hrabosky, Brad Lidge, Bruce Sutter, John Wetteland, Donnie Moore, Phil Regan and twenty other pitchers. Again, we’re not trying to sort among them and say which of these is the greatest season ever by a reliever; we’re just recognizing that these pitchers all had really good years as a reliever—as, for that matter, did the guys who rank at “9” or “8” or even “7”. Anything over six is a heck of a year.
IV. Off-topic Conclusions
We are now in a position to state objectively whether a team did or did not have a good bullpen, relative to the league. Let’s take 1983, for example. . ..I always like to back away 20 years or so to get away from the controversial present, and take advantage of our more settled view of the past. In 1983 the Chicago Cubs had Lee Smith, who had a 1.85 ERA and led the National League in Saves, backed by Warren Brusstar, who pitched 59 times with a 2.35 ERA, backed by Bill Campbell, Craig Lefferts and Mike Proly, who pitched 56 to 82 times apiece and were mostly pretty good. We credit Lee Smith with a 9-point season, Campbell with 4 points, Brusstar with 3, Lefferts with 2, Proly with 2 and Willie Hernandez with 1. . .altogether, 21 points for the Cub bullpen:
Lee Smith…………9
Bill Campbell……..4
Warren Brusstar…..3
Craig Lefferts……..2
Mike Proly………..2
Willie Hernandez….1
Cub bullpen, 1983…21
The Cleveland Indians, on the other end, had a bullpen score of 2—one point for Dan Spillner, one for Jamie Easterly. The Cubs had a very strong bullpen; the Indians had a weak bullpen.
Again, I will re-iterate that it is not our purpose here to make a list of the best and worst bullpens. We are trying to get beyond that issue. However, we do at this point have an evaluation of every bullpen every season, and it is required for our purpose that this be a reasonable evaluation. If you don’t accept that our method works, you won’t accept our conclusions later on. It may be appropriate to ask, then, who had the best bullpens of all time?
The greatest bullpen of all time, in terms of bullpen points, belonged to the St. Louis Cardinals of 2004. These are their stats:
| NAME |
W |
L |
PCT |
G |
SV |
GF |
IP |
BB |
SO |
ERA |
Points |
| Jason Isringhausen |
4 |
2 |
.667 |
74 |
47 |
66 |
75.1 |
23 |
71 |
2.87 |
9 |
| Julian Tavarez |
7 |
4 |
.636 |
77 |
4 |
27 |
64.1 |
19 |
48 |
2.38 |
7 |
| Ray King |
5 |
2 |
.714 |
86 |
0 |
9 |
62 |
24 |
40 |
2.61 |
6 |
| Steve Kline |
2 |
2 |
.500 |
67 |
3 |
22 |
50.1 |
17 |
35 |
1.79 |
6 |
| Cal Eldred |
4 |
2 |
.667 |
52 |
1 |
10 |
67 |
17 |
54 |
3.76 |
3 |
| Kiko Calero |
3 |
1 |
.750 |
41 |
2 |
4 |
45.1 |
10 |
47 |
2.78 |
3 |
| Al Reyes |
0 |
0 |
.000 |
12 |
0 |
4 |
12 |
2 |
11 |
0.75 |
1 |
That is, I think you will admit, one hell of a bullpen. The closer, Jason Isringhausen, did not have a historic season, but he wasn’t chopped spinach, either, with 47 saves, a 2.87 ERA, and a strikeout/walk ratio better than three to one. Behind him you’ve got two left-handers and a right-hander pitching 67 to 86 times apiece with ERAs between 1.79 and 2.61, and behind them you have two more very good relievers in Cal Eldred and Kiko Calero. I credit them with 35 points as a bullpen—three more than any other team in the history of baseball. It’s just a shame they never got to pitch in the World Series.
These are the top nine bullpens in baseball history:
| 1. St. Louis , 2004 |
35 |
| 2. Atlanta , 2002 |
32 |
| 3. LA Dodgers, 2003 |
31 |
| 4. Texas , 1999 |
30 |
| 5t. Dodgers, 1996 |
29 |
| 5t. Minnesota , 2002 |
29 |
| 5t. San Francisco , 2002 |
29 |
| 5t. Cleveland , 2005 |
29 |
| 5t. White Sox, 2005 |
29 |
Then we have a list of teams at 28. However, as you see, almost all of these teams are recent. As bullpens continue to get deeper and deeper, it seems likely that other teams will occasionally put together outstanding six- and seven-man combinations, and these teams are likely to be pushed off the top of the list.
By this method, of course, there is no “worst bullpen ever”, because there were hundreds of teams pre-1957 which had bullpen scores of zero. There is another way to look at the issue. Since the standards in this area change so dramatically over time, perhaps a more accurate way of looking at the issue is to compare each team to the league average in that season.
By that method, the greatest bullpen of all time was that of the Chicago White Sox of 1968. The White Sox in 1968 had only four contributing relievers—Wilbur Wood, Hoyt Wilhelm, Bob Locker and Don McMahon. Wood pitched 88 times, 159 innings, with an ERA of 1.87. Wilhelm pitched 72 times with an ERA of 1.72. Locker pitched 70 times with an ERA of 2.30, and McMahon, traded away at mid-season, pitched 25 times with an ERA of 1.96. Among the four of them they pitched 265 times, just short of 400 innings, with an ERA in the ones (1.95). I credit the team with 24 bullpen points, in a season in which the American League average was 7.5. The White Sox were +16.5.
The worst bullpen of all time, relative to the league, was the bullpen of the 2002 Tampa Bay Devil Rays. The Rays had one contributing reliever, Esteban Yan, who is credited with 4 points although he had a 4.30 ERA. The American League average was 16.0 bullpen points. The D Rays lost 106 games.
Maybe it is time to get to one general conclusion. Teams with very strong bullpens tend to be good teams. These are the ten best bullpens of all time, relative to the league averages, with the won-lost records of those teams:
| Team |
Lg |
Year |
Tm Total |
Lg Avg |
Tm V Avg |
Won |
Lost |
| Chicago White Sox |
AL |
1968 |
24 |
7.5 |
16.5 |
67 |
95 |
| St. Louis Cardinals |
NL |
2004 |
35 |
20.4 |
14.6 |
105 |
57 |
| Texas Rangers |
AL |
1999 |
30 |
16.8 |
13.2 |
95 |
67 |
| Minnesota Twins |
AL |
2002 |
29 |
16.0 |
13.0 |
94 |
67 |
| Oakland A's |
AL |
1975 |
18 |
5.6 |
12.4 |
98 |
64 |
| St. Louis Cardinals |
NL |
1992 |
28 |
16.0 |
12.0 |
83 |
79 |
| Atlanta Braves |
NL |
2002 |
32 |
20.1 |
11.9 |
101 |
59 |
| Los Angeles Dodgers |
NL |
2003 |
31 |
19.4 |
11.6 |
85 |
77 |
| Cleveland Indians |
AL |
1995 |
22 |
10.7 |
11.3 |
100 |
44 |
| Pittsburgh Pirates |
NL |
1979 |
21 |
9.8 |
11.3 |
98 |
64 |
| Average Won-Lost |
|
|
|
|
|
93 |
67 |
All of them were good teams except the 1968 White Sox. These were the teams with the ten weakest or thinnest bullpens of all time, relative to the league:
| Team |
Lg |
Year |
Tm Total |
Lg Avg |
Tm V Avg |
Won |
Lost |
| Tampa Bay Devil Rays |
AL |
2002 |
4 |
16.0 |
-12.0 |
55 |
106 |
| Kansas City Royals |
AL |
1999 |
5 |
16.8 |
-11.8 |
64 |
97 |
| Arizona Diamondbacks |
NL |
2004 |
9 |
20.4 |
-11.4 |
51 |
111 |
| Chicago Cubs |
NL |
1988 |
3 |
14.3 |
-11.3 |
77 |
85 |
| Kansas City Royals |
AL |
2000 |
7 |
18.1 |
-11.1 |
77 |
85 |
| Texas Rangers |
AL |
2005 |
9 |
20.0 |
-11.0 |
89 |
73 |
| Arizona Diamondbacks |
NL |
1998 |
6 |
16.7 |
-10.7 |
65 |
97 |
| Kansas City Royals |
AL |
2004 |
8 |
18.4 |
-10.4 |
58 |
104 |
| Houston Astros |
NL |
2000 |
6 |
16.3 |
-10.3 |
72 |
90 |
| St. Louis Cardinals |
NL |
1980 |
3 |
12.9 |
-9.9 |
74 |
88 |
| Average Won-Lost |
|
|
|
|
|
68 |
94 |
Of course, teams which are outstanding in any area tend to be good teams overall, so we don’t want to get ahead of ourselves in talking about the importance of the bullpen. But, in general, teams with outstanding bullpens are good teams, and vice versa.
Another thing we can do with the data we have so far is to compare relievers’ careers. What we are essentially asking here is not “who was the greatest relief pitcher of all time?” but rather “which relief pitcher, over the course of his career, met the most standards of performance for a season’s work?”—defining standards as we have in this study. But the two questions may be taken to be related, and perhaps the pedantic count we have to offer here may be considered a contribution to the larger discussion.
The first generation of career relievers retired in the very early 1970s. Elroy Face had a career total of 62 points for his bullpen work, Ron Perranoski 60, Lindy McDaniel 57, Don McMahon 47, Stu Miller 46—and Hoyt Wilhelm 83. Wilhelm was clearly the best of the first generation of career relievers, and he was the only one of that generation elected to the Hall of Fame.
Wilhelm’s record total of 83 bullpen points was surpassed by Rollie Fingers, who reached 88 bullpen points in his career—and became the second reliever in the Hall of Fame. Fingers’ record was surpassed by Lee Smith, who reached 92 points—but has not been selected, and may well not be.
Lee Smith is an interesting figure. I don’t know that he left a great many friends behind him in baseball, and I don’t know that he was ever regarded, while active, as a towering figure. His records are more notable for endurance and consistency than for dominance. He didn’t dominate hitters the way Sutter did, or Gossage, or Eckersley, but in his own way he was a powerful and impressive pitcher. He wasn’t out there throwing tricky stuff, like Doug Jones or Stu Miller, Quisenberry or Hoyt Wilhelm. He was throwing gas.
Lee Smith does hold the career record for saves. Perhaps more impressively than that, Smith had an ERA better than league average (park adjusted) every season from the beginning of his career in 1980 through 1992—thirteen consecutive years. In 1993 he was average (4 runs worse than average in the National League, but then 4 runs better than average in the American League)—and then he began another run of three seasons of better-than-league performance. Seventeen straight seasons, 1980 through 1996, he was average or better every single season, and better than average 16 out of 17. Only in his final season, 1997, did the league get the best of him. Compare that to a handful of Hall of Famers. It’s pretty hard to find another pitcher who had 17 consecutive seasons without a bad one—and Smith was pitching 65, 70 games every year, and he was pitching most of the time with the game on the line.
Relief pitchers are being selected to the Hall of Fame with some regularity now—Eckersley, now Sutter. The standards for outstanding performance as a relief pitcher have been in constant upheaval since 1924. There are no historic norms by which to evaluate relievers, no magic numbers hallowed by time. It’s harder to focus on who the right relievers are, because we just don’t have the reference points that we have in other areas. The oft-cited corruption of the numbers of the steroid era is trivial compared to the seismic shifts that occur every few years in save totals and in how those saves are earned.
Dennis Eckersley, I think, is a Hall of Fame selection that will stand the test of time. Eckersley’s career total is “only” 74 bullpen points, but
- 74 points is 12th on the all-time list, and
- Eckersley also made very substantial contributions as a starting pitcher.
But Bruce Sutter, I think, was frankly a mistake. Sutter was very impressive on the mound. His best seasons, half seasons, he was unhittable. His career, taken as a whole, is really not all that impressive. What did he do that Sparky Lyle didn’t?
I have in the past been critical of Rollie Fingers’ selection to the Hall of Fame, and I guess I still would be at a certain level. Compared to Bruce Sutter, Rollie was awesome. To me, Bruce Sutter should have been in line behind Goose Gossage and behind Lee Smith. In any case, these are the top career bullpen totals, through 2004:
| 1. Lee Smith |
92 |
| 2. John Franco |
89 |
| 3. Rollie Fingers |
88 |
| 4. Trevor Hoffman |
85 |
| 5. Mariano Rivera |
84 |
| 6t. Hoyt Wilhelm |
83 |
| 6t. Jesse Orosco |
83 |
| 8. Kent Tekulve |
79 |
| 9. Jeff Reardon |
76 |
| 10t. Doug Jones |
75 |
| 10t. Goose Gossage |
75 |
| 12. Dennis Eckersley |
74
|
| 13. Roberto Hernandez |
73 |
| 14. Robb Nen |
72 |
Continuing: Gene Garber, 70; John Wetteland, 70; Tom Henke, 67; Sparky Lyle, 65; Dan Quisenberry, 65; Bruce Sutter, 64; Michael Jackson, 64; Dan Plesac, 63; Jose Mesa, 63; Billy Wagner, 63; Elroy Face, 62; Jeff Montgomery, 62; Rick Aguilera, 60; Ron Perranoski, 60; Rod Beck, 60, Troy Percival, 60.
By the end of the 2006 season, Lee Smith will almost certainly have been supplanted atop the list by Mariano Rivera and/or Trevor Hoffman. There are other methods yet to be developed, and these may lead us to other conclusions, and in any case that’s really not what we’re here to discuss.
V. Conclusion in Re Close Games
It appears to be unquestionably true that teams with good bullpens tend to exceed expectations in one-run games and in close games in general, and even more true that teams with bad bullpens tend to under-achieve in these games. The effects measured are not large in terms of the 30- or 40-game separation between a first-place team and a last-place team, but they are quite large when compared to the relatively few runs saved by a bullpen or the relatively small differences between teams in terms of one-run victories.
The “direct” or “normal” benefit of having a good bullpen, in terms of runs saved, could be estimated at about 8-10 games per season. (Let’s assume that the bullpen pitches 540 innings or 60 games per season, and that the difference between a good bullpen and a poor bullpen is 1.50 runs per 9 innings. The difference in runs over the course of a season would be about 90 runs, or about 9 games.) Beyond that, however, there is an additional benefit of about 3 ½ games due to the fact that these runs saved (or not saved) tend to come at critical moments of the game.
I identified the teams with the 50 best bullpens of all time, relative to league norms, and the 50 worst bullpens. All of these tend to be recent teams; the first teams chronologically on the “good bullpen” list are the 1959 White Sox and the 1959 Cubs, and the first team chronologically on the “bad bullpen” list is the 1965 Mets. 84 of the 100 teams in this portion of the study are post-1980 teams.
I looked at two measures of performance in close games:
- The team’s record in one-run games, and
- The team’s actual won-lost record contrasted with their pythagorean expectation.
The 50 teams which had strong bullpens had an expected average, based on their runs scored and runs allowed, or 88.95. They actually won an average of 90.34 games, exceeding expectations by 1.39 wins.
Previous research, which may be unpublished, shows that a team’s expected ratio of wins to losses in one-run games is the same as the ratio of their runs scored to runs allowed. The teams with strong bullpens had an expected won-lost record in one-run games of 26-23 on average, a .526 winning percentage. Their actual winning percentage in one-run games was .550. These teams exceeded their expected one-run wins by 1.19 games.
The data on teams with bad bullpens is more dramatic. The teams with weak bullpens under-achieved by 2.06 wins overall, and by 1.95 wins in one-run games. The teams with poor bullpens had an expected winning percentage of .475 in one-run games—but an actual winning percentage of .433.
Most of you probably know that winning percentages in one-run games tend to be nearer to .500 than overall winning percentages. .600 teams tend to play about .550 ball in one-run games, and .400 teams tend to play about .450 ball. But the teams with bad bullpens actually had a worse winning percentage in one-run games (.433) than overall (.439). The data seems to leave no doubt that a weak bullpen means trouble in one-run games.
VI. Since you asked
Are the standards of bullpen performance different between the leagues, you asked? I knew you would.
The norms for bullpen performance were essentially the same between the leagues prior to the adoption of the DH rule in 1973. The DH Rule did create a separation of standards between the leagues. This separation reached its widest point in 1977, when the average National League bullpen score was 13.2, the American League, 6.7.
By 1990, however, the American League had caught up, and since 1990 the norms have been essentially the same in both leagues. Bullpen usage in modern baseball is controlled by pitch limits for starters, the use of closers and left/right matchups versus hitters. The DH rule sometimes effects when a substitution occurs, but, in modern baseball, it has nothing at all to do with how much the bullpen is used.
VII. Conclusion in re Post-Season Play
With regard to the issue of whether teams with strong bullpens tend to do well in post-season play, we encounter our old friend “no definitive evidence”.
I looked at all post-season matchups in history, trying to find cases in which a team with a strong bullpen played a team with a much weaker bullpen. In 1979, for example, the Pittsburgh Pirates (we-are-fam-a-lee), who had a strong bullpen by the standards of the time (Kent Tekulve, Enrique Romo and Grant Jackson) played the Baltimore Orioles in the World Series. The Oriole bullpen, led by Don (Full Pack) Stanhouse, was of nowhere near the same quality, even adjusting for the difference in league norms. The Pirate bullpen was credited with 21 points vs. a league norm of 9.8, while the Orioles were credited with 9 points vs. a norm of 7.0.
My standard was that the stronger bullpen had to have an advantage of at least eight points in raw terms and also of at least eight points vs. league norms. The Pirates were +12 in raw terms and +9.3 vs. league norms, so this qualifies.
I found 38 qualifying post-season matchups in baseball history—and the team with the stronger bullpen did in fact win 24 of the 38 series. This certainly suggests the possibility that a strong bullpen does become especially important in post-season play.
But we can’t reach that conclusion, for two reasons. First, the teams with good bullpens were almost always better teams in other regards—they had better offenses, for example—and thus very probably would have won more than half of these series even if the bullpens were even. The 1979 example, in which the Orioles were otherwise probably a better team, is atypical. A more typical example would be the 1988 Oakland A’s vs. the Red Sox, in the league championship series, or the 1989 A’s vs. the Giants in the world series. The A’s did have a much stronger bullpen than the Red Sox in ’88 or the Giants in ’89—but they also had much stronger teams in other respects. My best estimate is that the teams with better bullpens could have been expected to win 21.5 of these 38 series based on their overall strength, without suggesting that there was any special importance to the bullpen.
The “good bullpen teams” did exceed expectations in these 38 series—24 wins versus 21.5 expected wins—but that is nowhere near a statistically significant separation of the data. The chance that the same separation would occur as a random event is 26%.
In fact—this is the second problem—even if there were no bias in the data, a 24-14 record is still nowhere near statistically significant. Even if the teams were even overall, there would still be a 7% chance that the teams with better bullpens would go 24-14 or better in 38 matchups. So. . . .sorry to have led you on, but we just can’t reach any conclusion on that one. It is possible that there is such a special significance to bullpens in post-season play; it is possible that there is not.
Thanks for reading.
Bill James
Ft. Myers, Florida |