Comparing baseball players from different eras is always an endeavor that requires we acknowledge that, in the history of the sport, some periods of time favored hitters, others favored pitchers. We also know that ballparks are different. You can put the best pitcher in baseball on the Colorado Rockies and he’s going to struggle to keep his ERA under 3.50.

With relief pitchers, we have the added challenge of measuring not just performance but evolving patterns of usage. The firemen of the the 1970’s and ’80’s often came into the game with runners on base. In the last 30 years, most elite closers have had the luxury of coming into a clean 9th inning, with a one-to-three run lead and the bases empty.

Here is the list of the top 10 pitchers all-time in saves, plus those in the Hall of Fame.

WP Table Builder

Well, it’s clear that you can’t just evaluate save totals and conversion percentages when comparing pitchers across eras, otherwise you would have to conclude, based on their save conversion percentages and ERAs, that Joe Nathan and Jonathan Papelbon should be in the Hall of Fame ahead of Dennis Eckersley, Rollie Fingers, Goose Gossage and Bruce Sutter.

It’s not uncommon for professional athletes to look with disdain upon successive generations. One man who has never been reluctant to share an opinion is Richard “Goose” Gossage. This spring, in an interview with N.J. Advanced Media, the Goose went off on today’s one-inning closers and decided to compare himself to Mariano Rivera, who is universally regarded as the best relief pitcher in the game’s history:

“I’m not taking anything away from what Mo did, but don’t compare me to him. It’s insulting. It really is.”

“These guys that pitch one inning with the three-run lead (and get a) save. It shouldn’t even be a save for one inning and a three-run lead. This is not a knock against Mo. I’m just trying to make a point that I’d like to know how many of Mo’s saves are of one inning with a three-run lead. If everybody in that (expletive) bullpen can’t save a three-run lead for one inning, they shouldn’t even be in the big leagues.”

— Goose Gossage (to N.J. Advanced Media, Feb. 18, 2017).

Well, Goose, let’s answer your question. A year ago, when researching the Hall of Fame case for Trevor Hoffman, I did a deep dive into the game logs for all of the top relief pitchers in history. Using the power of sorting and data splitting through Microsoft Excel, I was able to break down the top relief aces and what their save percentages were in multiple scenarios.

In his career, Rivera saved 652 games (compared to Gossage’s 310). Of those 652, a whopping 483 of them occurred in a “clean” 9th inning (no runners on). Here is how Rivera and Gossage stack up in terms of the situations they pitched in to accumulate their saves.

WP Table Builder

You’ll notice that Rivera, in the 9th inning games in which he entered with runners on base (the “fireman” situations) didn’t have to put out much more than a small campfire. It’s a popular managerial strategy to use a set-up man in the 9th inning when the team has a 4-to-5 run lead but, if that pitcher allows two or three runners to reach base, then they reach for the closer. Remember that one of the criteria for a save is when the tying run is on deck.

You can also see that, in situations in the 8th inning in which runners were on base, Gossage’s numbers are superior. And, of course, as a modern closer, Rivera had virtually no save situations before the 8th inning in his entire career.

In his autobiography, this is how Gossage described the role of a closer:

“It’s imperative that closers realize that, no matter how overpowering their stuff, they are always one pitch away from disaster. Being a closer on a baseball team is like working a high wire without a net. There’s no margin for error.”

— Goose Gossage (in “The Goose is Loose” (2000))

The key line here is “no margin for error.” The truth is that, in the modern game, the closers actually do have a margin for error more often than not. You’ll notice that, in 395 of his career saves, Rivera had the safety net of being able to give up a home run without blowing the game (this is the bottom row on the next chart).

WP Table Builder

In his career, Rivera pitched in 113 more games than Gossage did in his career and, no matter what situations the two men faced, Rivera’s success rates were a little bit better. That’s not a slight on the Goose. If you stack up any relief pitcher against the Super Mariano, Mo is going to look better because he was the best at his craft in the game’s history.

Still, when it comes to raw save totals, 602 for Mo and 310 for Goose doesn’t tell remotely close to the whole story.

In 193 of his 310 career saves, Gossage worked more than one inning. Rivera only recorded four or more outs in 119 of his 652 career saves. By the way, that’s still quite a few compared to his contemporaries of the 1990’s and 2000’s but a small amount compared to the Goose.

Gossage pitched two innings or more in 125 of his saves; Rivera only did this 11 times. These are traits also found in Goose’s contemporaries.

The chart below shows the number of innings pitched in all games that ended with a save. We’re showing the same 14 pitchers we showed before plus Dan Quisenberry.

WP Table Builder

I included Quisenberry for a reason. The side-arming closer for the 1980’s Kansas City Royals only finished his career with 244 saves but pitched nearly as many innings in those saves as Billy Wagner and Francisco Rodriguez, members of the 400-save club. The Quiz and his Hall of Fame contemporaries (Goose, Sutter and Fingers) often had to save the game twice.

The next chart shows the sheer volume of multi-inning saves that the firemen of the 1970’s and ’80’s had compared to the one-inning closers of the modern game.

WP Table Builder

I think any serious fan would not be surprised by the numbers here but it’s still striking when you look at them. The four Hall of Fame pitchers not named Eckersley all logged at least two innings in more than a third of their career saves (for Wilhelm, it was more than half). So, the fact that all four have career save percentages in the 75 percent range should absolutely not lead to a conclusion that they were inferior to their successors. Think about the difference between a 3-out save and a 6-out save. Again, if you’re getting 6 outs, you in essence have to save the game twice.

So, what does this all mean? How do we properly quantify the added value of the multi-inning saves authored by the stoppers of the ’70’s and ’80’s compared to the high number of one-inning saves put up by the closers of today? And how do we quantify the differing degree of difficulty of a “clean 9th inning” save as opposed to a “on fire” runners on base save?

The answer is imperfect but there is one statistic that actually gets us close. It gets us close, not all the way there.

 

That statistic is called WPA (Win Probability Added). We introduced the concept of WPA in Part Five of this series but, for those who may have skipped it (shame on you), a recap:

WPA takes each individual play and assesses to what degree it increases or decreases the player’s team’s chances for a win. Therefore, an out to end a one-run game has a dramatically higher WPA than an out in the seventh inning of a 10-0 contest. If you’re the type to lose patience with some of the advanced statistics, hang in with me for a bit on this one. It’s easy to understand how WPA works and it’s important for the evaluation of relief pitching.

WPA is a “counting” stat that can count backwards too. If you’re a closer and you enter a game with a 3-run lead in the 9th, you should close out that game over 96% of the time so, depending on the ballpark and the season, you would get approximately 0.04 WPA “points” for bringing your team from a 96% win probability to 100% (an actual win). If, on the other hand, you give up four runs and lose that game, you’ll get -0.96 WPA “points” if it’s a road game. (If you’re pitching at home, your team still has a chance to come back to win so you can’t go from 96% to 0%).

Now, if you enter the game in the 9th with just a one-run lead and convert the save, that’s worth anywhere between 0.16 and 0.24 WPA. Why such a big difference? It’s because of the ballpark and the season. Using Trevor Hoffman as an example, a 9th inning save with a one-run lead at PetCo Park was worth about 0.16 WPA. A 9th inning save with a one-run lead at Coors Field was worth about 0.24 WPA because it’s much easier to score runs at Coors Field.

To help you understand how it works (and impacts relief pitching), here are some famous examples from October baseball with the relief pitcher involved and the positive or negative impact they had on each game.

WP Table Builder

Don’t you love it when the value of a mysterious, previously unknown metric is understandable and actually displays some good old fashioned common sense?

So, here is a list of the career Top 10 for relief pitchers for Win Probability Added plus Lee Smith and all of the existing Hall of Fame relievers. These are regular season numbers and they ONLY include games in which the listed pitchers appeared out of the bullpen.

WP Table Builder

Well, this is interesting, isn’t it? A moment ago I discounted the high save percentage numbers of Nathan, Rodriguez and Papelbon because they were one-inning specialists. And yet the three of them (plus Billy Wagner and Troy Percival) are all in the top ten of this game-context-dependent metric. In the meantime, Hall of Famers Fingers, Sutter and Eckersley are not even in the Top 20.

What potential conclusions can we draw from this?

  1. Fingers, Sutter and Eckersley may have been overrated as relief pitchers.
  2. WPA favors the pitcher who enters with a clean inning as opposed to the pitcher who comes in with the game “on fire,” runners in scoring position. Think about it. If you’re entering a clean inning and you give up a single, doom has not occurred. If you enter a game with two runners in scoring position and you give up a single, you may have instantly lost the lead for your team. A two-run swing will punish your WPA much more than a single with the bases empty.
  3. The modern usage of top closers for just the 9th inning, while perhaps now unimaginative, yields predictable results. Rivera, Hoffman, Nathan, Wagner, Papelbon, Rodriguez, and Percival all closed at remarkably high efficiency and that brought high value to their teams.

We’ll address #1 in Part Seven of this series.

Point #2 is important: there is definitely a bias that favors the one-inning closers when compared to the multi-inning stoppers of yesteryear. If you take a list of all of the relief pitchers of the LCS era with over 100 saves and calculate their WPA per inning pitched (not overall WPA), the top 20 are ALL one-inning closers and include nine hurlers who are still active (Craig Kimbrel, Greg Holland, Kenley Jansen, Aroldis Chapman, Joakim Soria, Mark Melancon, Huston Street, David Robertson and Zach Britton). Rivera, of course, tops the list, with Nathan (9), Wagner (10) and Hoffman (11) also in the top 12 positions.

To find a closer from the ’70’s or ’80’s, you have to go down to 35th place (Dan Quisenberry). Gossage is in 39th, Sutter 44th.

That bias is more than just a statistical flaw. Rather, it’s a one of many statistical manifestations of a human factor. Simply put, the job of a one-inning closer of today is easier than the job of a multi-inning closer of yesterday. Today’s bullpens contain multiple pitchers who throw fastballs in excess of 95 miles per hour. There are a variety of reasons for this:

  1. The natural evolution of human performance.
  2. The “not as natural” evolution of human performance (better fitness, diet and conditioning and, PEDs (both legal and illegal)).
  3. The law of maximum effort.

Here, point #3 is the key. The majority of pitchers of today are conditioned to throw with maximum effort all of the time. This is especially true with closers. Could Aroldis Chapman throw nine innings’ worth of 100 MPH fastballs? Probably not. But for one inning? Relief pitching today is the equivalent of a 100 meter sprint whereas relief pitching of the era of Gossage was like a 400 meter race.

If you watch MLB Now on the MLB Network (Brian Kenny’s show), he’s constantly waxing poetic about the way teams used relief pitchers in the era of Gossage, Fingers, and Sutter. He often talks about how teams aren’t properly using their best relief weapons when they save them for just the 9th inning. He’s correct of course when it comes the idea that managers actually manage according to whether their “closer” can get a save or not (see Part Four of this series).

Kenny envisions a game in which all teams utilize their best guns the way Indians’ manager Terry Francona utilizes Andrew Miller. When the tall Indians’ lefty is sitting in the bullpen, he doesn’t know when he’s going to pitch (if at all) or for how many batters. The vast majority of high leverage relievers today know that they’re almost always going just one inning with the occasional 4-out save request. Simply put, the 6’8″ Miller is an anomaly in today’s game. Whether he turns out to be a prototype for a new type of relief pitcher (or a “retro” type, if you will) or whether he remains a spectacular oddity remains to be seen. However, the bias of WPA metric towards the one-inning pitchers who “know their role” and come in for a clean inning indicates that maybe, to some extent, the games’ managers know what they’re doing. What a concept?

So, have I led you down a rabbit hole with WPA? The answer is “no” but with conditions. What WPA measures does matter. If the majority of innings that you pitch involve the risk that you could turn a win into a loss, that’s significant. However, since we’ve seen that there’s an inherent bias favoring the relievers of the “one-inning closer” variety, it’s most relevant when measuring relief pitchers against others who pitched at the same time in history.

Despite its flaws, no less an authority than statistical wizard Nate Silver of (fivethirtyeight.com) calls WPA “arguably the best way to value relief pitchers.”

But Silver, being the numbers guy that he is, decided to try to find a better way to evaluate relievers. Just as Jerome Holtzman coined the “save” statistic, Silver has coined the term “goose egg” (in honor of Gossage, or to “troll him,” he writes). The basic version is that a relief pitcher gets credit for a “goose egg” in each inning pitched as follows:

  1. They pitch a scoreless inning without giving up a run in situations in which the pitcher’s team has a lead of two runs or less in the seventh inning or later or when the score is tied.
  2. A pitcher can get credit for a goose egg with leads of more than two runs provided the tying run is on base or at bat.
  3. If the pitcher records less than three outs, he can still get a goose egg if the total number of outs recorded and inherited runners on base totals three or more.
  4. A pitcher can get more than one goose egg in a game (which helps credit the stoppers of the 1970’s and 1980’s when “saving” a game twice or more).
  5. In order to get the goose egg, no runs can cross the plate after the pitcher enters the game in that inning.

Anyway, from Silver’s research (you can access the full piece by clicking here), here are the top 20 pitchers with most “goose eggs” all-time:

WP Table Builder

What you have here is another counting statistic that still requires some context but it is the best that I’ve seen that properly credits the extra work of the multi-inning stoppers. At the very least, it’s a significant improvement over the “save” statistic. By the way, if you’re wondering where Eckersley is here, Silver credits Eck with 352 goose eggs (compared to 390 saves), which is not even close to the top 20.

Of course, there’s the flip side of the “goose egg,” which Silver calls the “broken egg,” when a pitcher gives up at least one earned run and does not finish the game with his team on the winning side. And yes, at the top of this category are the same three leaders in “goose eggs,” Fingers, Gossage and Wilhelm. Silver then proceeds to go really deep into the weeds, creating a “goose egg” version of WAR, which he calls GWAR. I won’t go into detail here because I have one issue with the methodology (it’s either that I don’t fully understand it or that it favors the modern closer, probably that I don’t fully understand it). Suffice it to say that Mariano Rivera still tops it, by a lot.

 

There’s one final way that I can recommend that one can compare pitchers across eras and that’s by comparing, on a year-by-year or multi-year basis, how they ranked against their contemporaries. If you want to look at which relief pitchers had lowest ERA on a year-by-year basis that’s one way to do it. Or you could check out the leaders each year in WHIP, or strikeout-to-walk ratio, or the advanced metrics of WAR, WPA and ERA+. I like using ERA+ more than the traditional ERA because it adjusts for ballpark effects. An ERA of 3.00 in the Oakland Coliseum is not the same as an ERA of 3.00 at Wrigley Field.

I’m going to share the full results of the research I did about all of this in the Lee Smith comment in Part Seven of this series, but I’ll tease you with this: Mariano Rivera is still #1. In ten different seasons in his career, Mariano was in the Top 5 of the major leagues in ERA+. By comparison, Goose Gossage did this five times. Nobody else has even managed it more than four times.

Ultimately, even with the various methods shared in this piece, comparing relief pitchers across eras remains a complex endeavor and one that is easily influenced by personal bias. But this, of course, is what the Hall of Fame debate is all about. The exercise is to get 75% of the baseball writers with the voting privilege to concur. More often than not but not always, the writers have gotten it right.

In Part Seven, we’ll take a look at whether there are already too many relief pitchers in the Hall of Fame (or whether there will be too many after the inevitable inductions of Hoffman and Rivera). And, of course, we’ll give a thumbs up, thumbs down, or TBA for the best relief pitchers in the history of the game.

Click here to continue to Part Seven.

Thanks for reading.

Chris Bodig

Leave Your Thoughts, Comments or Snide Remarks