Defensive Runs Saved (DRS) is a system of metrics that has been developed over the last 12 years (going back to 2003) by John Dewan, Ben Jedlovec and the team at Baseball Info Solutions (BIS). Bill James himself has played a key role in the development of DRS as it was his suggestion to create a plus/minus type method at the outset of the research. DRS has multiple components that convert numerous techniques that measure individual aspects of defense (like catcher throwing, to name one) into a common currency. That common currency is one that all baseball fans understand: runs. Using a common currency was vital to insure that the numbers produced by the system could be easily understood.
In Bill James’ massive effort called The Fielding Jones on Bill James Online, Bill takes exception with some of the methods being used to develop Defensive Runs Saved. However, much of those exceptions are based on a misunderstanding of the system. Ben Jedlovec, President of Baseball Info Solutions, has been my partner in furthering the developing of Defensive Runs Saved since he joined BIS in 2008. Below are Ben’s comments regarding Bill’s misunderstanding that he wrote in an email to Bill right after Bill published his thoughts. I apologize in advance that it’s a bit technical in nature, but we want to set the record straight. The comments here are primarily about Bill’s concern regarding the way we estimate the run impact of each play, as described in Section XXXIII of The Fielding Jones entitled "The Marginal Run Question".
Ben’s email to Bill James:
"If my understanding of your argument is correct, I believe you are missing one or two crucial points. Let me try two different explanations, a short one and a long(er) one.
1) Your argument would be valid if we were to simply apply .75 times the number of plays made, and/or .75 times the number of hits allowed. However, that's not what we do because we factor in the difficulty of each play, which you could call the "expectation" that the play will be made. All of our Runs Saved components factor in the "expectation" for the event, given an average fielder at the position.
2) Offensive Linear Weights similarly have an "expectation", or baseline. It's zero. In other words, when the batter steps up to the plate, he could single (+0.45 runs), homer (+1.40 runs), strike out (-0.30 runs), or any number of possible outcomes. (Obviously, the actual run expectancy change of the play will depend on the runners and outs, but we're dealing with averages here as Linear Weights does.) The batter will make an out (-0.30) far more often than he'll get a hit, but the average of every outcome's run value will be 0.00. By definition. If the frequency of events changes, the run values of each event change to keep the average at 0.00.
You could explain our Range and Positioning calculation in the following way. Back to our batter stepping up to the plate. The expectation of this play is zero, 0.00. The pitcher and batter do their thing, and sometimes the ball is put in play. The ball is in the air or on the ground, weak grounder or screaming liner, on its way to something. En route, even before the play has been resolved, we can consider the expectation of the play to have changed.
It might be a screaming liner that is a sure single 100% of the time. In this case, the value of the play is now 0.45, the run value of a single.
It might be a pop up on the infield, a sure out 100% of the time. In this case, the value of the play is now -0.30, the run value of an out.
It might be a hard grounder towards third. Let's say we estimate that it's a single 50% of the time, and it's an out 50% of the time. In this case, the expectation of the play is now 0.50*0.45+0.50*(-0.30) = 0.075. (We're using a simple binary example, but note that we can use any number of outcomes with their respective estimated frequencies and run values.) The play has jumped from 0.00 to 0.075, so that hard grounder was a positive outcome for the batter.
Now we get to evaluate the fielder. If he makes the play, it's an out. We credit the fielder 0.075 - (-0.30) = +0.375. If he fails to make the play, it's a single, and we penalize the fielder 0.075 - 0.45 = -0.375. So, while the difference between the hit and the out is 0.75, we're actually giving/penalizing the fielder a fraction of that, depending on the difficulty (expectation) of the play.
The Defensive Runs Saved system's job is to accurate estimate the difficulty, or "expectation" of each play given a league average fielder, since we use a league average baseline for each position. "
Bill’s New Technique
Bill wants to do fielding metrics a different way, as described in Section XXXVIII "What I would Like to See in Fielding Statistics". He wants to re-create what we called Where Hits Landed, as published in the first Fielding Bible. He lamented how this information should be made available to the public, or in his words "to foster the creation of a organized universe of fielding data." But he forgot that this information was published before and provided it to Bill (and the public) back in 2006 in the first Fielding Bible. There was minimal interest at that time in this data by the public, or by Bill himself, so we discontinued the report.
However, we will again provide that information to Bill to use as he wishes. Baseball Info Solutions is giving Bill a spreadsheet that gives all of this data going back to 2003. If and how Bill wants to make this available to the public through Bill James Online, we will leave that up to him. He may decide to provide only the most recent data because, as he puts it "the value of data from 2006 is minimal in any case." Or he may provide just his analysis. It’s up to him.
There is no question in my mind that Bill can do some valuable things with this data. He will come up with some very good information and it will be interesting to see how it compares to the one component of DRS that he is trying to measure: the Range and Positioning component of DRS. Having said that, I don’t think it is possible to use less data that has far less detail and get a better result.
Is it possible that Bill will come up with a better way to present defensive data? Certainly. People will be developing baseball information 100 years from now and will continually come up with better and better ways to create and present defensive analytics. One of the most important reasons for this is that there will continue to be better and better data to be analyzed. Plus, there are certainly plenty of avenues of research that we haven’t yet pursued with the DRS system.
The proof is in the pudding. If a new baseball metric is measuring something well, it needs to be demonstrated. The DRS system has been studied in this manner in many, many ways, by ourselves in several independent studies. Research on each component of the system has shown each to be consistent and predictive. On top of that, the results need to be consistent with what people predominantly see with the naked eye. A high percentage of the players known for good defense should come out with good numbers otherwise credibility is lost. That is true of the DRS system.
Of course, not everyone has bought into defensive metrics. We’ve spent more time and energy on the subject that just about anybody (we’ve published four books on the topic, for goodness sake!), so we know the Defensive Runs Saved system and its results better than anyone. However, not everyone takes our word for it, and that’s fine. We’re hopeful Bill can find better ways to present the information. He’s been very successful over his entire career doing exactly that. We hope that it will even add to our understanding of defense.
But will it change the DRS bottom line, that Andrelton Simmons is an incredible defender and that Matt Kemp is a liability in the field? We doubt it.