Intro to Sabermetrics

April 26, 2008


Trying to explain what it is that we really do.



(I have been asked to speak to a Risk Management Seminar being organized by someone who has some connection with the Red Sox.  I thought about how I would explain Sabermetrics to a group of intelligent people who didn’t really get what we doing, and this is what I came up with.   The following is the text for a talk that will be delivered in mid-June, and, since I had to write it, I decided to share it with you.)






            Good afternoon, and thank you for inviting me to speak to you for a few minutes here.    I have been asked to say a few words about sabermetrics, about what I do for the Red Sox, and about how this might relate to risk management, if it does.

            Sabermetrics is descended from traditional sportswriting.   Sportswriting consists of two types of things—reporting, and analysis.  Sabermetrics came from that part of sportswriting which consists of analysis, argument, evaluation, opinion and bullshit.   I can tell you very precisely when and how we parted ways with traditional analysis.

            Sportswriters discuss a range of questions which are much the same from generation to generation.   Who is the Most Valuable Player?   Who should go into the Hall of Fame?   Who will win the pennant?   What factors are important in winning the pennant?   If Boston won the pennant, why did they win it?   If Kansas City finished last, why did they finish last?   How has baseball changed over the last few years?   Who is the best third baseman in baseball today?   Who is better, Mike Lowell or Eric Chavez?

            The questions that we deal with in our work are the same as the questions that are discussed by sports columnists and by radio talk show hosts every day.   To the best of my knowledge, there is no difference whatsoever in the underlying issues that we discuss.   The difference between us is very simple.   Sportswriters always or almost always begin their analysis with a position on the issue.   We always begin our analysis with the question itself. 

            If you find a sportswriter debating who should be the National League’s Most Valuable Player this season, his article will probably begin by asserting a position on the issue, and then will argue for that position.   If you find 100 articles by sportswriters debating issues of this type, in all likelihood all 100 articles will do this.  

            What we do is simply to begin by asking “Who is the National League’s Most Valuable Player this season?” rather than to begin by stating that “Albert Pujols is the National League’s Most Valuable Player this season, and let me tell you why.”  That’s all.   That is the entire difference between sabermetrics and traditional sportswriting.  It isn’t the use of statistics.   It isn’t the use of formulas.   It is merely the habit of beginning with a question, rather than beginning with an answer.

            From this very small difference, profound changes arise.   A person who begins by asserting a position on an issue naturally focuses on what he KNOWS.   “What facts do I know, he asks himself, which will help me to explain why Mike Lowell is better than Eric Chavez?”

            The person who begins with the question itself. ...who is better, Mike Lowell or Eric Chavez. . ..the person who begins with the question itself naturally focuses not on what he DOES know, but on what he does NOT know.   I misstated that just a little bit.   The person who begins with the question itself naturally focuses first on what he NEEDS to know.    As soon as he begins to think through what he needs to know, however, it will become apparent that he does not know many of the things that he needs to know to really answer the question.

            Every large general question in baseball can be broken down into smaller and more specific questions, which can be broken down into yet smaller and yet more specific questions, which can be broken down into yet smaller and yet more specific questions.   Eventually you reach the level at which the questions have small and definite answers, even though you may not know what those answers are.  The question of who is better, Mike Lowell or Eric Chavez, for example, leads to the question “What are the elements of a third baseman’s job?”   Well, there is hitting, fielding, baserunning, and off-the-field contributions if you want to count those.  

            The question of who is a better fielder breaks down into 20 other questions.   Who is quicker?  Who is more reliable?  Who makes more mistakes?  Who has a better arm?  Who is better at anticipating the play, and positioning himself correctly?   Who is better at going to his left, or to his right?  Who is better at applying the tag if he needs to make a tag on a runner?

            At that level, the questions that we ask ourselves are actually very much like the questions that scouts ask.   Scouts try to answer them by their expert judgment.   We are not experts, and so we try to find ways to answer them that do not rely on our judgment.  In that way, sabermetrics forms a kind of transition between journalism, from which it descended, and scouting.   In any case, the question of who is better at applying the tag if he needs to make a tag play leads to the question “How many times a year does a third baseman have to make a tag play?”    That is a question that has an actual answer.   I don’t know what the answer is, but there is an answer to that question.   In a few years, somebody will figure out a way to get that answer. 

            The question of who is better, Mike Lowell or Eric Chavez, contains within it probably several hundred smaller questions.   The person who begins by asking that question naturally realizes that he does not know the answers to most of those questions.   He then proceeds inevitably to ask, “How can I find the best answer to that question?” 

            When a person begins with the question itself he inevitably winds up confronting his own ignorance, and trying to find ways to fill in the gaps in his knowledge.   The person who begins with a position on the issue never sees his own ignorance, and, in fact, deliberately avoids seeing his own ignorance.  The person who begins with a position on the issue and argues for that position naturally tries to hide his ignorance of the other internal issues, since the things that he doesn’t know are a weakness in his argument.   The person who begins with the question itself, on the other hand, inevitably winds up reveling in his own ignorance, celebrating his ignorance, and sharing it freely with the world at large.  

            But the person who begins with a position on the issue, by this process, becomes a borrower from the Bank of Knowledge.   He borrows from the things that he knows, and uses them to construct an argument.

            The person who begins with the issue itself, on the other hand, eventually becomes a contributor to the Bank of Knowledge.   Forced to confront his own ignorance, he is forced to find ways to figure out the information that he is missing—ways to count things that haven’t been counted, or ways to estimate the parameters of things that are unknown.    Through this process, he winds up knowing things that were not known before.  

            This is essentially what we do:  We try to construct knowledge to fill in some of the spaces in our massive ignorance.   We are not people who know things.   We are people who are honest enough to admit that we don’t understand things, and frankly, we don’t believe that you understand them, either.  

            Because we have been involved in the effort to figure out the things that we don’t understand for a long time now, we have developed an inventory of a few hundred standard methods that can be used to analyze a new question.   What I do for the Red Sox is to try to draw from that inventory tools that can be used to study whatever it is that the Red Sox need to study.   Suppose that we are considering trading for a new pitcher, for example.   Our General Manager might ask me “What do you think is the chance that this pitcher will retain value over the life of his contract?”. .. the “risk management” that I am supposedly discussing here.   There are several ways to approach that problem.   One is to look for similar pitchers.   If you can identify 25 other very similar pitchers over the last twenty years, you can study their health records as they moved forward.   You can do a characteristics analysis.   There are observable characteristics of durable pitchers, such as a high strikeout rate and a record of moderate but consistent workloads, and you can compare this pitcher to the characteristics of healthy pitchers.

            When our GM or one of his associates asks me a question, I almost never give him an immediate answer—nor does he expect me to.   If he wanted an immediate answer, he would ask an expert, or he would ask a scout.   What I always say, when asked anything, is “I’ll study the issue.”

            I shouldn’t say “never”.   Occasionally I’ll slip up and give an immediate answer.   A year ago, for example, Theo asked me “How confident are you that Dustin Pedroia will hit what your projections show that he will hit?”   Without thinking about it, I said “Absolutely 100%.”  I knew three things that caused that to slip out.   First, I knew that our hitting projections for minor leaguers have a very high degree of reliability—much higher than most people assume that they do.   Second, I knew that our projections for Dustin Pedroia were actually very conservative, that he was actually a better hitter than we were saying that he was.  Third, I knew that, even though Pedroia had hit .191 as a late-season callup in 2006, he had made contact with more than 90% of his swings.  If you swing as hard as he does and you make contact with every swing, how can you possibly fail to hit?

            That’s risk management, of course. . . .knowing how reliable the projections are.   But that’s also very much the exception to the rule, actually—the exception to two rules.   The first rule is:  We are not experts.   We are not people who “know” things about baseball players.   We are people who know how to study things about baseball players.   When Theo asks me a question about a player or about the game in general, he is not asking me for what I know.   He is asking me to study the issue. 

            And the other rule is, we are never certain.  I said that I was 100% certain that Pedroia would hit in the majors, but in reality, we are never 100% certain of anything.   The talk show hosts are 100% certain.   The sports columnists are 100% certain.   We are just doing the best we can.   Our methods are always flawed, and our answers are usually tenative and muddled.  

            But the difference between knowledge and BS is that knowledge moves forward, whereas BS moves in circles.   When I develop a new method to analyze a baseball question, someone will point out the flaws in that method, and then someone else will suggest an entirely different method to approach the problem, and then somebody else will point out the flaws in that entirely different method, and then somebody else will figure out a way to combine the best features of both methods into one method that is better than either one.  We wind up with methods that get better over time. 

            And also, we wind up with lots of numbers, which is why people perceive us as statisticians.  Going back to the number of tag plays made at third base.   It’s a significant piece of information that doesn’t exist.    It’s very difficult to take a throw from right field—a long throw—and apply the tag to a fast-moving target which is taking evasive action.   It’s not an easy thing to do, and if the third baseman makes an extra play in that situation, that has the same value as hitting a triple.   Eventually, somebody will find some way to measure this, and the product of this will be stated as a number, which, since we are dealing with baseball, will be called a statistic.  

            We are no more statisticians than we are historians, or scouts, or accountants, or computer programmers.   I suspect that everything we do is much the same as what many of you do.  We look to the past, and we try to organize the things we have seen so that they make some sense.   We ask ourselves “how many of those were there?” and “how many of those others were there?” and “How many of them ended well?” and “How many of them ended badly?”, just as I would imagine most of you do.  

            Often people wonder if we will run out of things to study, and what I always say is that there will never be a shortage of ignorance.   I have realized recently that some people take this wrong, and that when I say that there will never be a shortage of ignorance people think that I am referring to some other group of people, some “ignorant” group of people, when in reality what I am talking about is myself.   I will never understand baseball; I will never understand 1% of what I need to understand.   My view of our work is that we are attacking a mountain range of ignorance with a spoon and a used toothbrush.   The things that we do not know are inexhaustible.  

            But our time here is limited, and I appreciate your patience. 


COMMENTS (10 Comments, most recent shown first)

The idea that some are "borrowers from the bank of knowledge" and that others are creators and contributors to the bank of knowledge is apropo and profound. Like the ape staring at the bone during the opening of '2001: A Space Odyssey', creating knowledge - imposing intelligence on dumb matter - is what mankind is meant to do.

Alas, I think you have to move to the Eastern countries to find a modern culture which now celebrates creation and innovation: we Americans don't want to create or build (we don't even want to create life, just look at our birth rates), we just want to destroy and disparage.

A wonderful article with great ambition behind the frank and modest tone. These ideas should be intrinsic, but I guess it's better to find a voice in the wilderness than nothing at all.

Now I just hope that the corporate-crats weren't so inspired by it that they found newer ways of suckering people into buying their worthless crap!
2:40 AM Mar 20th
So you really shouldn't have answered Theo "100%--absolutely" re Pedroia ; maybe you should have said : "As close to 100% as these things get"...
6:08 AM Jun 6th
Wow, Bill!!! I'm inclined to go back through this piece and insert job specific situations and hand it to my new boss!! I've been with NIKE for 16 years and have been a successful senior analyst for the last 4 years. I can't tell you how many times I have been requested by my new boss to not only deliver a number in 5 seconds but to also give the 'stories' behind the number (or take a position). I keep trying to tell her that it's my job to provide her with the best data possible so she can make a intelligent, informed decision and choose a position or 'story' as she likes to call it. I refuse to take a position because I know nothing is 100% certain. My job is to question all the 'stories' that people in her position and above are continuously stating. To me, that's the only way we are going to continue to improve as a company; to ensure we are basing our assumptions on accurate data. I think your article will help my new boss better understand what it is I'm aiming to do for her, our team and NIKE as a whole. In short, if you are wondering if your speech will be relevant to this business audience, my educated guess is YES!! with a high degree of certainty!
7:51 PM May 24th
This is more than just a description of sabermetrics. It's actually a very cogent description of research, in general. I do research in health policy and the way I address an issue is very similar to way Bill addresses one (although I'm sure his results are more interesting). In fact, one of the fun things about my job is when I can demonstrate that the conventional wisdom is wrong. Which, as we baseball statheads all know, happens frequently.

Despite Bill's admonition that this isn't really statistics, I have to believe there is a role here for statistics (in the academic sense). Where there are numbers, one can determine correlations, significance, etc.

Nice job, Bill, as usual.
5:05 PM May 16th
Important content, simple and compelling presentation. Deserves broad distribution, well beyond blog readers and Risk Managers.

It seems to me that there is a parallel in our political life. We are overrun with talking heads and political columnists taking and defending positions on the critical issues of our times -- but we need a Sabermetric approach to analyze the questions behind these issues, to make clear what we don't know and crystallize what we do so that we can make better decisions.

7:09 AM Apr 29th
Now this is EXACTLY why I subscribed to this website. This is a classic Bill James piece, which I am now going to print 100 times and hand out along with the 1985 BA comment on the Astros (for which I have supplied my own title: Why Men Like Baseball Statistics"). A statement of the issue, an extended comment on the issue which shows why ignorance still exists, a (RE-)statement of why it's incorrect to call him a statistician, and an off-hand statement which shows where he thinks this might go as far as "value" is concerned, and a little bit of humor to ease it down with. This is not "journalism", this is great writing.

2:12 PM Apr 28th
Terrific piece. I think for many people "stats" is the easiest shorthand and you did a great job of explaining the approach.
11:37 AM Apr 28th
Bravo Mr. James
1:58 AM Apr 27th
I appreciate the insight on the insight - certainty is never 100% - but enhanced probability in the direction of certainty is the goal. best regards to you web site - I enjoy it very much
8:56 PM Apr 26th
I like the manner in which this is presented....excellent overview.
8:48 PM Apr 26th
©2021 Be Jolly, Inc. All Rights Reserved.|Powered by Sports Info Solutions|Terms & Conditions|Privacy Policy