More Log5 stuff

December 13, 2015
                                                                   More Log5 Stuff

 

            At some point apparently in the recent past, the Journal of Sports Analytics published an article entitled "Bias in the Log5 Estimation of outcome of batter/pitcher matchups, and an alternative," the article written by Leslie C. Morey, from the Department of Psychology at Texas A & M, in collaboration with Mark A. Cohen from the Department of Computer Science at "the Massachusetts College of Liberal Arts".   If you are wondering what the Massachusetts College of Liberal Arts is or where it is, I BELIEVE that it is in North Adams, Massachusetts.    I was sent a link to this article by a gentleman who is on our site, and I would thank that gentleman here if I were certain that there were no accidental or incidental violations of copyright law involved.   Thanks, anonymous guy; you can sign on and take credit if you’re a’ mind to.

            Anyway, this article contains several statements which either (a) are just totally incorrect, or (b) I failed to understand.    If I failed to understand these comments, just to fill out the decision tree, that could be either (a) because I lack the educational background to participate in this discussion, or (b) because I am just too impatient to fully decode whatever it was they were actually saying.   We’ll proceed on the assumption that I DID understand the article, and then we’ll sort out the discrepancies later, OK?   Here’s one:

            As an example, some HR% values for such "outlier" home run hitters approach or exceed .60, meaning that log5 might estimate that Barry Bonds could be expected to hit 300 home runs in 500 AB if placed in a league resembling the 1920 NL with respect to HR%..

 

            Uhh. . .No.    Again, not SURE that this is what they were trying to say, but the Log5 method would predict that, in the National League in 1920, Barry Bonds (2001) could be expected to hit 16 home runs in 500 at bats—one more than the number of home runs that the National League Home Run Leader in 1920 (Cy Williams) actually hit.   Williams hit his 15 homers in 590 at bats, not 500 home runs, so Williams (in context) is not quite the equal of Bonds in 2001.

 

            This can be shown in two relatively easy steps.   In the examples I printed last week we were dealing with 5 steps or more than five steps, because those examples also involved an individual pitcher or a specific defense in basketball.   This example involves only a batter and two different environments, no specific pitchers, so it is a much easier calculation.   First, we compare the ability of the National League to prevent home runs in 1920 to the ability of the league to prevent home runs in 2001.  In the National League in 2001 there were 2,952 home runs in 88,100 at bats, which means that home runs were NOT hit in 85,148 at bats, which is .966 of all at bats.  In the National League in 2001 there were 261 home runs in 42,197 at bats, which means that there was NOT a home run in 41,936 at bats, or 99.4% of at bats.

 

            The National League in 1920 was thus obviously much stronger at preventing home runs than was the National League in 2001; we might accidentally say here (somewhere) that the PITCHERS were stronger at preventing home runs, but in fact the method assumes that the pitchers of 1920 were exactly the same as the pitchers of 2001, and that the difference was accounted for by the conditions of play—the balls that were used, the bats that were used, the rules, the fields, the mounds, the umpires, etc.    Steroids.  Anyway, the environment in 1920 is obviously much stronger at preventing runs than the environment in 2001:

 

 

NL Environment 1920 vs NL Environment 2001

.994

.966

80.337

14.422

.848

 

            The NL Home Run prevention environment of 1920 has a Log5 of 80.337, whereas the NL Home Run prevention environment of 2001 has a Log5 of 14.422.   Thus, when the NL in 1920 is compared to the NL in 2001 in this respect, 1920 has a winning percentage of .848. 

 

            Bonds in 2001 hit home runs in 15.3% of his at bats.    All we have to do now is to plug that number which describes Bonds’ home-run hitting ability (.153) into an environment which is much stronger at preventing home runs than was the National League in 2001:

 

NL Environment 1920 vs NL Environment 2001

.994

.966

80.337

14.422

.848

Bonds vs. 1920 NL Environment

.153

.848

.090

2.785

.031

 

            Bonds’ could thus be expected to hit home runs in 3.1% of his at bats—15.7 home runs in 500 at bats.   By this simple two-step process, you can place any player from any era in any home run environment.    Barry Bonds, 2001, in the National League in 1954:

 

NL Environment 1954 vs NL Environment 2001

.973

.966

18.363

14.206

.564

Bonds vs. 1954 NL Environment

.153

.564

.090

.646

.123

 

 

            The first figure there (.973) is the percentage of NL at bats in 1954 which did NOT result in a home run.  Bonds in 1954 NL could be expected to hit 58 home runs in 476 at bats, which was his actual 2001 at bat total.   The NL leader in Home Runs in 1954 was Ted Kluszewski, who hit 49 homers in 573 at bats.   Kluszewski could be expected to hit 62 home runs in the NL in 2001, given the same number of at bats and the assumption that the quality of talent is the same.  

 

 

NL Environment 1954 vs NL Environment 2001

.966

.973

14.206

18.363

.436

Kluszewski vs. 2001 NL Environment:

.086

.436

.047

.387

.108

           

            Kluszewski, hitting home runs in 8.6% of at bats, could be expected to hit homers in 10.8% of at bats facing a weaker home-run prevention environment.    If we compare Bonds to Kluszewski to Cy Williams in 1920, we can see that one is not all that much different from another, relative to the environment:

 

Bonds Vs. Average Pitcher (1920)

.153

.034

.091

.017

.839

Kluszewski Vs. Average (1954)

.086

.027

.047

.014

.775

Williams Vs. Average Pitcher (1920):

.025

.006

.013

.003

.807

 

 

            Bonds has a "raw Log5" of .839 as a home run hitter in 2001, Williams of .807 in 1920—a figure that we don’t actually need in this calculation.   Bonds in 2001 hit home runs four to five times as often as an average National League player.   Cy Williams in 1920 also hit home runs four to five times as often as an average National League Player.   Thus, it should be intuitively obvious that, when we transfer Bonds (2001) to the 1920 season, he is NOT going to hit 300 home runs.  

 

            Tango wants to worry about the Random factors here, which I think generally are a waste of time.  However, in the case of Cy Williams in 1920, they would have to be a real thing that you would have to worry about.   One more home run for Cy Williams would be a 7% increase in his home runs, which makes a huge difference in adjusting him to a different environment.    When there are not very many home runs hit, then randomness is a larger factor in determining the EXACT relationship between the player and the league. 

 

 

            Another place where the authors have written things with which I might take issue is as follows:

 

            Comparing these values to the batter, pitcher, and league average numbers is informative to the hypothesis of biased estimation results from the Log5 formula.   First, consider Table 1, reflecting a total of 20,000 different comparisons of batter, pitcher and league average batting average (BA).   In every one of these 20 trials, application of the Log5 formula resulted in an estimated BA that was higher than any of these three parameters used to calculate it—batter, pitcher, or league average. 

 

            This then (as you see) refers is to Table 1, which has these values;

 

 

Batters-

Pitchers-

League--

Sample

Mean

Mean

Mean

1

.30465

.27113

.26737

2

.29925

.27469

.26583

3

.30070

.27314

.26710

 

            And the table contains 17 more samples and several more columns of interpretation.  

 

            What the gentlemen failed to notice was that in every case, in every sample, 20 out of 20, both the Batter’s Mean and the Pitcher’s Mean was HIGHER than the League Mean.    That means that BOTH the batter and the pitcher are pushing the "outcome" batting average HIGHER, relative to the league average.     In every one of these 20 trials, application of the Log5 formula resulted in an estimated BA that was higher than any of these three parameters used to calculate it—batter, pitcher, or league average because that is the correct answer.   The outcome (in these cases) SHOULD be higher than any of the other three parameters, so it is.  

 

            One would think that this was obvious.     Let’s start with won-lost records, because won-lost records are one step more simple than batting averages.     Let us suppose that a .600 team is playing a .400 team in a league in which the overall winning percentage is .500.    The .600 team beats ALL teams—thus beats an average team—60% of the time.   When they play a .400 team, will they win:

 

            a)  Less than 60% of the time, or

            b)  More than 60% of the time.

 

            Obviously, a .600 team is going to beat a .400 team more than 60% of the time.    In fact, they are going to beat them 69.2% of the time, and if you don’t believe me, look it up.  

            So, in the language of the authors of the study, there are three parameters here--.600, .500, and .400, and the outcome predicted (.692) is higher than any of these three parameters.  

 

            Well, OF COURSE it is higher than any of the three parameters.   If both factors are pushing the percentage UP, it goes higher than either individual parameter or the league average.  

 

            The same with batting averages.   If you put a .305 hitter against a .271 pitcher in a league in which the overall batting average is .267, he WILL hit higher than .305.    This should be obvious.    It’s not a "bias"; it is a correct calculation. 

 

            A third point on which I would disagree with the authors of the study has to do with this phrase, in the Introduction to the article:

 

            James called this method the "log5" method (although it is based upon the Bradley-Terry, 1952, model for pairwise comparison.)

 

 

            I have absolutely no idea who Bradley and Terry were, and my research quite certainly is not "based upon" this model.   It may well be that Bradley and Terry independently discovered or independently developed the same method that I did, 25 years before I did it, and if that is true than they should indeed be given credit for that, but the correct phrase would be that the method or some elements of the method are parallel to the Bradley-Terry approach, rather than that my work is based upon theirs.  

 

 

 
 

COMMENTS (10 Comments, most recent shown first)

jgf704
One more comment about Morey and Cohen... I don't understand why their equation 3 would be right. Whenever I see equations, I try to check if expected results are recovered for simple cases. If the opposing pitcher is a league average pitcher (p_P = p_L), I would expect the predicted p_B.P (the event probability given a particular batter-pitcher matchup) would be equal to the nominal batter probability (p_B). But this is not the case. You get:

p_B.P = (p_B - p_L)/sqrt(2) + p_L

Am I missing something?
8:11 AM Dec 17th
 
jgf704
A couple points:

* I think it actually is possible that Morey and Cohen got ungodly high HR rates in some case. But I'm guessing this happened because they applied the wrong formula (i.e. they directly applied the Bill James / Dallas Adams formula, i.e. their equation 2).

* I was a bit confused by Bill's explanations in his two recent log5 articles. The formulas corresponding the explanations certainly give correct results, but I was confused nonetheless. I tried to flesh things out a little differently.

See post:

boards.billjamesonline.com/showthread.php?5839-Comments-on-LOG5
1:05 AM Dec 17th
 
studes
Dave, I lived in North Adams for a summer. If it's your favorite small town in America, I'm worried about you.
10:48 AM Dec 16th
 
rfleming
One wonders if a response will be forthcoming from the authors.

Certainly the third point can be handled easily by just agreeing that the wording should have been something like "although it parallels or was preceded by the work of Bradley-Terry."

The first point also seems to be just a transposing of numbers as tangotiger suggests. And so should result in an admission of such a mistake.

The second point seems the more problematical. It appears to be a hasty judgement or lazy bit of reading or work based on not looking fully at the data. This is where a response might be expected. But as I began by asking: will the authors even take notice?
9:46 AM Dec 16th
 
DaveFleming
MCLA is, in fact, located in North Adams, MA.

North Adams is one of my favorite small towns in America. No...that's not accurate. It's my favorite small town in America, full-stop.
5:23 PM Dec 14th
 
tangotiger
"meaning that log5 might estimate that Barry Bonds could be expected to hit 300 home runs in 500 AB if placed in a league resembling the 1920 NL with respect to HR%"

Bill is right. And I'll bet what the authors did was transpose the league average numbers! Instead of asking what would a guy who hit 70 HR in a league of .03 HR / AB do in a league with .01 HR/ AB (or whatever it was), it instead asked what would a guy who hit 70 HR in a league of .01 do instead in a league of .03! I'm sure they did something absurdly wrong like that.

Otherwise, as they noted, they have r=0.99 in all their data. You can't possibly have an outlier of this size and still have r=0.99.

Still reading...
1:57 PM Dec 14th
 
tangotiger
"However, this approach scales these binomial probabilities using a Z-score or standard score metric (mean?=?0, SD?=?1). These binomial probability Z-scores are then aggregated for the batter and the pitcher, and this aggregate is rescaled into the expected league average distribution"

This sounds like what Michael Shuckers does, transforming data to specific distributions, then untransforming them.

So far, the authors are bringing up valid points.

Still reading...​
1:50 PM Dec 14th
 
tangotiger
This part is true, and I've shown this in the context of hitters facing Pedro several years back:

"Specifically, this particular odds-ratio estimation strategy may be most effective when the true mean proportion of observed probabilities is.500 (the “5” in log5 is actually in reference to the.500 mean proportion), and when the relevant variables to be estimated are normally distributed around that mean. "

Specifically, if you applied log5 to OBP to each of Pedro's hitters, and added them up, you wouldn't end up with Pedro's OBP allowed. But, it's not too far off either. You'd be off by say .002 or something. Tiny bias.

Still reading...
1:48 PM Dec 14th
 
tangotiger
The article is here:

content.iospress.com/articles/journal-of-sports-analytics/jsa0005
1:44 PM Dec 14th
 
tangotiger
I remember seeing the headline of that article in the journal, and skipped over it, presuming the authors were going to simply give the reader a primer on Log5. Looks like I need to read it.

***

In terms of "invention" and "timeline", one can say that it is equivalent to Odds Ratio and Bradley-Terry. But one obviously can't say it was "based on" if it was done independently.

It would be like saying that Voros' DIPS was based on Bill's DER (defensive efficiency record). Or, you can say that it is equivalent to the flip side of DER.
12:18 PM Dec 14th
 
 
©2019 Be Jolly, Inc. All Rights Reserved.|Web site design and development by Americaneagle.com|Terms & Conditions|Privacy Policy