This study is not about similarity scores, although it does use a method which is somewhat like similarity scores, and it is not about fluke seasons, although it does point the finger toward fluke seasons.  It is not about consistency, although the subject also overlaps with consistency.   I have written about all of those things before.  My focus at the moment is on the question of whether two seasons are or are not likely to be products of the same hitter.

                Let us suppose that we take these two seasons:

One might guess that those two seasons were two seasons of the same hitter, and indeed, you would be correct if you were to make that guess; those are the 1974 and 1975 seasons of Marty Perez, who was at that time the Atlanta Braves’ second baseman.

On the other hand, one might guess that these two seasons are not two seasons of the same player:

And, of course, they are not two seasons of the same man; those are the seasons of Mickey Mantle in 1956, and Ozzie Smith in 1986.

In those two cases it is easy, but what if it is not easy?     Let us take these two seasons:

Or these two:

Or these two:

Are those two seasons of the same hitter, or are they not?   And can one tell?

Well. . .no, one can’t reliably tell; let me get that issue off the table.   It does happen that different players have seasons which are similar on every scale, and it does happen that one player will have two seasons which are radically different.   You can’t always tell.   The six seasons above are by Lou Brock (1964), Ian Desmond (2010), Steve Yerkes (1915), Fernando Vina (2002), Rico Petrocelli (1971) and Rico Petrocelli (1972), but. . .you can’t always tell.  The zero caught stealing by Yerkes is not an actual zero; it is an absence of information.

This method has to do with, for example, scoring the accuracy of predictions—and in particular, the accuracy of predictions for rookies or for players from Japan or Cuba, for whom we have to translate the data.  We have traditionally scored our predictions by the use of similarity scores.  Similarity scores, however, rely upon and are content with accidental similarities.  If we say that a player should drive in 65 runs and he drives in 72, we’re happy.

A lot of those similarities, though, are just luck.   We have to guess the player’s playing time.  If we guess the player’s playing time accurately, everything else will fall in place most of the time.   That’s just luck.   On the other hand, if we project a player to bat 570 times and he gets hurt, the projection looks bad, but that’s just luck, too.  I’m looking for a method that takes some of the luck out of it.

It has been pointed out that if you take our projections for rookies and "score" the accuracy of, for example, batting average, it would be just as accurate—or more accurate--to predict that every rookie would hit .260.   That’s true, but it’s not exactly accurate.  The normal year-to-year variation in batting average for regular players (400 or more plate appearances) is almost as large as the standard deviation of batting average.     The standard deviation of batting average with 400 or more plate appearances is 31.8 points, less than that if you stick to batting averages within an era.    The normal year-to-year fluctuation in batting average with 400 or more plate appearances is 24.4 points.

Since the normal year-to-year fluctuation in batting average is nearly as large as the standard deviation of batting average, it is nearly impossible to predict batting averages more accurately than simply predicting that everybody will hit .260.  Since many of the rookies that we print projections for will have only six at bats and will hit .000, that creates extreme projection-to-reality variation, which means that it may be entirely impossible to predict batting averages for rookies more accurately than simply predicting that everybody will hit .260.

                We’re asking the wrong question, and I’ve been aware for years that we were asking the wrong question, but I’m finally trying to do something about it.   The question we have been asking is, "How similar are these two records?"   The question we need to ask is "How improbable is it that these two records—the projection and the actual record—would be produced by the same player?"   That’s the question here:  How improbable is it?

                I’ve designed a method to tackle that question, and I’ll explain the method, but that will be the last thing I’ll do; before I explain the method I’ll give you all the other stuff.  First, highly improbable seasons that were in fact produced by the same player in consecutive seasons.  The all-time king is Fred Dunlap, 1884-1885:

                Dunlap hit .412 and scored 160 runs in 101 games in 1884, and hit about like Marty Perez in 1885.  Obviously, something fishy was going on there.   Dunlap in 1884 was playing in what was not only a minor league, but a very bad minor league, which, by the whims of history, happened to be designated by bad historians as a "major" league, and which therefore shows up in our data.   The very dramatic changes in his performance are exaggerated by changes in the data; we have him with zero RBI and zero strikeouts in 1884, because the bad minor league he was playing in didn’t keep records of those things.

                Actually, most of the most improbable sequences in history were produced in the 19^th century, for reasons similar to Dunlap’s—the available data is always changing, and new leagues were always forming and folding.    Second on the list is Tip O’Neill, 1886-1887, third is Orator Shaffer, 1884-1885, and fourth is Chicken Wolf, 1890-1891.

                OK, we’re going to ignore the 19^th century from now on; it’s not really major league baseball.  The most improbable combination of adjacent major league seasons since 1900 was by George Sisler, 1922 and 1924:

                Sisler’s batting average dropped 115 points, his strikeout to walk ratio deteriorated, and he lost 60% of his stolen bases.  We know why this happened:  Sisler missed the 1923 season with an infection of the eyes, and when he returned he was not quite the same player as before.

                Of course, Sisler’s 1922 and 1924 seasons—while we’re not going to ignore them, like the 19^th century—don’t actually qualify as consecutive seasons, either.   The leaves as the most improbable consecutive seasons by any player since 1900 a pair of seasons that we have recently been discussing on this site:  George Scott in 1967-68:

                It is not a normal thing that a player will hit .303 with 19 homers one season, and .171 with 3 homers in 124 games the next season.  We’re looking at the outer boundaries of what is possible here.  How dissimilar can two seasons be, and still be produced by the same player?   That’s the limit—George Scott, 1967-68; that’s how dissimilar it can get.   After George C. are Rogers Hornsby, 1925-26, George Burns, 1917-1918, Darin Erstad, 2000-2001 (actually 1999-2000-2001), Norm Cash, 1961-62, Max Carey, 1925-26, Jeff Heath, 1940-41, Dave Roberts, 1973-74 Harry the Hat Walker (1946-47), Tommy Harper, 1969-1970, Andres Galarraga, 1992-93, Adam Dunn, 2010-2011, Adrian Beltre, 2003-2004, and Kirby Puckett, 1985-86.

Rogers Hornsby

George Burns

Darin Erstad

Norm Cash

Max Carey

Jeff Heath

Dave Roberts

Harry Walker

Tommy Harper

Andres Galarraga

Adam Dunn

Adrian Beltre

Kirby Puckett

                I don’t know if you guys remember this story.    Bob Costas was broadcasting a Minnesota Twins game early in the season in 1986; I think it may have been April 26, 1986.   Kirby Puckett, who had hit no home runs at all in a very fine rookie season in 1984 and had hit only 4 home runs in 1985, had already hit 5 home runs in the first 17 games of 1986, and they were talking about this power surge on the air.   Costas’ wife was pregnant, due to deliver almost any day now, and Costas said, "If he hits another home run here I’ll name my son after him."   And sure enough, about two seconds later—gone.  He named his son Keith Michael Kirby Costas.  Nice kid; he’s a grown man now.   I’ve gone to ballgames with him a couple of times.  (The story as it appears in Wikipedia is a little bit different, but in this particular case I trust my memory more than Wikipedia.)

                Anyway, there is an "Improbability Score" which evaluates the improbability of any two seasons being produced by the same player.   The Improbability Score of George Scott’s 1967 and 1968 seasons being produced by the same player is 386, the highest since 1900 for any player in consecutive seasons.  The Improbability Score for Kirby Puckett, 15^th on the list of improbable but actual combinations, is 278.  There are 64 cases in history of players having consecutive seasons with Improbability Scores of 200 or greater. 

                But if you randomly scramble seasons and compare one to another, then there are thousands of more improbable combinations than George Scott, 1967-68, and the average Improbability Score (of randomly matched seasons) is almost 200.  This is the scale that I established:

                You get different results if you compare consecutive seasons by the same player than if you compare non-consecutive seasons by the same player.    In the case of the exceptionally consistent Henry Aaron, for example, the two most poorly-matched consecutive seasons of his career are the 1959 and 1960 seasons:

                Although his power numbers were the same, Aaron’s batting average fell by 63 points in 1960, which is an unusually large change.   His doubles fell from 46 to 20, while his strikeouts and walks both increased.    All of his "speed" numbers were way up; his stolen bases doubled, his triples increased, and (not shown) his Grounded Into Double Play count dropped from 19 to 8.   One of the things we do to try to figure out if it’s the same player or not is to compare the Speed Scores.    If a player is fast one year and slow the next, that’s an indication that it’s not the same player—but Aaron appears to be quite significantly faster in 1960 than in 1959.  

                That’s an "improbability score" of 102, which puts him in the indeterminate range.   But if you compare Aaron’s prime seasons with his end-of-line, playing-out-the-string numbers in Milwaukee in the mid-1970s, then you would get scores much higher than 102:

                That’s an Improbability Score of 282—but, of course, Aaron was not really the same player in 1975 that he was in 1963.   He was about the same player until 1971; after 1971 he was clearly losing the ability which defined the rest of his career.  (Aaron’s 1976 season is even worse than 1975, but since he batted only 271 times in 1976, the Improbability Score is lower.) 

                When we compare players in consecutive seasons—what will be marked as "consecutive" in the chart below—the average Improbability Score is 28, with a standard deviation of 33.

                When we compare players in non-consecutive seasons, like Aaron in 1963 and 1975—what will be marked "non-consecutive" in the chart below—the average Improbability Score is 44, with a standard deviation of 52.

                When we compare randomly matched seasons—what will be marked "random" in the chart below—the average Improbability Score is 191, with a standard deviation of 182.  

                All of these numbers (and those below) were calculated using only seasons beginning in 1900, with 400 or more plate appearances.  When we say "randomly matched seasons", we mean seasons of different players, although of course one time in a thousand the random numbers will match up two seasons of the same player, but 99.9% of the time those are seasons of different players:

                A couple of interesting examples of contrasting seasons.    Take these two seasons.   Do you think they’re by the same player, or a different player?

                That’s actually Don Padgett, 1939, and Don Padgett, 1940.   If you look at it, everything is pretty much the same in the two seasons—except that he had an in-play batting average of .406 one year and .229 the next.     We give it an Improbability Score of 259, because, even though everything is functionally identical except the batting average, and even though the at bats are limited, it is just very, very strange that a player would hit .399 one year and .242 the next.    Here’s another one like that:

                That’s actually Todd Haney, 1995 and Todd Haney, 1996.    That’s an Improbability Score of 155, which is pretty amazing given the limited number of at bats and the fact that the strikeouts and walks don’t really change.  Can’t you just imagine Todd Haney trying to explain to his relatives at the Thanksgiving Dinner, "No, really.   I hit the ball exactly the same this year as I did last year.   I just couldn’t find a hole."

                Getting back to the rookies. . ..if we create projections, for rookies, which reasonably could be records of the same hitter as the actual record, that’s really as much as we can do.  If we project him for 550 at bats and he gets 20, the projection will be "wrong", but we don’t have any way of knowing whether he will bat 550 times, or 20, so we can’t worry too much about that.  If it could be the same player, I’m happy. 

How Improbability Scores are Figgered

Outline

                We begin by figuring a "reliability score" for the comparison, based on the number of plate appearances in each season.

                We compare the two seasons, A to B, on nine elements of dissimilarity, and "score" each discrepancy, then sum up the discrepancies.  We compare the seasons A to B and B to A, since sometimes it will score differently one direction than the other, and the lower sum is the Improbability Score.

                The nine elements of dissimilarity are:

                A)  Batting Average,

                B)  Extra Bases Hit Frequency,

                C)  Home Runs,

                D)  Strikeout and Walk Frequency,

                E)  Strikeout to Walk Ratio,

                F)  Stolen Base Attempt Frequency,

                G)  Hit By Pitch Frequency,

                H)  Sacrifice Hit Frequency, and

                9)  Speed Score.

                Hit by Pitch Frequency and Sacrifice Hit Frequency are not "important" elements of a hitter’s record, in the sense that they are things we would normally pay a lot of attention to in assessing a hitter’s value, but they can be "signature" elements which help us establish his identity.  A player does not ordinarily have zero sacrifice bunts one year and 15 the next, so if one season has 15 sac hits and the other has zero, that’s an indication that it may not be the same player.

Reliability Score

                The Reliability Score is figured as

                Where PA1 is Plate Appearances in one of the two seasons, and PA2 is Plate Appearances in the other of the two seasons.   This formula will normally result in a reliability score of less than one if the player has fewer than 500 plate appearances in one season or the other, but greater than one if the player has more than 500 plate appearances in each season.   Since Pete Rose had 770 Plate Appearances in 1974 and 764 in 1975, we have an unusually reliable data set to compare, and the Reliability Score for comparing those two seasons would be 1.526.

Batting Average

                The "Batting Average Discrepancy Score" is the most complicated of our nine measurements, except possibly home runs.

                1)  Subtract the lower batting average (of the two seasons) from the higher batting average, and subtract 35 points (.035).

                2)  Square that.

                3)  And multiply it by 25,000.

                So that, for example, if the difference in batting average between the two seasons is 40 points, that’s a Discrepancy Score of 0.625; 50 points in batting average, 5.625 points of Discrepancy; 60 points batting average, 15.625 points Discrepancy; 80 points batting average, 50.625 points Discrepancy, and 100 points batting average, 105.625 points Discrepancy. 

                4)  However, if the player has batted less than 25 times in either of the two seasons, the Batting Average Discrepancy Score is zero.

                5)  If the difference in batting average between the two seasons is less than 35 points, the Batting Average Discrepancy Score is zero.

                6)  If there are less than 300 at bats in either season, then the Batting Average Discrepancy Score is discounted as follows:

                Count 1 point if one player had 100 at bats in a season,

                Count 2 points if he had 300 at bats in a season,

                Add the two together, add 2, and divide by six.

                So that, for example, if one player had 80 at bats in a season but the other had 350, then we multiply the Batting Average Discrepancy Score by .667  (0 + 2 + 2) / 6 = .667.   If one player had 150 at bats in a season and the other had 500, we multiply the Batting Average Discrepancy Score by .833 (1 + 2 + 2) /  6 = .833.

                7)  One more control on the Batting Average Discrepancy Points.   Take the differences between the two batting averages, multiply by 1000 (so that a 120-point difference in batting average becomes 120), and add 50 (so that a 120-point difference in batting average becomes 170.)    If the Batting Average Discrepancy Points as figured in parts 1-6 of this rule are greater than that number, then substitute this second number.   (This almost never happens.   It’s just a weird-situation control.) 

                8)  Multiply this total (from parts 1-7) times 2.

Extra Base Hit Frequency

                Designate one of the seasons as Season A and the other as Season B.   Take the player’s Extra Base Hits in Season A, and project them into his Plate Appearances in Season B.    Subtract his actual Extra Base Hits in Season B.

                If the discrepancy is 13 extra base hits or less, no points.   If the discrepancy is greater than 13 extra base hits, count six (6) Extra Base Hit Discrepancy points for each extra base hit.

                Babe Ruth in 1921 had 693 Plate Appearances, and had 119 Extra Base Hits.   Glenn Beckert in 1966 also had 693 Plate Appearances, but had only 31 Extra Base Hits.  That’s a discrepancy of 88 Extra Base Hits, of which we will ignore the first 13.   The other 75, we charge at 6 points apiece—an Extra Base Hit Discrepancy Score, comparing those two seasons, of 450 points.  

                On the other hand, Don Kessinger in 1975 also batted 693 times, and had 36 Extra Base Hits.   So comparing Beckert (1966) and Kessinger (1975), the Extra Base Hit Discrepancy Score would be zero, since the difference is less than 13.

                We compare A to B and B to A.  Sometimes a player may have 4 Plate Appearances and 2 Extra Base Hits; when you compare him to a player with 700 plate Appearances, he should have 350 Extra Base Hits.   But since, in the end, we only use the smaller of the two comparisons, this "projection error" has no effect on the Improbability Score at the end of the process.

Home Runs

                Project the Home Runs from Season A into Season B.   We will call these figures X1 (actual home runs in one season), and figure X2 (projected home runs in the other season.)

                Subtract X1 from X2, and divide by 30.   This we will call X3.

                Add 7 to X1, add 7 to X2, and divide the larger number by the smaller.    Subtract 1.   This figure we will call X4. 

                Multiply X3 by X4.    This figure we will call X5.

                If X5 is less than .100, just ignore it; there is no penalty for a home run discrepancy if X5 is less than .100.  

                If X5 is greater than .100, then multiply it by 100.   This figure we will call X6.

                Take the player’s Plate Appearances in Season A, plus 1, and his Plate Appearances in Season B, plus 1.  (The +1 is just to avoid the possibility of dividing by zero.)     Divide the smaller figure by the larger.    This figure we will call X7.

                Multiply X6 by X7. 

                The result is the Home Run Discrepancy Penalty.

                Let us suppose we are comparing two players.   Player A hit 21 homers in 271 plate appearances, and player B hit 3 homers in 164 Plate Appearances.   

                Figure X1 is "3", the actual number of home runs hit by Player B.

                Figure X2 is "13", the projected number of home runs to be hit by Player A in 164 Plate Appearances.  Actually, it is 12.739; 13 rounded off.

                Subtract X2 from X1; the difference is 9.739.  Divide by 30; that’s .325.   Figure X3 is .325.

                Add 7 to each one, and divide the larger by the smaller; that is 19.739 divided by 10, which is 1.9739.    Subtract 1; that’s .974.    Figure X4 is .974.

                We use these two figures, Figure X3 and Figure X4, because we’re looking at the Home Run difference both as a relative difference, and as an absolute difference.   13 home runs compared to 3 is an absolute difference of 10 home runs—which is fairly small—but a relative difference of 4 to 1, which is huge.  And we add the "7" to both figures before we figure the relative differences because, if you don’t do something like that, then zero home runs compared to 1 home run is the same as zero compared to 65 homers.

                Multiply Figure X3 by Figure X4; that’s .325 by .974, which is .316.    That’s figure X5.

                Multiply Figure X5 (because it is larger than .100) by 100; it becomes 31.6.  That’s figure X6.

                Take the player’s Plate Appearances in Season A (271), plus 1 (272), and his Plate Appearances in Season B (164), plus 1 (165).    Divide the smaller figure by the larger (165 / 272 is .6066.)   This figure we will call X7.

                Multiply X6 times X7; 31.6 times .6066 is 19.2. . .so the Home Run Discrepancy Penalty between these two seasons is 19.2 points.

                These two players are Art Shamsky, 1966, and Art Shamsky, 1967.   It’s the same player; he just hit a bunch of home runs one season, and then didn’t hit them the next.   Home runs are tremendously important in assessing a player’s value, but less important in looking at his identity, particularly for a part-time player.  Some seasons players just don’t hit the home runs that they expect to hit, particularly if they don’t get a lot of at bats.

Strikeout and Walk Frequency

                Add the player’s strikeouts and walks together.  Project the strikeout/walk total in Season A into Season B, and subtract this from the actual strikeout/walk total in Season B.

                If the difference is 15 or less, just ignore it; no penalty.   If the difference is greater than 15, then one point for each additional strikeout or walk.

For example, Adam Dunn in 2012 had 105 walks, 222 strikeouts in 649 plate appearances. Irish Meusel in 1923 also had 649 plate appearances, but he had only 16 walks and 38 strikeouts, a total of 54 strikeouts and walks.   Dunn’s total was 327.   That’s a difference of 273.    We ignore the first 15 (making 258), and count one Strikeout/Walk Discrepancy point for each of the others, which makes a 258-point discrepancy.

Strikeout to Walk Ratio

Figure the player’s walks as a percentage of his strikeouts and walks.

Take this percentage for one player, and project it onto the strikeout and walk total of the other player.

Subtract this from his actual walks.

If the difference is ten or less, no penalty.  If the difference is greater than 10, 2 points for each walk (greater than 10).

Adam Dunn and Irish Meusel again. . .. their strikeout to walk ratios are actually about the same.  Meusel had 16 walks out of 54 strikeout/walk events, or 29.6%; project that into Dunn’s strikeouts and walks, it’s 97 walks.   That’s within 10 of his actual number, so. . .no penalty points.

Dunn had 105 walks among 327 strikeout/walk events, or 32.1%.   Project that into Meusel’s strikeouts and walks, it’s 17 walks.   Meusel actually had 16, so. . .no points.

If you compare Ozzie Smith, 1985, and Alfredo Griffin, 1985, they’re very similar hitters apart from their strikeout and walk data.   

There actually are no Improbability Points in this matchup, apart from the strikeouts and walks and a small difference in the sacrifice hit category. 

But the strikeouts and walks suggest that it is not the same hitter.   First, strikeout and walk frequency.    Afredo Griffin had 70 strikeouts and walks in 646 Plate Appearances.   Ozzie Smith had 615 Plate Appearances.   Project Griffin’s total into Ozzie’s Plate Appearances; that’s an expectation of 67 strikeouts and walks.    Ozzie actually had 92 strikeouts and walks, so that’s a discrepancy of 25 strikeout/walk events.   We ignore the first 15 of those, and count one point per event above 15, and there’s a 10-point penalty for the strikeout and walk frequency.

But Griffin, given 92 strikeout and walk events, would probably have had 26 walks, 66 strikeouts.    That’s a discrepancy of 39 walks.   We ignore the first 10 of those and charge two points each above 10; that’s a 58-point penalty.    And that throws the comparison of these two players—otherwise extremely similar—into the "inconclusive" range.    The same player does not ordinarily have a 50-20 strikeout to walk ratio one season, and 27-65 another season, although such things have happened on occasion.

Stolen Base Attempt Frequency

Take the Stolen Base Attempts of Season A, and project them into Season B, based on the number of times the player was on base.  Compare this to the actual Stolen Base Attempts in Season B.

The number of times the player is on base is defined for this purpose as hits, plus walks, plus hit by the pitch, minus extra base hits.

If the discrepancy is 8 or less, just ignore it; no penalty.  If the discrepancy is greater than 8, 2 points for each additional stolen base attempt.

Rickey Henderson, 1986, and Rico Petrocelli, 1971, had quite similar seasons—apart from the fact that Henderson had 105 stolen base attempts, and Rico had only 2:

Rickey Henderson was on base 187 times, not counting the extra base hits; Rico was on base 176 times.   If Rickey had been on base only 176 times, he would probably have attempted only 99 stolen bases.   This is 97 more than Petrocelli attempted.  We ignore the first 8 of these, and score them at 2 points apiece after 8.  That’s a 178-point penalty.   That marks them as almost certainly different players, since a player does not ordinarily attempt 99 stolen bases in one season, and 2 in another. 

Hit By Pitch Frequency

Take the HBP total from Season A, and project it into the Plate Appearances from Season B.   Subtract this from the actual Hit By Pitch Total of Season B.

If the difference is 5 or less, just ignore it; no penalty.   If the difference is greater than 5, 2 points for each additional HBP. 

Sacrifice Hit Frequency

Take the Sacrifice Hit (SH) total from Season A, and project it into the Plate Appearances from Season B.   Subtract this from the actual Sacrifice Hit Total of Season B.

If the difference is 4 or less, just ignore it; no penalty.   If the difference is greater than 4, 3 points for each additional HBP. 

Speed Score

Make a "Simple Speed Score" for each hitter in this way:

3B times 10,

Plus SB times 5,

Minus CS times 3,

Minus GIDP times 4,

Plus .200 times plate appearances,

Times 20,

Divided by (Plate Appearances + 10).

If the result is greater than 10, level it off to 10; if it is less than zero, move it back to zero.

Dale Mitchell in 1949 stole only 10 bases in 13 attempts, 677 plate appearances; however, he hit 23 triples and grounded into only 8 double plays, which is half of a normal number for 677 plate appearances.  Based on this, he has a speed score of 10:

That works out to 8128 divided by 687, which is greater than 10, so his speed score for that season is 10.   Rickey Henderson’s Simple Speed Score is 10 almost every year. 

Subtract the Speed Score of Player A from the Speed Score of  Player B, minus 1; make it an absolute number  (6.29, rather than -6.29).   Multiply that by 10, and multiply that by the reliability score of the two seasons.  The result is the "Speed Discrepancy" between the two seasons.