This is a continuation of a debate which began here on December 12, 2012, just after Wil Myers was traded to Tampa Bay, in the article "On the Differences between Pitching Prospects and Hitting Prospects".   There is a rather long preamble here which re-states at length arguments that many of you probably read through in the original, but. …it’s an important issue, and if somebody is reading this article in ten or fifteen years I want him to have a fair chance to understand everybody’s positions.  My local sports columnist had written the following:

In time, Wil Myers might develop into one of the top power hitters in the game. At 21, he hit .314 with 37 home runs and 109 RBIs in 134 games of a season split between Double A (35) and Triple A (99). His pitch recognition might develop to the point he can strike out at a less disturbing rate than 140 times in 522 at bats. He’s an excellent prospect, all right.

The word "might" and "prospect" need not enter discussions about James Shields, the main player acquired by the Royals in the deal with the Tampa Bay Rays.

To which I responded:

                Oh, I can give you a long list of mights that enter into the James Shields evalaution, but let’s fast forward. In time, Wil Myers might be something; he isn’t anything yet, but later on, down the road, he might become something. Not trying to parody the sportswriters words or to state them unfairly; I think that’s an accurate summation of his point; Wil Myers isn’t anything yet, but later on he might be something.

          Most of us guys, I suspect, see the situation a little differently: that Wil Myers is a very good baseball player, right now. He was a very good baseball player in 2012; there is every reason to believe that he will be the same player in 2013, although his statistics will not be the same because the players he will be playing against are better. Later on, he may develop to an even higher level, true, but he is the same thing now that he will be in a year, and therefore the distinction between "prospect" and "player" is, on some level, a silly distinction. It relies on doubt that exists only because of ignorance, and thus exists only for the ignorant.

          We cannot make absolutely accurate projections as to what any player will hit next season, whether he is a rookie or whether he has been in the league for ten years. But we can project what Wil Myers will hit in 2012 as accurately as we could project the same if he had been in the league for ten years, and this is a fairly high level of accuracy. The sportswriter thinks of Wil Myers as he does because he fails to understand this. He believes that there is an element of doubt in the equation that is not really there, or does not need to be there. Thus, he is basing his analysis of the trade on a categorization of the players, and basing the categorization of the players on his own ignorance, his own lack of sophistication. It’s an analysis that is based, at the deepest level, on the ignorance of the writer.

                The original article (On the Differences Between Pitching Prospects and Hitting Prospects) contains more comments in a similar vein, but I am just trying to reconstruct the essence of the debate, in order to make the points that I wanted to make next.    Responding to this, a poster using the name "myachimantis" wrote the following:

I'd like to just mention that it may be easier to project major leaguers simply because there is more information. Perhaps teams collect batted ball data for their minor league systems, but it isn't publicly available. With batted ball data, one can take a look at a hitter and say whether they had a season that was largely the result of a BABIP that was too high given their batted ball profile or that their season was a true measure of their talent and their BABIP accurately reflected their batted ball data. Same goes for pitchers, especially when looking at HR/FB%. We have more information about major league players, making it easier to subtract out the luck from their performance, and make projections going forward.

To this I responded that "You can give many reasons why it SHOULD be easier to project major leaguers than minor leaguers—but the fact is that it isn’t. Minor league hitters can be projected as major league hitters as accurately as major league hitters can be projected as major league hitters. This is probably the most valuable insight of sabermetrics—and has yet to be fully digested or accepted by the sabermetric community, thus remains as an advantage to be exploited by major league teams who do understand this."

                MWeddell posted in response to this:

Seems like we are talking past each other in this article's comments. No one is saying that one can't project future performance based on minor league batting statistics. The debate is whether, if everything else is the same, are past minor league batting statistics just as useful at projecting future performance as past MLB statistics are?

Since reading www.insidethebook.com/ee/index.php/site/article/minor_to_major_correlations/ a few years ago, I have considered that minor league batting statistics are less useful than MLB batting statistics are at predicting future performance. If there is evidence to the contrary, it'd be great to know.

At the time Mr. Weddell posted this I let it go without checking the article that he had referenced.   But having now checked out that article, I am puzzled by what his point is.   Mr. Weddell seems to believe that the article he references argues against the reliability of minor league stats as major league predictors—but in fact, as I would read the article, it actually argues IN FAVOR of the reliability of minor league batting statistics as much as it argues against them.   The article he references is straightforward; it doesn’t really pitch an argument one way or the other, it merely presents facts.   The data that it presents seem to me to be consistent with my point of view, but perhaps I’m just not reading the data right, I don’t know.  

Anyway, someone named Bruce (or claiming to be named Bruce) next posted the following—this still on the day the original article was posted. 

Why is it so "absurd" to distinguish between players who have proven the ability to be successful at the major league level and those who haven’t?

Here again, something I didn’t see at the time. . ..Bruce "quotes" the word "absurd", although actually that word had not been used in the discussion up to that point.   I had said it was a silly distinction, not that it was absurd.     Anyway, to this I responded "Proven, to who?   Wil Myers has proven to my satisfaction that he has the ability to be successful at the major league level.  I am absolutely, 100% satisfied that he does.   When you say that he has not proven this ability, then, you are talking about a distinction that exists only in your mind. . .or only in my mind; that doesn’t matter, in whose mind the distinction rests.   It is absurd to divide players based on a distinction that exists only in your mind."

This, by the way, is still what I think; certain things that I said before were poorly stated and/or were stated in such a way that I couldn’t defend them in a more thoughtful debate, but so far we haven’t hit those things.   So far, what I was saying then is still what I believe today.      

Someone using the name "jemanji" then posted

Am intrigued by the implication that Wil Myers is (roughly) as good a bet to be a difference-maker in the majors as are (say) James Shields or Alex Gordon or other players who ARE difference-makers in the majors.

In saying that these people ARE difference-makers in the majors, what he actually means is that they HAVE BEEN difference-makers in the past.    He thus ASSUMES that the present is more closely connected to the past than to the future, which is not true, and which obviously constitutes a bias the debate we are having, but let’s move along.  Jemanji continues:

I’m a big believer in MLE projections myself.

Small point, which will become important later on.  MLEs are not projections.    I may well have spoken of them as projections myself, in the past; let’s not get hung up on that.  MLEs are like exchange rates, how many Yen equals a dollar.    They’re not predictions.   

I’m a big believer in MLE projections myself, and a bigger believer in 21-year-olds who rake at AAA. . . But the plot thickens with this second axiom: a few months ago you acknowledged that a prospect who had hit well in the majors for a short time had passed an important test. There are lots of AAA hotshots who turn out to have weaknesses that are exposed by ML precision (Jose Lopez was a cleanup hitter in AAA at age 20 who totaled about 6 WAR for his career). .... My question: what SPECIFICALLY about Myers are you looking at, as opposed to any other 21-year-old who had a 900 OPS+ in AAA? Or are these kids IN GENERAL as valuable as ML impact players?

                There are three responses to this, which I gave in various postings over a period of days:

1)  That would depend on the generalization. If by "these kids", you are generalizing 21-year-olds to include 22-year-olds, then it is less true. If by "these kids" you include defensive liabilities with young center fielders, then it is less true. If by "these kids" you're including players with an .880 OPS along with players with a .930 OPS, then it is less true. It depends on how you generalize.

                2)  The "test" that players have to pass is earning the confidence of the major league community.   A young actor might well have the acting skills of Alec Guinness.   If Hollywood doesn’t believe in him, he’s working local theater.  

                This "perception of skills" issue is not as large in baseball as it is in acting, or as it is in many "skill" professions.    In acting, I would bet that 90% or 95% of those who have the ability to be stars never get the chance to shine.  In baseball, most players who have ability will get a fair shot.   But it’s not a zero issue in baseball, either.  

                3)  The Jose Lopez example is actually extremely instructive about the profound misunderstandings in this area. If you look at Lopez, it could not possibly be any more apparent that the change in his level of hitting ability occurred at the MAJOR LEAGUE level, after he had been in the majors for five years and had just short of 3,000 plate appearances as a major league player. In 2008--four years after the minor league season that you reference--Lopez hit .297 for the Mariners, with 41 doubles and 17 homers giving him a.764 OPS. The season AFTER that he hit .272 with 42 doubles and 25 homers, giving him a.766 OPS. At the end of that season he had 2,977 major league plate appearances. THEN he stopped hitting. ....3,000 plate appearances into his major league career, he stopped hitting.

The uncertainty of projection in Lopez’ case occurred in the middle of his major league career--yet in discussing him, you attribute this. . .you miss-attribute this. . .to his minor league/major league transition, and thus miss-attribute the uncertainty to his minor league performance. By doing this, you both overstate the uncertainty of projection based on the minor league performance, and understate the uncertainty of projection based on the MAJOR LEAGUE performance. This sustains you in your mistaken belief that minor league hitting stats are not reliable indicators of performance.

Why do you do this? Are you a fool, or are you determined to deceive us?

Well, of course you are not; you are merely doing what all of us do all the time. You have an organized way of thinking about this problem, and so your mind re-arranges the facts to be consistent with that way of thinking about the problem--even though, in truth, those facts are not AT ALL consistent with that way of thinking about the problem. We all dislike re-thinking our assumptions. This self-deception protects your mind from having to re-think your assumptions.

                At this point Tom Tango entered the debate.    Tango’s first post related to this was:

Yu Darvish and Felix are the same age, but our uncertainty level was higher for Yu entering 2012. And it's still higher today. Wil Myers is around the same age as Starlin Castro, Mike Trout, and Brett Lawrie. In terms of our uncertainty level as to how these four players will play over the next say five years, we have more uncertainty with Myers, simply because he hasn't faced the quality of competition that the other three have. Whether that uncertainty level is "a lot" or "a little" more than for the other three, that's really the question. I think the "proven" discussion makes it seem like it's black/white, when really, we're talking about shades of uncertainty levels.

                My first response to this post was, I think, OK.  "I agree with you about Darvish vs. Felix," I said.  "That was part of my thesis, that this distinction DOES exist with pitchers."   I think that was the right thing to say, to resist the mixing up of pitchers with hitters, since the foundation rock of the previous debate was that young pitchers are NOT like young hitters.   With the second half of my response to Tom, I think I went awry:

The only uncertainty for Myers is playing time. If he plays, there is no more uncertainty in projecting his performance than Trout's or Lawrie's or any other player's. Where, then, is the "uncertainty" located, if not in the eye of the beholder?

                In this post—taking Tom’s interpretation of the issue as my own—I blundered past a critical distinction.    Tom is certainly correct in saying that there is a "transitional uncertainty" when a player comes to the major leagues.   There is a transitional uncertainty when a player comes from Japan to the United States; there is a transitional uncertainly when a player changes teams; there is a transitional uncertainty when a minor league player comes to the major leagues.   It may well be that this transitional uncertainty is greater for the minors-to-majors transition than for these other transitions.    I do not agree that Myers’ major league production is less certain or more speculative than that of Starling Castro or Brett Lawrie, but in stating my argument in such a way that I denied the impact of the transitional uncertainty, I was absolutely incorrect.

                Also, at some point in this discussion, Izzy 2112 made the point that the track record for a minor league player is rarely as long as the track record for a major league player, which creates a measure of uncertainty for the minor league player—also a valid point.   Wil Myers has played only 99 games of Triple-A baseball, and only 233 games above A ball.  99 games for a player at any level may be atypical of his true skill level.    In 2011 Jhonny Peralta hit for a higher batting average than Derek Jeter, each with more than 500 plate appearances.  99 games is not enough to get a true read on a player’s skills.      

                But this is does not fundamentally undermine my point.    We know what Wil Myers is capable of, as a major league player.   We know this just as much, in the case of Wil Myers, as we would if Myers had played those 99 games in the majors, rather than in the minors.

                It’s like this:   Suppose that we think of minor league players as sea peoples, or even as sea creatures, and major league players as land animals.   The sea creatures are trying to make land, and there is a transitional difficulty inherent in that.   Some people drown when they are trying to reach the shore, due to the undertow.   

                The undertow, at the shore, results not from any force directly pushing the swimmer out to sea, but rather, from the force of the tide pushing toward the shore, but then bouncing off and heading back out to sea.   The energy is pushing in, but sometimes the recoil drags people back out.   The same thing with rookies.  Their focus is on reaching the majors, but sometimes the energy, the intensity of that experience works against them, and pulls them back out to sea.   In trying to make the sea-to-shore transition, the majors-to-minors transition, some players will drown.   If I ignored this fact in my earlier comments, I was wrong to do that. 

                Even when they get to shore, the sea peoples still face a transitional difficulty.     They have to learn to walk, rather than swim—or, more modestly, they have to learn to walk on land, rather than walking around on the boats.  They have to learn to watch out for lions and hippopotamus, rather than sharks and rays.    They have to learn to eat the apples, rather than the guppies and koi.  Of course this transitional stage will present difficulties for them, and of course the player’s statistics will be different while he is in the transitional stage than when he is either in the sea or when he is established on land.    It was stupid of me to trap myself into denying this. 

                But that doesn’t fundamentally change what I am trying to say.      My local sports columnist speaks of major league players and "prospects" as if they were fundamentally different things, like hammerheads and Dobermans.   This view of minor league players is common not only in sportswriters, but among baseball professionals—and it is totally baseless.  They’re not fundamentally different things.   They’re exactly the same thing.  

                Can you take a team of major league players and a team of minor league players, mix and match them, put some of them on one team and some on the other, and play a game?   Of course you can—in fact, once you put the uniforms on them, you’d never know which was which.  

                Major league and minor league players do in fact play against one another all the time.   They play against one another, mixed up into different teams, in spring training.   They play against one another in winter ball.   In regular season players goes back and forth from the majors to the minors constantly.   There are 200 players every year who split their season between major league time and minor league time.   They’re not fundamentally different things.

                And they do not have fundamentally different statistical profiles.   Minor league statistics, probably adjusted for context, provide every bit as accurate a package of information about a player’s skills as do major league statistics.   We are not at the end of the debate, but that is still my belief.

                Continuing now with the exchange.  Tango:

"This is the most important thing you could learn from me if you would stop refusing to learn it. " I would like to see more evidence in that case. My position is simply that every difference in context adds a layer of uncertainty. If a hitter goes from Coors to Oakland to  St.Louis (Holliday), or if he moves from Japan to Yankees, or if he moves from AA to MLB, all those changes in context are severe enough that it has to add a level of uncertainty. And the more severe the change in context, then the more uncertainty we have (all other things equal).

Me:

I would certainly agree that there are uncertainties associated with all transitions, at the major league level or majors to minors. A player who is in his first year on a new team--like Carl Crawford coming into Boston in 2011--is demonstrably more likely to have a catastrophic season than is a player who is playing in the same place he was playing the year before.

I do NOT agree that these uncertainties are larger in going majors to minors than in going from one major league setting to another, and I would ask to see the evidence that they are larger.

In my view, I have been providing evidence for my position constantly for 30 years, and the world and the sabermetric community have been explaining it away and refusing to learn for 30 years, because it requires that people re-think their established assumptions. When Juan Gonzalez came to the majors in 1991 or 1992, we published projections for him that proved to be absolutely accurate. When Jason Heyward came to the majors in 2010, we presented projections for him that proved to be extremely accurate. . . .Jason Heyard, and Reid Brignac, and Trevor Crowe, and Ian Desmond. We publish very accurate projections for a dozen or more rookies every year in the Handbook. Why is this not evidence that it is possible to do this?

Let me try this another way. .. .if you are asserting a general theory of statistical uncertainty based on transitions, I doubt that I would disagree with you, and I would tend to accept the theory while awaiting proof. If, on the other hand, you are asserting a specific theory of statistical uncertainty applying uniquely to minor league hitting statistics, then what I would say is that over a period of many years we have presented much more than sufficient evidence to show that these projections can be made accurately.

                Tango replied:

I did a very long post in comparing forecasts (2007-2010) by several prominent forecasters (PECOTA, ZiPS, etc), and I had broken it down in several ways. One of the breakdowns was based on "past MLB experience". And the average error in the forecasts for veterans was lower than that of part-time players which was lower than "pure rookies" (no MLB experience). In fact, the amount of error for the pure-rookies forecast was HIGHER than simply giving every pure-rookie an identical league-average forecast. It's extremely long, but this is about as detailed a test of forecasting systems that I've ever done. http://www.insidethebook.com/ee/index.php/site/article/testing_the_2007_2010_forecasting_

                This was posted on December 14, two days after the original article.   I said "Thanks, I’ll go look at it," and this where the discussion has rested until now.   I have, however, been unable to find the article in question, so I will have to make assumptions about it.

                First of all, let me point out one issue.   In the years 2000 to 2011 first-year players in the majors had an average of 104 plate appearances.   Second-year players had an average of 200 plate appearances (200.2727, so you don’t think I just rounded that off.)  Third-year players had an average of 260 plate appearances, fourth-year players an average of 311, fifth-year players an average of 352, sixth-year players an average of 370.   The graph actually peaks in the tenth year; players in their tenth season in the major leagues had an average of 384 plate appearances.

                Of course we cannot project batting averages, slugging percentages or on base percentages accurately for players who are getting 104 plate appearances apiece, and of course the projections for those players would be more accurate if you just used league norms, rather than individual records, for those players.   That’s the James/Stein paradox, established by Charles Stein and some other William James—not me—in Operations Research in the 1970s.   (James and Stein studied batting averages of players based on the first two months of the 1970 season, and found that their batting averages for the rest of the season were predicted more accurately by the group norm than by individual performance.)     I assume that Tom did something to adjust for this difference in plate appearances between rookies and veterans, but I don’t know what was done, and I will point out that it would be extremely difficult, if not impossible, to adjust this difference entirely out of existence. 

                Setting that issue aside, of course we have a transitional issue when we focus only on the batting averages of rookies.   But that does not mean that minor league batting statistics are less reliable indicators of batting ability than major league statistics.   It merely means that there is a certain perturbation of the data that takes place during the transitional stage.   

                Let us take, for example, the case of Felix PA, formerly known as Felix Pie.   These are Felix Pie’s major league stats for 2007:

                And this is the projection that we had for Pie in the 2008 Handbook, based obviously mostly on his minor league performance.

                And this chart adds in his actual 2008 major league performance:

                OK, we were entirely wrong; his transitional difficulties persisted into the 2008 season—he had only 250 major league at bats at the end of that season—and he continued to have difficulty eating the apples and avoiding the lions.   He never actually did get by that stage; we’d have to say that the lions ate him, or the hippopotamus squashed him, or the snake bit him, or something; one of them land animals did him in.   Based on the 2007 or 2008 seasons, it would certainly be true that one would get a better projection for him by just taking the league norms.

                But was our projection for him really wrong?   Let’s look at what Pie did in 2009 and 2010, which were the only major league seasons to date in which Pie got 200 at bats:

                Our assessment of Pie’s major league abilities not only was not inaccurate; it was, in fact, extremely accurate.   Add together his 2009 and 2010 seasons, the only seasons in which he batted 200 times, and compare that to the numbers we had projected for him.   It’s uncanny how accurate we were—once Pie got more-or-less past the transitional stage.

                Pie’s minor league record fails to "predict" his rookie-season and second-season performance not because his minor league records are misleading, but because his rookie season and second-season records are misleading.   The minor league records, the MLEs, are right.   It’s his major league numbers that are screwy.   

                Here’s Erick Aybar’s 2007 projection, compared to what he actually would do in 2007:

                Not a good projection of the 2007 season.  But were we fundamentally right about Erick Aybar, or were we fundamentally wrong?  

                Here’s our projection for Alcides Escobar, from the 2010 Handbook, compared to what Escobar really did in the 2010 season:

                Not too good, huh?   Even though our playing time projections for him happened to be right, we missed his batting average by a whopping 53 points.   But which was the "accurate" statement of his abilities, and which was the "misleading" statement of his abilities?  You decide:

                The fact that his minor league records do not match his rookie-season production is not because his minor league records are misleading; it is because his rookie-season records are misleading.

                I remember when Javy Lopez came to the majors 20 years ago, our projections showed him as a .300 hitter.    He hit .245 as a rookie—but then settled in and hit around .300 for most of the next ten years.  

                Look, I hope you don’t think I am trying to mislead you about the accuracy of our projections for rookies.   Projections for rookies are problematic.  To me, that’s not the real issue.   When a young player comes up, what I believe that most people want to know is, what kind of a player is he?  Is he a .260 hitter, or a .290 hitter?   What does he do well, and how well does he do it?  That’s the question that I want to try to answer, to the best of our ability.  

                There are two issues here:  Projection (or prediction), and the clarity or accuracy with which we can perceive the present reality of a player’s abilities.  

                The importance of this issue, to a major league executive, is this:  that if you believe that the skills of a minor league player cannot be accurately assessed based on his minor league hitting record, then you are the captive of the major league population.   You cannot blend into your team players who have been stuck in the minor leagues—even very good players who have been stuck in the minor leagues—because well, you just never know what they will hit.  

                But if you realize that you can project what a player will hit in the majors—allowing for the markdown in the transitional stage—then you realize that you are surrounded by an ocean of talent.   You don’t have to sign a mediocre 30-year-old player to an $8 million a year contract.   You can get players just as good out of the minor league pool—the minor league ocean—for the major league minimum.

                OK, I oversold my case there.   The oceans are not teeming with major league talent.   80 or 85% of the players who have been trapped in AAA don’t break through in the majors because they can’t.    The transitional difficulties are a sort of "tax" on the usage of minor league players as major leaguers.   You can use these players in the majors, yes, but that 28-year-old second baseman you like will be 30 years old before he gets his feet on the ground in the major leagues, and he’s going to cost you for a year before he gets going, and unless he is Ben Zobrist he may drown in the process.

                But that doesn’t justify talking about "prospects" as if they were an entirely different species than major league players.    There is a continuum of talent that connects Ryan Braun to the JuCo prospect who won’t get drafted.   Pretending that there is something "special" about "proven" major league talent merely weakens your position, as a major league executive, and leaves you less able to solve your problems.    It is still my position that the distinction between a "player" and a "prospect" is a silly distinction, because it relies on the ignorance of the observer.