Out Ratio vs. Catch Probability

March 28, 2017
 Earlier this month at both the Sloan Analytics Conference in Boston and the SABR Analytics Conference in Phoenix, MLBAM’s StatCast team released details on a new statistic that they have started to report on flyballs to the outfield: Catch Probability. Catch Probability is a measure of how hard a catch is to make based on how far the batted ball is from the nearest outfielder and how long it has been in the air. Here is a full explanation of how it works from MLB.com. Because of our long-time passion for defensive metrics, we at Baseball Info Solutions (BIS) are particularly interested in their new statistic, especially in how it compares to Out Ratio, which is the basis of the Range and Positioning metric that serves as the foundational component of Defensive Runs Saved.

Catch Probability and Out Ratio are similar. Both measure how hard it is to catch a specific type of ball. Both rely on hang time to factor in difficulty. However, by measuring how far the fielder has to run, Catch Probability is measuring range by itself. Out Ratio measures range and positioning taken together in that it does not factor in the starting position of the outfielder.

The two metrics have differences, but they both get at the same question of how difficult a flyball was to catch. And both methods group plays with other plays having similar measurements from recent seasons to answer that question.

The MLB.com article shows a couple of examples that we can use to compare the two systems. The first is an awesome diving catch from Billy Hamilton from the Reds’ April 26 game against the Mets last season in the bottom of the fifth inning:



Catch Probability gives that flyball only a 7 percent chance of being caught. Hamilton had a long way to run on this play. Out Ratio gives center fielders a 35 percent chance of making the play. Catch Probability is just trying to measure range and ignores positioning.

Next up is Matt Kemp, who made his own diving catch last year on April 5 versus the Dodgers in the top of the fourth inning.



Similarly, Out Ratio is higher than Catch Probability. StatCast gives the flyball a 75 percent chance of being caught. Meanwhile, Out Ratio pegs the flyball at 100 percent; all 33 flyballs hit to this location with the same hang time from the last two years have been caught. Kemp’s positioning made it a tougher play.

As you can see from these two examples, bad positioning makes plays more difficult than they would be for outfielders if they simply had to rely on their range. This is as we would expect. Positioning has always been important in baseball, even more so in recent years. But while these two examples show instances of outfielders being positioned a bit worse than the average Out Ratio suggests, we would actually expect fielders to be positioned better than Out Ratio’s average more often than not. And so Catch Probability from StatCast is generally going to be higher than Out Ratio from BIS because of that advantage of positioning.

We look forward to more excellent work like this from the StatCast group. The kind of in-depth data the StatCast system collects will allow for measurement of defensive skills in finer and finer detail. Effectively splitting range and positioning into separate metrics is just the start. We are getting close to a day when we can tell you not only how much range a fielder has, but how quickly he reads the ball off the bat, how quick his first step is, how fast he is on approach, how well he takes routes to balls, and how sure-handed he is; and we will be able to hang a Runs Saved number on each part. The summation of all of those individual parts may not be too different from the numbers you’ve seen for years, but the level of detail will be incredible. For baseball analytics fans, the more the merrier.


COMMENTS (10 Comments, most recent shown first)

I don't see a reason why positioning is more important today than at any other time in baseball history. Curious about an explanation. (One thought is that with a higher percentage of runs being scored on homeruns, positioning might be less important today. Offsetting that, perhaps, is the idea of bigger ballparks.)
4:33 PM May 20th
Statcast has a system of radar and cameras that track the position of the ball and players throughout a play. So, we know the x,y position of every fielder when a ball is released. We know the x,y position of the ball where it lands, as well as how long it took to get there. The "eye" portion is for us to record if the player got an assist or putout or it was a hit or error.
11:28 AM Apr 3rd
But shthar, the whole point of StatCast is that it's based on measurements. I give BIS a lot of credit for trying to reduce the influence of individual judgment, but the video scouts were never intended to be robots. Whereas StatCast, if I understand it correctly--Tom will surely correct me if I'm wrong--doesn't rely on opinion at all, except perhaps somebody's opinion that a particular thing is worth measuring in the first place.
7:13 AM Apr 1st
another defensive stat based on the opinion of a guy watching a game.

I can do that already.​
4:02 AM Apr 1st
Sure, Tom, but it's more than that. You get to decide what data to collect. BIS depend on Fox Sports Podunk for their video. Onward and upward.
9:35 AM Mar 30th
steve: the way I see it, the more data, the more opportunities. We're closing one door, and opening three.
9:00 AM Mar 29th
OBS asks the right question. Presumably positioning is based on probabilities derived from spray charts, thus the tendencies of individual hitters. To say that Hamilton or Kemp were positioned 'poorly' is an improperly value-laden way of saying that the ball was less likely to be hit where it actually was. Low-percentage events happen regularly in baseball.

BIS have significantly advanced our knowledge of defense in the last dozen years or so by detailed analysis of game video. StatCast will advance it even farther with data gathered directly in the ballpark. John's video scouts won't become obsolete overnight, but they should start updating their resum├ęs.
7:03 AM Mar 29th
Thanks John, good stuff.

5:55 AM Mar 29th
We're eventually going to show it both ways, so that we'll say: Kemp was +.25 for his range, and -.25 for his positioning. The tally will be in separate boxes, just like walks and singles and HR are in separate boxes. If you want to merge those into OBP, you can. If you don't, you won't. Same for fielding.
5:55 AM Mar 29th
Should positioning be hung on the bench more than the fielder?
6:40 PM Mar 28th
©2021 Be Jolly, Inc. All Rights Reserved.|Powered by Sports Info Solutions|Terms & Conditions|Privacy Policy