What Umpires Still Get Right When They Get It Wrong

We act as if we tell ourselves enough times that it doesn’t exist, then it doesn’t. That if we tell ourselves that something crazy and unfair didn’t happen for that reason but some other, then we are right and it’s all good. But the truth is, in infinite, infinite ways, there is a difference. There are tiny (and not so tiny) pinpricks.

The race of the pitcher, we found, also mattered, but not as much as other factors. Umpires were 10 percent less likely to expand the strike zone for African-American pitchers than for Caucasian pitchers, but race did not seem to influence whether an umpire called a pitch a ball when it was actually a strike.

THIS season Major League Baseball is allowing its officiating crews to use instant replay to review certain critical calls, including home runs, force plays and foul balls. But the calling of the strike zone — determining whether a pitch that is not swung at is a ball or a strike — will still be left completely to the discretion of the officials. This might seem an odd exception, since calling the strike zone may be the type of officiating decision most subject to human foible.

In research soon to be published in the journal Management Science, we studied umpires’ strike-zone calls using pitch-location data compiled by the high-speed cameras introduced by Major League Baseball several years ago in an effort to measure, monitor and reward umpires’ accuracy. After analyzing more than 700,000 pitches thrown during the 2008 and 2009 seasons, we found that umpires frequently made errors behind the plate — about 14 percent of non-swinging pitches were called erroneously.

Some of those errors occurred in fairly predictable ways. We found, for example, that umpires tended to favor the home team by expanding the strike zone, calling a strike when the pitch was actually a ball 13.3 percent of the time for home team pitchers versus 12.7 percent of the time for visitors.

Other errors were more surprising. Contrary to the expectation (or hope) that umpires would be more accurate in important situations, we found that they were, in fact, more likely to make mistakes when the game was on the line. For example, our analyses suggest that umpires were 13 percent more likely to miss an actual strike in the bottom of the ninth inning of a tie game than in the top of the first inning, on the first pitch.

We also found that the pitch count had an influence over the umpire’s perception of a pitch. When the count was 3-0, and another ball would end the at-bat, the umpires mistakenly called a strike 18.6 percent of the time, compared with a 14.7 percent error rate when the count was 0-0. But when the count was 0-2, with another strike yielding a strikeout, the umpires expanded the strike zone only 7.3 percent of the time, half the error rate for 0-0. The umpires, in other words, appeared biased against ending an at-bat.

The race of the pitcher, we found, also mattered, but not as much as other factors. Umpires were 10 percent less likely to expand the strike zone for African-American pitchers than for Caucasian pitchers, but race did not seem to influence whether an umpire called a pitch a ball when it was actually a strike.

In a way, it is surprising to find such nonrandom errors, because M.L.B. umpires are keenly aware that their ball-strike calls are being scrutinized and that they will be evaluated accordingly. They have an incentive to be as accurate as possible. And yet they still make routine errors behind the plate. We think that the sorts of errors we observed are not deliberate and may reflect an unconscious and biased decision-making process.

One of the sources of bias we identified was that umpires tended to favor All-Star pitchers. An umpire was about 16 percent more likely to erroneously call a pitch outside the zone a strike for a five-time All-Star than for a pitcher who had never appeared in an All-Star Game. An umpire was about 9 percent less likely to mistakenly call a real strike a ball for a five-time All-Star. The strike zone did actually seem to get bigger for All-Star pitchers and it tended to shrink for non-All-Stars.

An umpire’s bias toward All-Star pitchers was even stronger when the pitcher had a reputation for precise control, as measured by the career percentage of batters walked. We found that pitchers with a track record of not walking batters — like Greg Maddux — were much more likely to benefit from their All-Star status than similarly decorated but “wilder” pitchers like Randy Johnson.

Baseball insiders have long suspected what our research confirms: that umpires tend to make errors in ways that favor players who have established themselves at the top of the game’s status hierarchy. But our findings are also suggestive of the way that people in any sort of evaluative role — not just umpires — are unconsciously biased by simple “status characteristics.” Even constant monitoring and incentives can fail to train such biases out of us.

Technologically, Major League Baseball is in a position, thanks to its high-speed camera system, to enforce a completely accurate, uniform strike zone. The question is whether we, as fans, want our games to be fair and just, or whether we are compelled to watch the game because it mimics the real world, warts and all.

Brayden King is an associate professor of management and organizations at the Kellogg School of Management at Northwestern University. Jerry Kim is an assistant professor of management at Columbia Business School.