When Algorithms Take the Field – Inside MLB’s Robo-Umping Experiment

Conor here: An abomination, IMHO. And that’s just for the game. What goes unmentioned in the following piece is the labor issue. There are 76 umpires in the Major League Baseball Umpires Association who combined earn roughly $22.8 million per year, as well as per diem and travel, full medical, dental, and vision insurance, life insurance coverage, strong pension plans, etc. For now, MLB says robot umps are intended to aid, not replace, humans, but we’ll see how long that lasts. What would happen if the umps go on strike, for example, as they did in 1979, or resign en masse as they did in 1999 in an attempt to get a new labor agreement?

By Arthur Daemmrich, Professor of Practice in the School for the Future of Innovation in Society at Arizona State University, and Eric S. Hintz, Historian, Lemelson Center for the Study of Invention and Innovation, Smithsonian Institution. Originally published at The Conversation.

Baseball fans tuning into spring training games may have noticed another new wrinkle in a sport that’s experienced a host of changes in recent years.

Batters, pitchers and catchers can challenge a home plate umpire’s ball or strike call. Powered by Hawk-Eye ball-tracking technology, the automated ball-strike system replays the pitch trajectory to determine whether the umpire’s call was correct.

To minimize disruptions, Major League Baseball permits each team a maximum of two failed challenges per game but allows unlimited challenges as long as they’re successful. For now, the technology will be limited to the spring exhibition games. But it could be implemented in the regular season as soon as 2026.

Count future Hall of Famer Max Scherzer among the skeptics.

“We’re humans,” the Toronto Blue Jays hurler said after a spring training game in which he challenged two calls and lost both to the robo umps. “Can we just be judged by humans?”

Technological advances that lead to fairer, more accurate calls are often seen as triumphs. But as co-editors of the recently published volume “Inventing for Sports,” which includes case studies of over 20 sports inventions, we find that new technology doesn’t mean perfect precision – nor does it necessarily lead to better competition from the fan perspective.

Cue the Cameras

While playing in a cricket match in the 1990s, British computer scientist Paul Hawkins fumed over a bad call. He decided to make sure the same mistake wouldn’t happen again.

Drawing on his doctoral training in artificial intelligence, he designed an array of high-speed cameras to capture a ball’s flight path and velocity, and a software algorithm that used the data to predict the ball’s likely future path.

He founded Hawk-Eye Innovations Ltd. in 2001, and his first clients were cricket broadcasters who used the technology’s trajectory graphics to enhance their telecasts.

By 2006, professional tennis leagues began deploying Hawk-Eye to help officials adjudicate line calls. Cricket leagues followed in 2009, incorporating it to help umpires make what are known as “leg before wicket” calls, among others. And professional soccer leagues started using the technology in 2012 to determine whether balls cross the goal line.

Reaction to Hawk-Eye has been mixed. In tennis, players, fans and broadcasters have generally embraced the technology. During a challenge, spectators often clap rhythmically in anticipation as the Hawk-Eye official cues up the replayed trajectory.

“As a player, and now as a TV commentator,” tennis legend Pam Shriver said in 2006, “I dreamed of the day when technology would take the accuracy of line calling to the next level. That day has now arrived.”

But Hawk-Eye isn’t perfect. In 2020 and 2022, the firm publicly apologized to fans of professional soccer clubs after its goal-line technology made errant calls after players congregated in the goal box and obstructed key camera sight lines.

Perfection Isn’t Possible

Critics have also raised more fundamental concerns.

In their 2016 book “Bad Call,” researchers Harry Collins, Robert Evans and Christopher Higgins reminded readers that Hawk-Eye is not a replay of the ball’s actual position; rather, it produces a prediction of a trajectory, based on the ball’s prior velocity, rotation and position.

The authors lament that Hawk-Eye and what they term “decision aids” have undermined the authority of referees and umpires, which they consider bad for the games.

Ultimately, there are no purely objective standards for fairness and accuracy in technological officiating. They are always negotiated. Even the most precise officiating innovations require human consensus to define and validate their role. Technologies like photo-finish cameras, instant replay and ball-tracking systems have improved the precision of officiating, but their deployment is shaped – and often limited – by human judgment and institutional decisions.

For example, today’s best race timing systems are accurate to 0.0001 seconds, yet Olympic sports such as swimming, track and field, and alpine skiing report results in increments of only 0.01 seconds. This can lead to situations – such as Dominique Gisin and Tina Maze’s gold medal tie in the women’s downhill ski race at the 2014 Sochi Olympics – in which the timing officials admitted that their equipment could have revealed the actual winner. But they were forced to report a dead heat under the rules established by the ski federation.

With slow-motion instant replays, determining a catch or a player’s intention for a personal foul can actually be distorted by low-speed replay, since humans aren’t adept at adjusting to shifting replay speeds.

One of the big issues with baseball’s automated ball-strike system has to do with the strike zone itself.

MLB’s rule book defines the strike zone as the depth and width of home plate and the vertical distance between the midpoint of a player’s torso to the point just below his knees. The interpretation of the strike zone is notoriously subjective and varies with each umpire. For example, human umpires often call a strike if the ball crosses the plate in the rear corner. However the automated ball-strike system uses an imaginary plane that bisects the middle – not the front or the rear – of home plate.

There are more complications. Since every player has a unique height, each has a unique strike zone. At the outset of spring training, each player’s height was measured – standing up without cleats – and then confirmed through a biomechanical analysis.

But what if a player changes their batting stance and decides to crouch? What if they change their cleats and raise their strike zone by an extra quarter-inch?

Of course, as has been the case in tennis, soccer and other sports, Hawk-Eye can help rectify genuinely bad calls. By allowing teams to correct the most disputed calls without eliminating the human element of umpiring, MLB hopes to strike a balance between tradition and change.

Fans Have the Final Say

Finding a balance between machine precision and the human element of baseball is crucial.

Players’ and managers’ efforts to work the umpires to contract or expand the strike zone have long been a part of the game. And fans eagerly cheer or jeer players and managers who argue with the umpires. When ejections take place, more yelling and taunting ensues.

Though often unacknowledged in negotiations between leagues and athletes, fan enthusiasm is a key component of whether to adopt new technology.

For example, innovative “full-body” swimsuits contributed to a wave of record-breaking finishes in the sport between 2000 and 2009. But uneven access to the newest gear raised the specter of what some called “technological doping.” World Aquatics worried that as records fell simply due to equipment innovations, spectators would stop watching and broadcast and sponsorship revenue would dry up. The swimming federation ended up banning full-body swimsuits.

Of course, algorithmic officiating differs from technologies that enhance performance and speed. But it runs a similar risk of turning off fans. So MLB, like other sports leagues, is being thrust into the role of managing technological change.

Assessing technologies for their immediate and long-term impact is difficult enough for large government agencies. Sports leagues lack those resources, yet are nonetheless being forced to carefully consider how they introduce and regulate various innovations.

MLB, to its credit, is proceeding incrementally. While the logical conclusion to the current automated ball-strike experiment would be fully electronic officiating, we think fans and players will resist going that far.

The league’s challenge system is a test. But the real umpires will ultimately be the fans.