Ian Ayres (Super Crunchers) guest-posts on the Freakonomics blog, asking Why Don’t Sports Teams Use Randomization?
The Boston Red Sox are famous for relying on number crunching to gain a competitive edge. But why don’t they proactively make some powerful data by creating randomized treatment and control groups? They could use their minor league teams, for instance, to figure out whether catchers or pitchers make better calls.They could even have a randomized trial of randomization — they could randomly assign the pitches for half the at-bats to be called in the traditional way (by the coach or the catcher) and the other half could be called by a random strategy established in advance. It would be a double-blind study, because neither the pitcher nor the hitter would need to know which system called the pitch.
If it turned out that the random strategy reduced the batting average of your opponents, that would be pretty strong evidence that it was a better strategy.
It turns out that football teams should almost always go for it on fourth down:
Or you could run an experiment to find out whether football teams should go for it more often on fourth down. Economist David Romer has crunched numbers to suggest that professional football teams should go for it fourth down a lot more than they currently do.
[...]
Amazingly, the data suggests that if it’s fourth down and your team has the ball on the opponent’s 33-yard line, you should go for it even if you have 9 yards left for a first down. NFL coaches have resisted Romer’s advice (though Pulaski Academy has started acting on it). But this is another area in which a little randomized testing could go a long way to help figure out what works. There are thousands upon thousands of college and high school games, but we collectively go for decades without figuring out whether simple changes in strategy could really produce better outcomes.
There are very few risk-takers, like Coach Leach, out there, and even fewer scientific risk-takers coaching football.