An Interesting Test

Friday, November 2nd, 2007

David Foster looks at An Interesting Test:

In an old Heinlein SF novel, applicants to the Space Academy are required to take a variety of aptitude tests. One of these tests involves dropping beans into a bottle…with the eyes closed. Applicants are told that the test measures “spatial perception” or something along those lines — but it’s actually a test of honesty.

I was reminded of this scenario by an article titled For Love of the Game, which appeared in the 3/12 issue of Forbes. There’s an old test that was originally used by the military to find people with an aptitude for clerical positions. All you have to do look in a table for a four-digit number and circle it where it appears. It seems like it would be difficult for any literate person to fail at this. Yet this simplistic test turns out to have predictive power for career success across a wide range of fields, including those that have little or nothing to do with clerical ability.

The Bureau of Labor Statistics surveyed 12,700 people (ages 14-22) and then follwed them to see how well they were doing. The subjects were paid $50 to take several tests, including a traditional Army intelligence test and the coding-speed test described above. They had no particular incentive to do well on any of the tests.

Recent research by Carmit Segal of Harvard indicates that performance on the coding-speed test has significant predictive power for the individual’s income 20 years later. This is true even when holding IQ score constant. And for participants who never earned a college degree, the coding-speed measurement has more predictive power than does IQ score.

The explanation suggested by Carmit is that what is really being measured by the coding speed test is intrinsic motivation: how much effort will someone put into the performance of a task when the only reward is the task itself? Just like Heinlein’s bean-in-the-bottle test measures what someone will do when no one is watching, the coding-speed test as performed by BLS measures what someone will do when no one is paying or otherwise rewarding good performance.

Leave a Reply