What’s Really Wrong With Standardized Tests

Tuesday, April 29th, 2025

Standardized testing is glorious, Bryan Caplan proclaims, but many standardized tests royally suck:

The worst prominent test is almost surely the Graduate Record Exam (GRE). About 9% of test-takers get a perfect score of 170 on the Quantitative part of the exam. A score of 169 puts you at the 87th percentile, and by 166 you’re already out of the top quarter. Most of the STEM majors taking the exam did the relevant coursework in middle school, so for fields that emphasize math, marginally lower scores largely capture not incomprehension but carelessness.

This is especially ridiculous when you remember that only top programs are highly selective. So when the focal standardized exam bunches all the top students together, the exam delivers near-zero value. At least in STEM fields, the point of the GRE is no longer to pinpoint the stars who deserve admission to top programs. The point is to weed out the manifestly unqualified. So the final cut almost has to be grotesquely “holistic.”

The regular SAT math is, by comparison, vastly better. Something like 1% get a perfect score — roughly one-tenth the share that get a perfect score on GRE math. But in absolute terms, the SAT still sucks. At least for STEM students, the problems are easy, so marginally lower scores again primarily capture carelessness rather than incomprehension. Since about two million students take the SAT, roughly 20,000 have perfect math scores — more than enough to fill all the spots in the Ivy League.

[…]

Perfect scores should be vanishingly rare. Instead of clumping the best candidates together, you should be able to clearly distinguish the 99th percentile from the 99.9th, 99.99th, and so on.

Out of all the well-known standardized tests, the Medical College Admission Test (MCAT) best satisfies these conditions. A perfect score is, bizarrely, 528. The fraction that gets a 528 in a given year is about .02%.

[…]

When your goal is to find the best of the best, the ideal test is so demanding that you get a big clump of scores not at the top of the distribution, but at the bottom.

At least one such standardized test exists: the Putnam Competition. As you’d expect, it’s a test of mathematical prowess. To call the test “hard” is a severe understatement: In 2025, the median score was 2 out of 120. Which is historically high! In many years, the median score is exactly 0 for the roughly 4000 test-takers, who are already highly selected. At the other end of the distribution, only five perfect scores have ever been achieved.

[…]

There are many lines of defense in the War Against Merit. The first is to get rid of standardized tests entirely. The second is to go “test-optional.” But if these approaches are too blatantly corrupt, there is a third option. A stealthy way to pretend applicants are far more equal than they truly are.

Just use lousy top-coded tests.

[…]

Admissions to graduate econ programs could be greatly improved simply by requiring the AP Microeconomics and AP Macroeconomics exams. They’re flawed tests, but if you can’t get 5’s on both, you’re not ready for grad school. Indeed, graduate admissions could probably be sharply improved across the board if programs required 5’s on all subject-relevant APs. Would-be historians should have 5’s on the U.S., European, and world history APs just to apply, and would-be literature professors should have 5’s on English literature and language.

Comments

  1. Bill says:

    There’s a fourth line of defense in the War against Merit.

    My son recently finished his math PhD [reams of parental kvelling redacted] and he and his undergrad math friends were frankly contemptuous of perfect SAT or ACT math scores, since anyone can get one after the recentering of the SAT in 1995.

    In 1972 (a year I choose completely at random), perhaps 0.1% of students got an 800 on the math SAT; now it’s more like 1-2%.

    The recentering benefited high scorers the most; a student scoring 500/500 (verbal/math) in 1972 would score maybe 520/530 in 2023. A student scoring 700/700 in 1972 would score about 730/790 in 2023.

    I don’t think the recentering was done to benefit students or educators. As far as I can tell, it helps parents feel better that their kids get higher numbers, it obscures who the really superior students are by cutting off the top of the grading curve and it greatly rewards students who spend a lot of time grinding it out in test preparation courses.

  2. Roo_ster says:

    The humanities AP exams are globo-homo narrative-buttressing trash. They are useful primarily to keep disruptive ‘tards from bothering average & above average students in public school.

Leave a Reply