What Did the Kindergarten Study Really Find?

Wednesday, June 26th, 2013

How Does Your Kindergarten Classroom Affect Your Earnings?” is one of the most impressive empirical papers ever written, Bryan Caplan says:

The data collection is amazing. The empirical analysis is clear and careful. Above all else, the authors’ methods are transparent. Lesser authors would have buried the conflicting findings. Chetty et al. took the high road.

Unfortunately, the world is so eager for stories about the power of early education that their paper is being badly misinterpreted.

How so?

Here’s what they really find about the effect of early schooling on adult income:

1. Project STAR was an experiment designed to test the effect of class size. The experiment found that students assigned to small classes earned $4 more per year. If you add demographic controls, students assigned to small classes earned $124 less per year. (more)

2. You can use Project STAR’s data to (non-experimentally**) test for other effects. When you do, almost all measures of teacher quality fail to increase adult earnings:

The few other observable teacher characteristics in the STAR data (degrees, race, and progress on a career ladder) have no significant impact on scores or earnings.

3. There is one measure of teacher quality that does matter: Whether the teacher has more than 10 years of experience. Chetty et al. find that students assigned to a kindergarten teacher over this experience cut-off eventually earn $1093 extra dollars per year. But bear two reservations in mind. (a) The t-stat is only 2.4 — extremely low for a non-experimental test with 6005 observations. (b) If you measure experience in years, rather than using their binary “more than 10 years of experience” variable, the point estimate is a statistically insignificant $57 per year.

4. Teacher experience only matters in kindergarten:

The effect of teacher experience on test scores is no longer statistically significant in grades 1-3. Consistent with this result, teacher experience in grades 1-3 also does not have a statistically significant effect on wage earnings.

By my count, Chetty et al. have five measures of teacher quality (degrees, race, progress on a career ladder, >10 years experience, and years of experience), study four different grades (K, 1, 2, and 3), and discover precisely one statistically significant effect on income. With 20 different measures, you’d expect one to be statistically significant at the 5% level by chance alone. And these are just the non-experimental results. Experimentally, Project STAR finds that class size does not raise adult income.

Comments

  1. Ben says:

    I don’t know.

    I say we give it another decade of study at least. Probably one or two really well-funded Federal Programs will help the under-achievers. Probably a group of K Street NGOs will be able to help by creating a Foundation. Next election is coming up; it’s critical we get this issue on the platform.

    Next Up: Use of Flash Card in utero proven to increase salary for those over 85, p<.50.

Leave a Reply