The problem with Bloom’s Two-Sigma Problem, Kurt van Lehn found, was that Bloom compared classroom mastery learning against one-on-one tutor-based mastery learning, without holding the goal-level of mastery constant:
For 25 years, researchers have been seeking solutions for Bloom’s (1984) “2 sigma problem.” Although one would expect many of the studies of human tutoring to show a 2.0 effect size, only two studies did. This section discusses those two studies, which now seem like outliers.
Bloom (1984) summarized six studies of human tutoring reported in the dissertations of Anania (1981) and Burke (1983). All six studies had effect sizes close to 2.0. Of these studies, only Anania’s Experiment 3 was included in this review because only it involved one-on-one tutoring. The other five experiments summarized by Bloom involved each tutor working daily with a group of three students. However, Anania’s one-on-one experiment did produce an effect size of 1.95, so let us examine it more closely.
A common explanation for the effectiveness of tutors in the studies discussed by Bloom is that they were highly trained, expert tutors. However, the original sources for Bloom’s review say that the tutors were “undergraduate education majors” (Anania, 1981, p. 58) who “met the experimenter each day for one week before the instruction began” (Burke, 1983, p. 85) for training on both tutoring and the task domain: probability. This suggests that the Bloom tutors were not the “super tutors” that they have sometime been thought to be.
Anania’s third experiment (and the other five Bloom experiments as well) included a third condition, which was mastery learning in the classroom. That is, after students had finished classroom instruction on a unit, they took a mastery test. If they scored 80%, then they were considered to have mastered the unit and could go on to the next unit. Students who scored less than 80% had to resume studying the unit and repeat the mastery test. In all six experiments, the mastery learning students scored about 1.0 standard deviations higher on posttests than the ordinary classroom students.
Moreover, the tutoring conditions of all six experiments also involved mastery learning. That is, the tutees took the same mastery tests, restudied, and so on, but they worked with a tutor instead of a classroom teacher. However, the mastery threshold for the tutoring conditions was set at 90% instead of 80% for the classroom implementation of mastery learning (Anania, 1981, pp. 44–45). That is, the tutors were holding their students to a higher standard of mastery than the classroom teachers. This alone could account for the advantage of tutoring (2.0 effect size) over mastery learning (1.0 effect size).