PISA has the capacity to spread embarrassment far and wide, in rich as well as poor countries, and Dr. James Thompson is “all in favour of that” — but that means that many countries opt out of it (and TIMMS), especially poorer countries in Africa, but a new working paper from the Center for Global Development gets around this:
Internationally comparable test scores play a central role in both research and policy debates on education. However, the main international testing regimes, such as PISA, TIMSS, or PIRLS, include very few low-income countries. For instance, most countries in Southern and Eastern Africa have opted instead for a regional assessment known as SACMEQ. This paper exploits an overlap between the SACMEQ and TIMSS tests—in both country coverage, and questions asked— to assesses the feasibility of constructing global learning metrics by equating regional and international scales. I compare three different equating methods and find that learning levels in this sample of African countries are consistently (a) low in absolute terms, with average pupils scoring below the fifth percentile for most developed economies; (b) significantly lower than predicted by African per capita GDP levels; and (c) converging slowly, if at all, to the rest of the world during the 2000s. While these broad patterns are robust, average performance in individual countries is quite sensitive to the method chosen to link scores. Creating test scores which are truly internationally comparable would be a global public good, requiring more concerted effort at the design stage.
The results are grim:
Substantively, the results here are daunting for African education systems. Most of the national test-score averages I estimate for the thirteen African countries in my sample fall more than two standard deviations below the TIMSS average, which places them below the 5th percentile in most European, North American, and East Asian countries. In contrast, scores from the SACMEQ test administered to math teachers are much higher, but fall only modestly above the TIMSS sample average for seventh- and eighth-grade pupils, in line with earlier analysis by Spaull and van der Berg (2013). African test scores appear low relative to national GDP levels; in a regression of average scores on per capita GDP in PPP terms, average scores in the SACMEQ sample are significantly below the predicted value using all three linking methodologies. Furthermore, there is little sign that African scores were improving rapidly or converging to OECD levels during the 2000s.
Really grim:
In some African countries teachers seem to have lower abilities than students in Europe or East-Asia!
Raising scholastic attainment is unlikely to be a simple question of investing money, Dr. Thompson notes:
A summary of investment in education suggests that the pay-off is front-end loaded: the first $5000 has a big effect, and then it tends to plateau thereafter. Another way of looking at it is to note that once countries get to $16,000 GDP per capita then schooling in those countries accounts for only 10% of the variance of student attainment. So, poor countries (most of Africa is well below this level) should have plenty of scope for educational gains.
The average IQ among the Bantu (the people once enslaved and brought to the New World) is in the mid 60s to mid 70s; for the Bushmen it is in the 50s to 60s; and for the pygmies it is lower yet. Lynn has the details (“Racial Differences in Intelligence,” Washington Summit Publishers, Augusta, GA, 2006). The IQs of African Americans average about 85 (std. dev 12) because of White admixture.
…but doesn’t question the value of educational gains.
The very fact their GDP diverges from their mathematical skills strongly challenges the idea of economies needing educated workforces.
Further, we should expect the self-righteous scholar caste (progressives) to say everyone needs to be like them and judge themselves like them, with zero or negative justification. We should expect them to bullshit it, in other words.
It’s rather likely that “student attainment” is zero sum, that when the economy really needs someone who knows math, it also finds a way to teach them, with or without schools, and that the actual proportion who need it is ridiculously tiny.
I am never really convinced that the test scores in Africa mean what we think they mean.
Especially, I fear that they are measuring crystallized knowledge rather than “raw horsepower.”
I think until you have a reading culture, a test-taking culture, and the whole academic ethos of Scarsdale, it’s not clear to what extent the scores are comparable across cultures and continents.
That is not to say that I disbelieve the scores, or that I consider the scores to be of no value. Rather, I’m arguing that the scores are “pseudo-precise.”
Robert Sternberg has said it better than I can. Dr. James Thompson is worth reading, and I learn a lot from his blog. I’m not sure we know how to interpret the test data coming out of Africa.