Complex systems won’t survive the competence crisis, Harold Robertson argues:
In a span of fewer than six months in 2017, three U.S. Naval warships experienced three separate collisions resulting in 17 deaths. A year later, powerlines owned by PG&E started a wildfire that killed 85 people. The pipeline carrying almost half of the East Coast’s gasoline shut down due to a ransomware attack. Almost half a million intermodal containers sat on cargo ships unable to dock at Los Angeles ports. A train carrying thousands of tons of hazardous and flammable chemicals derailed near East Palestine, Ohio. Air Traffic Control cleared a FedEx plane to land on a runway occupied by a Southwest plane preparing to take off. Eye drops contaminated with antibiotic-resistant bacteria killed four and blinded fourteen.
[…]
The core issue is that changing political mores have established the systematic promotion of the unqualified and sidelining of the competent. This has continually weakened our society’s ability to manage modern systems. At its inception, it represented a break from the trend of the 1920s to the 1960s, when the direct meritocratic evaluation of competence became the norm across vast swaths of American society.
In the first decades of the twentieth century, the idea that individuals should be systematically evaluated and selected based on their ability rather than wealth, class, or political connections, led to significant changes in selection techniques at all levels of American society. The Scholastic Aptitude Test (SAT) revolutionized college admissions by allowing elite universities to find and recruit talented students from beyond the boarding schools of New England. Following the adoption of the SAT, aptitude tests such as Wonderlic (1936), Graduate Record Examination (1936), Army General Classification Test (1941), and Law School Admission Test (1948) swept the United States. Spurred on by the demands of two world wars, this system of institutional management electrified the Tennessee Valley, created the first atom bomb, invented the transistor, and put a man on the moon.
By the 1960s, the systematic selection for competence came into direct conflict with the political imperatives of the civil rights movement. During the period from 1961 to 1972, a series of Supreme Court rulings, executive orders, and laws—most critically, the Civil Rights Act of 1964—put meritocracy and the new political imperative of protected-group diversity on a collision course. Administrative law judges have accepted statistically observable disparities in outcomes between groups as prima facie evidence of illegal discrimination. The result has been clear: any time meritocracy and diversity come into direct conflict, diversity must take priority.
The resulting norms have steadily eroded institutional competency, causing America’s complex systems to fail with increasing regularity. In the language of a systems theorist, by decreasing the competency of the actors within the system, formerly stable systems have begun to experience normal accidents at a rate that is faster than the system can adapt. The prognosis is harsh but clear: either selection for competence will return or America will experience devolution to more primitive forms of civilization and loss of geopolitical power.
[…]
After the early 1970s, employers responded by shifting from directly testing for ability to using the next best thing: a degree from a highly-selective university. By pushing the selection challenge to the college admissions offices, selective employers did two things: they reduced their risk of lawsuits and they turned the U.S. college application process into a high-stakes war of all against all.
In 1984, Yale sociologist Charles Perrow’s Normal Accidents: Living With High-Risk Technologies explained that catastrophic failures are unavoidable and cannot simply be designed around, when you have systems that are both complex and tightly coupled:
The biggest shortcoming of the theory is that it takes competency as a given. The idea that competent organizations can devolve to a level where the risk of normal accidents becomes unacceptably high is barely addressed. In other words, rather than being taken as absolutes, complexity and tightness should be understood to be relative to the functionality of the people and systems that are managing them. The U.S. has embraced a novel question: what happens when the men who built the complex systems our society relies on cease contributing and are replaced by people who were chosen for reasons other than competency?
The answer is clear: catastrophic normal accidents will happen with increasing regularity. While each failure is officially seen as a separate issue to be fixed with small patches, the reality is that the whole system is seeing failures at an accelerating rate, which will lead in turn to the failure of other systems. In the case of the Camp Fire that killed 85 people, PG&E fired its CEO, filed Chapter 11, and restructured. The system’s response has been to turn off the electricity and raise wildfire insurance premiums. This has resulted in very little reflection.
[…]
Americans living today are the inheritors of systems that created the highest standard of living in human history. Rather than protecting the competency that made those systems possible, the modern preference for diversity has attenuated meritocratic evaluation at all levels of American society. Given the damage already done to competence and morale combined with the natural exodus of baby boomers with decades worth of tacit knowledge, the biggest challenge of the coming decades might simply be maintaining the systems we have today.
