The CIA was shocked to the core by the fall of the Soviet Union, Martin Gerri notes:
I was there. Our biggest strategic antagonist for 45 years seized up and died, and we had no idea it was happening. The CIA missed the initial test of the Soviet atom bomb — and India’s bomb, and Pakistan’s as well. 9/11, the sort of disaster the Agency was erected to prevent, came as a complete surprise. In hindsight these episodes appear inevitable and thus predictable, but in fact most historic discontinuities are extreme low-probability events. Place the filter of Platonic truth over them, and they disappear from sight. It would have been career suicide for an analyst to brief, “Mr. President, we estimate there’s a 1 in 1,200 chance that terrorists will crash airliners into the World Trade Center and the Pentagon.” Money and glory attend to the immediate and obvious.
Trapped in an impossible situation, analysts developed survival mechanisms. For one, they wrote too much. A blizzard of classified material blew out not just from the CIA but the entire Intelligence Community. Nobody read the stuff, but if something unforeseen occurred, we could be sure that at least one document had mentioned the possibility. The analytic style was also a hedge against failure. Robert Gates, then head of the directorate of analysis and later director of CIA and secretary of defense, commissioned a logician to scrutinize the language of the President’s Daily Brief, or PDB. The logician discovered a large measure of unclarity in the PDB. The same words had different meanings across time. It was frequently hard to tell whether a prediction was being made or not. The analysts who did the writing tended to be brilliant wielders of the English language. It was the blessed Sherman Kent who contorted their work.
[…]
The most meaningful improvement at scale for the Agency would be to kill the cult of secrecy and redirect resources towards the dominant information structures of our century: the web and AI. The government will never lose its appetite for secrets. While technology can satisfy much of this hunger, we aren’t about to pension off our spies. It’s a question of perspective. “Stealing secrets” is expensive and carries great human and diplomatic risk. Covert sources can play us false — that’s what happened in Iraq. The culture must be liberated from an addictive dependence on classification; Top Secret should never correlate to great authority. This is particularly true in the age of sexting and performative elites, when enough secrets get spilled online to make a grown spy cry.
As Robert Redford’s character in Three Days of the Condor would remind us, even before the internet the immense majority of intelligence material was collected from “open sources” — news media, books, government and corporate publications, etc. After the arrival of the web, the disproportion ballooned exponentially. Open information is faster, nimbler, cheaper, and much less dangerous to obtain. The Agency knows this, and occasionally will acknowledge it with a wave of the hand. But it has never acted on it, never put its money there. Although criticism after 9/11 and Iraq forced the establishment of an Open Source Center — my home turf — the unit was ridiculously underfunded and subservient to operations.
If the CIA’s business is to understand the world, then a major part of that mission should be to understand the web at great depth. For every digital utterance, the analyst must be able to penetrate beyond author and site to provider, location, funding, ideology, past history, connection to similar posts elsewhere, affiliation with state and non-state actors. Analyzing video should have primacy over text — this is alien to government thinking but it’s the way of the web. The digital universe is a huge and shifting target. Powerful AI applications will keep track of billions of moving parts, constructing a dynamic map of digital space in the manner of the 16th century explorers, placing the warning when appropriate, “Here be monsters.”
Skeptics will argue that all online material is horribly tainted — that the internet is the mother of lies. That would be accurate and all to the good. To the propaganda analyst, disinformation is a moveable feast. Among many benefits, it can provide an answer to the most difficult intelligence question to ascertain: intent. The point, after all, isn’t to strive after Platonic truth but to extract knowledge about how the world works.