Can we solve this by building trustworthy systems out of untrustworthy parts?

Wednesday, October 2nd, 2019

The United States government’s continuing disagreement with the Chinese company Huawei underscores a much larger problem with computer technologies in general, Bruce Schneier points out:

We have no choice but to trust them completely, and it’s impossible to verify that they’re trustworthy. Solving this problem ­ which is increasingly a national security issue ­ will require us to both make major policy changes and invent new technologies.

The Huawei problem is simple to explain. The company is based in China and subject to the rules and dictates of the Chinese government. The government could require Huawei to install back doors into the 5G routers it sells abroad, allowing the government to eavesdrop on communications or — even worse — take control of the routers during wartime. Since the United States will rely on those routers for all of its communications, we become vulnerable by building our 5G backbone on Huawei equipment.

It’s obvious that we can’t trust computer equipment from a country we don’t trust, but the problem is much more pervasive than that. The computers and smartphones you use are not built in the United States. Their chips aren’t made in the United States. The engineers who design and program them come from over a hundred countries. Thousands of people have the opportunity, acting alone, to slip a back door into the final product.

There’s more. Open-source software packages are increasingly targeted by groups installing back doors. Fake apps in the Google Play store illustrate vulnerabilities in our software distribution systems. The NotPetya worm was distributed by a fraudulent update to a popular Ukranian accounting package, illustrating vulnerabilities in our update systems. Hardware chips can be back-doored at the point of fabrication, even if the design is secure. The National Security Agency exploited the shipping process to subvert Cisco routers intended for the Syrian telephone company. The overall problem is that of supply-chain security, because every part of the supply chain can be attacked.

Can we solve this by building trustworthy systems out of untrustworthy parts?

It sounds ridiculous on its face, but the internet itself was a solution to a similar problem: a reliable network built out of unreliable parts. This was the result of decades of research. That research continues today, and it’s how we can have highly resilient distributed systems like Google’s network even though none of the individual components are particularly good. It’s also the philosophy behind much of the cybersecurity industry today: systems watching one another, looking for vulnerabilities and signs of attack.

Security is a lot harder than reliability. We don’t even really know how to build secure systems out of secure parts, let alone out of parts and processes that we can’t trust and that are almost certainly being subverted by governments and criminals around the world. Current security technologies are nowhere near good enough, though, to defend against these increasingly sophisticated attacks. So while this is an important part of the solution, and something we need to focus research on, it’s not going to solve our near-term problems.

At the same time, all of these problems are getting worse as computers and networks become more critical to personal and national security. The value of 5G isn’t for you to watch videos faster; it’s for things talking to things without bothering you. These things — cars, appliances, power plants, smart cities — increasingly affect the world in a direct physical manner. They’re increasingly autonomous, using A.I. and other technologies to make decisions without human intervention. The risk from Chinese back doors into our networks and computers isn’t that their government will listen in on our conversations; it’s that they’ll turn the power off or make all the cars crash into one another.

All of this doesn’t leave us with many options for today’s supply-chain problems. We still have to presume a dirty network — as well as back-doored computers and phones — and we can clean up only a fraction of the vulnerabilities.


  1. Adar says:

    Just stop using the telephone.

  2. Graham says:

    Or start peppering one’s discourse with gratuitous insults against China. Or subtly backhanded praise. Or both. Keep em guessing, and try to use as many local or regional idioms and colloquialisms as possible.

  3. TRX says:

    I remember a similar furor over putting RFID chips in US passports. We must have RFID chips in our passports, now! Even though they introduced a whole new security hole since they were remotely readable and cloneable…

    Similarly, we must have 5G networking now because… we need to bump up our sales by forcing the purchase of new hardware?

  4. CVLR says:

    I think about this a lot and I’ve developed a few ideas. It comes down to the wholesale, in situ replacement of the “engineering” culture we currently have.

    Here are some technical points:

    * The network connection is the critical chokepoint. An unsecure (sic) airgapped computer is almost as good (possibly as good as) a secure networked computer.

    * Increasingly, computers are more and more specialized, which is to say, less and less general-purpose. With a decrease in general-purposeness comes a decrease in the scope of things that it must be possible for a networked device to transmit for its effective operation.

    * Therefore, it becomes practical to restrict the lifetime communications of some device to a predefined schema set at manufacture time. (Think along the lines of GraphQL.) SEARS once provided legendary schematics for its products; there is no reason for an interested layman of moderately above-average intelligence to be denied access to a document providing a comprehensive description of his device’s capabilities.

    * Furthermore, the network connection can be cleanly extracted from the main system, and should be so. It can run on its own hardware; reside on read-only memory; be written in Haskell. If a device needs more comms than it currently possesses, its manufacturer can issue a microSD-card-like “flash” update. Between real costs to updates and a rigorous-though-not-prohibitive certification process, it will be uneconomical to issue half-baked software. To become a certifier must be a grueling process, within grasp only for the elite of the elite, and the profession’s pay pegged at a minimum of 5X the pay of a similar position in the private sector. The return on investment will be in excess of 100X.

    * TH3. (GOTO: SQLite)

    * Physical hardware switches for every sensor and wireless antenna. You flip open your smartphone, the microphone activates. You press a small toggle button, the camera turns on. Your flip your smartphone closed, it becomes a pager: capable of receiving calls and notifications, incapable of transmitting data, including geomatical information.

    * Facebook, Instagram, Twitter, Snapchat, et al. should’ve been protocols; they aren’t, because in such a world the founders wouldn’t be billionaires and the world’s privacy wouldn’t be flooding into their datacenters. Similarly, with contemporary inexpensive, powerful, and energy-efficient hardware, it’s eminently possible, bordering on trivial, to design a “home cloud” distributed network of interoperable devices, sensibly firewalled from the Internet (see above). It hasn’t happened, for similar reasons.

    These are not, as some would have you believe, insurmountable problems.

Leave a Reply