Isegoria

The most relevant aspect of Neal Stephenson‘s work to an audience of AI researchers was the Young Lady’s Illustrated Primer from his 30-year-old novel The Diamond Age:

At the beginning of this book we see a conversation between Lord Finkle-McGraw, who is an Equity Lord in a futuristic neo-Victorian society, and John Hackworth, an engineer who works in one of his companies.

Finkle-McGraw is a classic founder. He didn’t come from a privileged background, except insofar as having a stable family and a decent basic education confers privilege. But when he was young he was brilliant, ambitious, hard-working, and had a vision. He built that into something valuable and as a result became rich and powerful. As so often happens, he used his money to make life good for his children by sending them to the right schools, connecting them to the right people, and so on.

He wasn’t entirely happy with the results. His kids didn’t end up having the traits that had made him successful. He suspects it’s because they didn’t have to work hard and overcome obstacles. Now he has a granddaughter. He knows that the parents are going to raise this girl in the same way, with the same results. He can’t interfere in a heavy-handed way. But the parents can’t possibly object if he gives his granddaughter an educational book. So he commissions Hackworth to make the Young Lady’s Illustrated Primer, an interactive book that will adapt as the user grows and learns. This book is powered by molecular nanotechnology, but any present-day reader will immediately recognize it as an AI system.

As the plot unfolds, three copies of the Primer are made and bestowed on girls from very different backgrounds. In two cases the result is a sort of fizzle. The Primer works as it’s supposed to for a while, but these girls lose interest and set it aside. The third copy falls into the hands of a girl from an abusive and underprivileged background, and it ends up giving her close to superhuman abilities.

Thirty years on, I think I have enough distance on this to grade my performance. I’m happy with the fact that the Primer, as described in the novel, doesn’t invariably produce great results. That seems like a measured and realistic outcome. Nevertheless it’s clear that when I wrote this thing I was influenced by a strain of techno-utopian thinking that was widespread in the mid-1990s, when the Internet was first becoming available to a mass audience. In those days, a lot of people, myself included, assumed that making all the world’s knowledge available to everyone would unlock vast stores of pent-up human potential.

That promise actually did come true to some degree. It’s unquestionably the case that anyone with an Internet connection can now learn things that they could not have had access to before. But as we now know, many people would rather watch TikTok videos eight hours a day.

[…]

The gist of it is that the system we’ve traditionally used for evaluating students’ performance — homework and tests — just happens to be exquisitely vulnerable to being hacked by students who simply use conversational AI systems to do all the work for them. And they are doing so on a massive scale, to the point where conventional education has essentially stopped functioning. The only way to fairly evaluate how much a student has learned now is by marching them into a classroom with no electronics, handing them a pencil and a blank blue book, and assigning them an essay to write or a math problem to solve. Even this is impractical given that many students never really learned to write by hand.

[…]

This question sent me down a rabbit hole on the topic of self-reliance. After all, if AI-driven education does nothing more than make students even more reliant on AI, then it’s not education at all. It’s just a vocational education program teaching them how to be of service to AIs. The euphemism for this role is “prompt engineer” which seems to be a way of suggesting that people who feed inputs to AIs are achieving something that should be valorized to the same degree as designing airplanes and building bridges.

If such a system actually did its job it would have the paradoxical effect of making students less, rather than more, reliant on the use of AI technology.

[…]

My thought last week was that Self-Reliance might contain some wisdom applicable to the challenge of how to educate people in the modern world to rely upon their own knowledge and skill set rather than using AI all the time.

Reader, I did not find anything like that upon re-reading this essay. More the opposite. The overall drift of what Emerson is saying here — and he says it over and over — is that each mind is uniquely positioned to see certain insights. The self-reliant person shouldn’t ignore those merely because they don’t match the conventional wisdom. “The eye was placed where one ray should fall, that it might testify of that particular ray…God will not have his work made manifest by cowards….He who would gather immortal palms (i.e. be honored for great achievements) must not be hindered by the name of goodness, but must explore if it be goodness. Nothing is at last sacred but the integrity of your own mind.”

That is all intoxicating stuff for a smart young man who styles himself as a free thinker and nonconformist, which is why, when I was in my early twenties, I inhaled it like fentanyl fumes off hot foil. But during the same years as I was poring over this essay and jotting down quotes in my notebook, I was writing by far the worst novel I have ever written—a book that has never been published and never should be.

Emerson grew up in Boston, attended Boston Latin and Harvard, then traveled around Europe and visited England where he hung out with Wordsworth, Coleridge, and Thomas Carlyle. His brain was preloaded with the best knowledge base that could possibly have been given a young person of that era. He’d been trained to think systematically and rigorously and to express himself with great fluency in English and probably Latin and other languages as well.

So, yes, when an idea popped into Emerson’s head, chances are it was a pretty damned good one. His own advice about self-reliance was actually worth taking in his own case. And I’d guess that the audience for this essay was similarly well educated. By the time any young person happened upon Self-Reliance, they were probably 99% of the way to being an intellectually mature, highly capable person, and just wanted a bit of self confidence to follow through on good ideas that were coming into their heads—as a result of being that well educated and trained.

When the same advice falls on the ears of people who are not as well informed and not as good at thinking systematically, though, it’s rubbish.

When I first read Self-Reliance, only a few years had passed since the premier of the first Star Wars movie. There’s a pivotal moment in that film when Luke Skywalker is piloting his fighter through the trench on the Death Star, making his bombing run against impossible odds, and he hears Obi-Wan Kenobi’s voice in his head telling him to use the Force. Luke switches off his targeting computer to the consternation of the brass in the ops center. We all know the outcome. It’s a great moment in cinema, and it perfectly encapsulates a certain way of thinking emblematic of the 1970s late hippie scene: the seductive proposition that no one needs a targeting computer, that all we need to do is trust our feelings. Who doesn’t love to hear that? I loved hearing it from Ralph Waldo Emerson, and spent a couple of years of my life building a terrible novel on that foundation.

I have discussed The Diamond Age before.

David Foster reviewed Gerhard Neumann‘s Herman the German ages ago, but I was recently reminded of it:

This is the autobiography of a man who was born to a Jewish family in Germany, apprenticed as an auto mechanic, attended engineering school, moved to China in 1938, was interned by the British as an enemy alien in 1939, transferred to the American forces, joined Claire Chennault’s Flying Tigers, repaired the first Japanese Zero fighter to be captured in potentially flyable condition, became a U.S. citizen by special act of Congress, and went on to run GE’s entire jet engine business, which he played a major role in creating. (The preceding may be the longest single sentence I’ve ever written in a blog post.) The book should be of interest to those interested in aviation, technology, management, social history, the WWII era, and/or China.

Gerhard Neumann was born in Frankfurt/Oder in 1917, where his father was owner of a factory that processed feathers and down. Gerhard’s parents were Jewish but nonpracticing–a Christmas tree was traditional in the Neumann home–and their approach to child-raising was closer to stereotypically Prussian than to stereotypically Jewish: “You did exactly as you were told by your parents. There was no such thing as saying no to them!…You were not to have a hand in your pocket while talking to grown-ups…Showing any emotion in Prussia was considered sissyish. There was no kissing between parents and children–only a peck on the cheek before going upstairs punctually at nine o’clock; and there was absolutely no crying.”

On the other hand, Neumann could do pretty much what he wanted with his spare time. In 1927, at the age of 10, he rode his bike out to a grass strip where someone was giving airplane rides for 5 marks, which he paid with money from his piggy bank. His parents weren’t angry at him for taking this flight without permission; indeed, they were so entranced with his description of the way the town looked from the air that they soon took an airplane ride themselves! At the age of 13, Neumann bought a folding kayak and, with some camping gear and a 12-year-old friend, took long journeys on the Oder River, all the way to the Baltic Sea. Few parents in America today–or in Germany either, I’d bet–would now allow this level of independence to a 12- or 13-year old.

Neumann had no interest in the family feather business; he wanted to be an engineer. A 2- or 3-year machinist or mechanic apprenticeship was mandatory for admission to any German engineering academy: Neumann’s father asked the 10 cab drivers of Frankfurt/Oder to recommend the garage where they thought the boy would learn the most, and the answers were unanimous: Albert Schroth’s. So began Gerhard Neumann’s apprenticeship, which, other than the technologies involved, could have been something out of the Middle Ages. “In winter my hands were frozen purple. Wear work gloves? ‘What’s the matter, boy, are you a girl?’ When my hands were bleeding, Herr Schroth pointed to the large bottle of iodine in the backroom and mumbled something about faules Fleisch (lazy flesh.) No Band-Aids, no pitying, no time out.”

At first, Neumann had second thoughts about the path he had chosen. “My friends were still continuing at the Gymnasium, spending their days in comfortable and clean surroundings; here I was, accustomed to a fine home and the luxury of two maids and a chauffeur, becoming a grease monkey for three long years.” But Neumann found the work interesting, and took pride in the high reputation of the shop.

At the conclusion of the three-year apprenticeship, Herr Schroth said “Thank you, Neumann”…the only time that he had ever said “thank you” to his apprentice, or called him anything other than “boy”…and sent a bouquet of flowers to Neumann’s mother. “I felt sincerely grateful when I, in turn, thanked Herr Schroth–the man whom I had always addressed as Meister and who had given me a solid groundwork for what I hoped would be a rewarding engineering future.”

Neumann says that up to the time he left Frankfurt to attend engineering college in the mid-1930s, he encountered no open anti-Semitism at all. Even at the Mittweide engineering college, where he was one of three Jewish students (each of their fathers had been soldiers in the First World War, which made them eligible for a college education), he says that he was never insulted by Nazi fellow students. Even allowing for the fact that attitudes toward Jews did differ considerably in different parts of Germany, these statements are hard to believe given that the Hitler regime had been in place since 1933.

About 15% of the Mittweide students were foreign, and they were exempt from the requirement, binding on German students, to have previously undergone an apprenticeship. “Because they had never dirtied their hands or bloodied their knuckles in apprentice-type training, they did not benefit from the Mittweida-type education anywhere as much as did a German student.” The Mittweida approach to engineering education included drawings in which errors had been deliberately inserted–“We were taught to ‘get a feel’ for drawings laid before us. The question constantly posed was, Would it really do the job if it were built just as shown on this drawing?”

Engineering students were exempt from the draft while in school, but not after graduation. A few weeks before the end of 1938, Neumann noticed an item on the college bulletin board: the Chinese government was looking for German mechanical engineers. And the Chinese Nationalist government had arranged with the German Nazi government that these engineers would have their German military service requirement deferred.

“The notice was vague about the jobs, which apparently were somewhere in the interior of the Chinese mainland and were connected with defense. A job in faraway China sounded incredibly exciting…”

Incredibly exciting it turned out to be, and most of the book is concerned with Neumann’s remarkable adventures in China. It was also, surely, lifesaving–had Neumann remained in Germany, it is most unlikely he would have survived the Holocaust.

Read the whole review.