Retire Standard Deviation

Wednesday, January 22nd, 2014

“The statistician cannot evade the responsibility for understanding the process he applies or recommends,” Sir Ronald A. Fisher said, so it is time to retire standard deviation from common use and replace it with mean deviation, Nassim Nicholas Taleb suggests:

Standard deviation, STD, should be left to mathematicians, physicists and mathematical statisticians deriving limit theorems. There is no scientific reason to use it in statistical investigations in the age of the computer, as it does more harm than good — particularly with the growing class of people in social science mechanistically applying statistical tools to scientific problems.

Say someone just asked you to measure the “average daily variations” for the temperature of your town (or for the stock price of a company, or the blood pressure of your uncle) over the past five days. The five changes are: (–23, 7, –3, 20, –1). How do you do it?

Do you take every observation: square it, average the total, then take the square root? Or do you remove the sign and calculate the average? For there are serious differences between the two methods. The first produces an average of 10.8, the second 15.7. The first is technically called the root mean square deviation. The second is the mean absolute deviation, MAD. It corresponds to “real life” much better than the first — and to reality. In fact, whenever people make decisions after being supplied with the standard deviation number, they act as if it were the expected mean deviation.

It is all due to a historical accident: in 1893, the great Karl Pearson introduced the term “standard deviation” for what had been known as “root mean square error”. The confusion started then: people thought it meant mean deviation. The idea stuck: every time a newspaper has attempted to clarify the concept of market “volatility”, it defined it verbally as mean deviation yet produced the numerical measure of the (higher) standard deviation.

But it is not just journalists who fall for the mistake: I recall seeing official documents from the department of commerce and the Federal Reserve partaking of the conflation, even regulators in statements on market volatility. What is worse, Goldstein and I found that a high number of data scientists (many with PhDs) also get confused in real life.

Tuesday, January 21st, 2014

Weapons Man brought up grenade-launchers recently, and I asked him a few questions:

What’s the mechanical accuracy of these grenade launchers? And what kind of overall accuracy can a skilled shooter expect versus a semi-trained shooter? Are “groups” on the order of 10 yards at 100 yards?

It turns out that grenade-launchers are plenty accurate:

Those are good questions. Because of the high trajectory, you’re limited to 100-200m against a point target. (Like shooting through a window, one of the ranges we used to use had a window at 175m and I could put a round through that pretty consistently). Anybody who’d fired a few rounds and used the sight properly could hit a window at 100m. Area target like troops in the open you could hit with plunging fire to about 300-350m. At that point you’re at high angle (in the artillery definition) and going any higher brings the round back towards you.

The old M79 had a ladder sight and a close in battlesight (60m) when you flip the sight down, but it was calibrated for 1960s ammunition. If you used that battlesight with 1980s or newer rounds, the round’s impact was more like 30 than 60m and you could (and I did) hit yourself with shrapnel from the round. At that range it was not very lethal — it only penetrated about 1/32? — but it was still hot enough to burn your skin.

The Army claims the following for all 40x46mm antipersonnel grenades: kill radius of 5m, and casualty radius of 130m (the latter is greatly exaggerated, in my opinion). During the Vietnam War they claimed a kill radius of 5m and an “incapacitation radius” of 15m for these same rounds like the M381. The M381 went out of service because it was hazardous to fire in forests due to its very short arming distance (as little as 3m). Real problem in Vietnam triple-cap.

So, if a soldier can put a grenade within a meter or two of his target, it sounds like a smaller grenade might be in order.

Jonathan Haidt on the Righteous Mind

Tuesday, January 21st, 2014

I enjoyed Russ Roberts’ recent EconTalk podcast with Jonathan Haidt on The Righteous Mind, but one passage jumped out at me as quite “meta” — when Roberts and Haidt reflexively asserted that slavery was a clear exception to any kind of relativism about political morality:

Let’s look at the people who appear to be victims in this society and if they themselves think they are victims, that’s enough of a reason for us to condemn it. So, African slaves did everything they could to flee; they hated it; there is no reason to think that this was a legitimate moral order that they approved.

As I commented there, many slaves, at least at the time of Abolition, did not see themselves as victims, did not try to flee, and did not see the system as illegitimate, as you can see from the slave narratives collected by the WPA as the last living slaves were in their dotage.

An Underlying Nastiness

Tuesday, January 21st, 2014

Adam Curtis looks back at when British tabloids took on an underlying nastiness, and a willingness to traffic in human misery:

The News of the World was in trouble — it’s circulation was falling. Part of the problem was television, but also its tradition of titillating court reports — randy vicars caught with their trousers down — was feeling tired and out of date. So early in 1960 Sir William Emsley Carr, the alcoholic proprietor of the News of the World appointed a new editor called Stafford Somerfield.

On his first day as editor, Somerfield called his staff together and — as he described it — “pushed the boat out”.

“What the hell are we going to do about the circulation? It’s going down the drain. We want a series of articles that will make their hair curl.”

In a brilliant book about the British Press, the writer Roy Greenslade describes what Somerfield introduced — “two new forms of provocative content: kiss-and-tell memoirs and saucy investigations”

And right away he found the perfect combination of these in Diana Dors.

Somerfield persuaded her to tell the intimate secrets of her life in a series of articles for the News of the World. He had been fascinated by the Yeardye–Hamilton guns-and-sex drama and was convinced there was far more to be mined from her life. To get the story he paid Diana Dors £35,000 which was an extraordinary amount for that time.

But he got what he wanted. He sat Dors down with a journalist who recorded everything — and then, as Dors later plaintively complained, took “all the mucky bits” and wrote the story of a scandalous, violent and seedy life.

In the articles Dors described how Hamilton and her had sex parties, how Hamilton used the two way mirror to watch couples having sex — taped them and then played the tape back to the entire household over breakfast the next day. She also described the violence in their marriage, and Hamilton’s financial scams.

It was a complete humiliation for Diana Dors, and it shocked the nation. The Archbishop of Canterbury described her as “a wanton hussey”. And Tommy Yeardye then joined in — offering other newspapers his stories too.

It worked brilliantly though — the circulation of the News of the World soared. But Greenslade argues that by bringing this provocative new content into journalism, Somerfield had also introduced a new “nastiness” into the popular press.

Journalists have always been cynical and “hard-boiled” in their view of the world — but Greenslade says that underneath the froth of silly headlines there was now in the News of the World, “an underlying nastiness, and a willingness to traffic in human misery”

And he wasn’t the only one to think this. In 1969 Rupert Murdoch bought the News of the World. By now Stafford Somerfield had made the paper an enormous success and Murdoch kept him on. But a year later he sacked him. Murdoch later explained why:

“I sacked the best editor of the News of the World. He was too nasty even for me.”

Anticipatory Shipping

Monday, January 20th, 2014

Amazon knows you so well it wants to ship your next package before you order it:

According to the patent, the packages could wait at the shippers’ hubs or on trucks until an order arrives.

In deciding what to ship, Amazon said it may consider previous orders, product searches, wish lists, shopping-cart contents, returns and even how long an Internet user’s cursor hovers over an item.

Absence of Fanfare, Prolixity, and Self-Promotion

Monday, January 20th, 2014

My recent Torches of Freedom post caught Nick B. Steves’ attention — and what caught my attention from his post was naturally this passage:

Isegoria, who for over a decade (what were YOU doing for The Reaction® in 2003, hmmm?) has been doing the yeoman’s work of highlighting and commentating upon items of interest to the wider reactionary community, put up a post Tuesday Torches of Freedom. He did so with his now familiar utter absence of fanfare, prolixity, and self-promotion.

Pardon the self-promotion — but, while we’re at it, I do enjoy Al Fin‘s description of my site: A long lived eclectic blog with spirit. Huzzah!

Nick B. Steves sees Curtis’s work as a Lefter than Thou critique:

Oh, look at those dirty (big) businesses targeting unsuspecting normal people to buy stuff they don’t need (cigarettes, cars, and clothes). And just look at what those dirty Nazi did with it. And then, dontcha know, the political right gets its grubby paws on individualism and self-actualization to sell Reagan and Thatcher to the unsuspecting rubes. And finally, per impossibile, even the left gets in on the act. Let us collectively shake our heads.

[...]

This isn’t democracy, pleads Adam Curtis. This is tyranny! And the Reactionary fully agrees but wonders, “Well, what did you expect?”

Mikaela Shiffrin’s Swift, if Unplanned, Ascent to World Champion

Monday, January 20th, 2014

Mikaela Shiffrin’s swift, if unplanned, ascent to World Champion hasn’t followed the usual pattern:

Not surprisingly, any predetermined strategy was remarkably elemental and always focused on process, not results. Jeff and Eileen, former college-level racers, believed in basic tenets, like keeping a light race schedule for their children as they loaded up on practice days filled with deliberate, skills-based drills and exercises.

And, they said, it was imperative to keep family close by.

Yet, these were controversial theories in the ski racing community — both when Mikaela was starting out and when she advanced to the World Cup.

Youth coaches were livid when Mikaela, faster than skiers several years older, chose to stay close to home to practice with her family instead of chasing the prestige of distant championships. Several years later, Eileen said, the United States ski team was adamantly against her accompaniment of Mikaela on the World Cup circuit in Europe. Eileen went anyway, and the Shiffrins paid for it for three years.

“Our plan produced the first 17-year-old World Cup champion the U.S. has ever had,” Eileen said. “They should thank us for our \$500,000 donation to the U.S. ski team.”

The message, the Shiffrins insist, is that their approach, which stressed skill development and shunned goal setting, and always involved the family, has been the secret. If there was a secret.

“If you look at it, what we always sought was normalcy,” Eileen said.

Jeff recalled: “These top-level coaches would tell me that Mikaela was just ripping up a racecourse. And I would say: ‘Yeah, I agree, but she’s just 9 years old.’ And they’d say, ‘What are your plans for her?’ And I’d answer: ‘Plans? Well, tomorrow she’s trying out for a part as the angel in the Christmas play.’

[...]

When Taylor did not make a travel soccer team with all his new friends, the reaction of the Shiffrins was emblematic of their approach to almost everything.

“My mom went on Amazon and bought about 10 different World Cup soccer DVDs,” Mikaela said. “And she bought 12-foot-high rebound nets and bungee cords and all these contraptions so we could set up our own soccer complex in the backyard. Every day that summer, we had our own soccer camp for six or eight hours.”

Mikaela recalled that they bought a unicycle because Eileen had read that it was good for balance, which she considered a pivotal athletic skill. The Shiffrin children also learned to juggle to improve their coordination.

“We then started going around our block, which was two miles long, riding the unicycle and juggling at the same time,” Mikaela said. “And if I was doing that, then Taylor would be behind me dribbling a soccer ball as he ran around the block.”

Eileen was confident it would pay off, even as she worried what people were saying about her children.

“You would see the neighbors coming out to watch the Shiffrins going around the block juggling on a unicycle,” Eileen said. “I’m sure they thought we were completely nuts.”

But the next summer, Taylor and Mikaela made their travel soccer teams.

As Eileen recalled: “The coach said to me: ‘What did you do? They’re great now.’ And I said, ‘You don’t really want to know.’ ”

Jeff said: “Some people might call our approach intense. But it’s not, because the motivation is not to be better than other people at something. The motivation comes from a belief that almost anything can be mastered if you’re willing to put in the hours to master it. If you’re going to do something, do it as best as you can.”

[...]

“I believe there is always a faster way to do things,” she said in September after playing tennis with her mother. “Whether it’s learning to hit a backhand in tennis, learning high school chemistry or getting better at ski racing, I really believe with hard work and analytic preparation, you can skip a few steps and find the faster way.”

It is something of a family motto: Be faster.

Daniel Coyle recommends this DIY approach because it is aligned with the way skill actually develops — which is not through splashy public accomplishments.

William Pitt (the Younger)

Monday, January 20th, 2014

Napoleon was not the only young leader of a state in his time:

William Pitt (the Younger) 1759-1806, was Prime Minister of England during most of the years between 1784 and 1805, and thus was France’s, and Napoleon’s, chief opponent. He entered Parliament at age 22 and became PM at 25. How did he manage it so young? His father was a former PM, who did not send the boy to school but reared him to be an orator and politician. Pitt entered Parliament when the Tory ministry in power was disgraced by defeats in America; and the opposition Whigs were split between Old Whigs and a Young Whig faction that Pitt’s father had led. Pitt joined neither, but began by fighting loudly for parliamentary reform, refusing office until the King gave him carte blanche.

He carried through a series of  government reforms, raising the House of Commons above the House of Lords, making the head of the Treasury supreme in government, and sharply reducing electoral corruption; government sinecures were abolished, revenue was rationalized by reforms in taxation and trade duties, finances were put in order, thereby enabling the huge military effort against France. In many respects Pitt paralleled what the French Revolution and Napoleon accomplished organizationally. Here again the “Great Man” emerges when large structural changes, long-proposed, are finally brought about by a super-energetic and impressive leader.

Pitt could impose his will, just as Napoleon could, because he had a clear goal and a sense that his quarreling compatriots could not carry it. Their micro-interactional skills were different; Pitt the master of swaying parliamentary factions by oratory; Napoleon the master of command in war and of holding together all the threads of organization. Both rose rapidly to the top in a political arena riven by multiple conflicts, which they resolved by a strong course of action that pulled others along with them. And both were workaholics, Pitt if anything even more so; he never married, apparently had no sexual interests, and avoided society. He was worn out and died as soon as he left office, aged 47; almost exactly the age that Napoleon was overthrown.

Malthus was Late

Sunday, January 19th, 2014

Malthus was wrong, Matt Ridley says, but Malthus wasn’t wrong, Steve Sailer notes — he was late:

Gregory Clark’s A Farewell to Alms documents, using English public records such as wills from 1200 to 1800, that the English over these 600 years were using Malthus’s 1798 advice to engage in “moral restraint” avant la lettre. The chief mechanism was not exposing babies on mountainsides or whatever, but was simply delaying marriage until a couple could afford the various accoutrements appropriate for their class.

How different in that regard was Jane Austen’s world from today? Some, but the similarities should be obvious.

The average Englishwoman married during these centuries between 24 and 26. In contrast, the average Chinese woman married around 18. Thus, the Chinese population would grow faster, but tended to collapse when good government broke down.

In contrast, most of sub-Saharan Africa didn’t have to worry about Malthusian traps until fairly recently. Population density outside of a few nice highland locations tended to be well below the agriculture capacity of the enormous amount of land. Diseases, competition with co-evolving wild animals (especially elephants, who consumed crops if there weren’t enough people around to drive them off), and lack of fortifications meant that much of Africa tended to be underpopulated. The great African fear was not overpopulation of a region, but of humans dying out all together in an area.

Thus, while European culture tended to encourage sexual restraint, African culture tended to encourage sexual exuberance — a pattern we can still see today in America.

Sailer seems to think that “the very young average age of first marriage for American women in the 1950s compared to previous decades represented a zenith of mass prosperity,” but Malthus noted the same pattern in 1798:

In the United States of America, where the means of subsistence have been more ample, the manners of the people more pure, and consequently the checks to early marriages fewer, than in any of the modern states of Europe, the population has been found to double itself in twenty-five years.

[...]

But the English North American colonies, now the powerful people of the United States of America, made by far the most rapid progress. To the plenty of good land which they possessed in common with the Spanish and Portuguese settlements, they added a greater degree of liberty and equality. Though not without some restrictions on their foreign commerce, they were allowed a perfect liberty of managing their own internal affairs. The political institutions that prevailed were favourable to the alienation and division of property. Lands that were not cultivated by the proprietor within a limited time, were declared grantable to any other person. In Pennsylvania there was no right of primogeniture; and in the provinces of New England, the eldest had only a double share. There were no tythes in any of the States, and scarcely any taxes. And on account of the extreme cheapness of good land, a capital could not be more advantageously employed than in agriculture, which at the same time that it supplies the greatest quantity of healthy work affords much the most valuable produce to the society.

[...]

The population of the thirteen American States before the war, was reckoned at about three millions. Nobody imagines that Great Britain is less populous at present for the emigration of the small parent stock that produced these numbers. On the contrary, a certain degree of emigration is known to be favourable to the population of the mother country. It has been particularly remarked that the two Spanish provinces from which the greatest number of people emigrated to America, became in consequence more populous. Whatever was the original number of British Emigrants that increased so fast in the North American Colonies; let us ask, why does not an equal number produce an equal increase, in the same time, in Great Britain? The great and obvious cause to be assigned, is the want of room and food, or, in other words, misery; and that this is a much more powerful cause even than vice, appears sufficiently evident from the rapidity with which even old States recover the desolations of war, pestilence, or the accidents of nature. They are then for a short time placed a little in the situation of new states; and the effect is always answerable to what might be expected. If the industry of the inhabitants be not destroyed by fear or tyranny, subsistence will soon increase beyond the wants of the reduced numbers; and the invariable consequence will be, that population which before, perhaps, was nearly stationary, will begin immediately to increase.

Serenity Prayer

Sunday, January 19th, 2014

The wisdom of the Serenity Prayer may be timeless, but the prayer itself is rather new:

The best-known form is:

God, grant me the serenity to accept the things I cannot change,
The courage to change the things I can,
And wisdom to know the difference.

Though clearly circulating in oral form earlier, the earliest established date for a written form of the prayer is Niebuhr’s inclusion of it in a sermon in 1943, followed closely by its inclusion in a Federal Council of Churches (FCC) book for army chaplains and servicemen in 1944. Niebuhr himself did not publish the Serenity Prayer until 1951, in one of his magazine columns, although it had previously appeared under his name.

The prayer is cited both by Niebuhr and by Niebuhr’s daughter, Elisabeth Sifton. Sifton thought that he had first written it in 1943, although Niebuhr’s wife wrote in an unpublished memorandum that it had been written in 1941 or ’42, adding that it may have been used in prayers as early as 1934. Niebuhr himself was quoted in the January 1950 Grapevine as saying the prayer “may have been spooking around for years, even centuries, but I don’t think so. I honestly do believe that I wrote it myself.”

In his book Niebuhr recalls that his prayer was circulated by the FCC and later by the United States armed forces. Niebuhr’s versions of the prayer were always printed as a single prose sentence; printings that set out the prayer as three lines of verse modify the author’s original version.

The original, attributed to Niebuhr, is:

God, give me grace to accept with serenity
the things that cannot be changed,
Courage to change the things
which should be changed,
and the Wisdom to distinguish
the one from the other.
Living one day at a time,
Enjoying one moment at a time,
Accepting hardship as a pathway to peace,
Taking, as Jesus did,
This sinful world as it is,
Not as I would have it,
Trusting that You will make all things right,
If I surrender to Your will,
So that I may be reasonably happy in this life,
And supremely happy with You forever in the next.
Amen.

An approximate version (apparently quoted from memory) appears in the “Queries and Answers” column in The New York Times Book Review, July 2, 1950, p. 23, which asks for the author of the quotation; and a reply in the same column in the issue for August 13, 1950, p. 19, where the quotation is attributed to Niebuhr and an unidentified printed text is quoted as follows:

O God and Heavenly Father,
Grant to us the serenity of mind to accept that which cannot be changed; courage to change that which can be changed, and wisdom to know the one from the other, through Jesus Christ our Lord, Amen.

The prayer became more widely known after being brought to the attention of Alcoholics Anonymous in 1941 by an early member.

The prayer has many precursors:

Epictetus wrote: “Make the best use of what is in your power, and take the rest as it happens. Some things are up to us [eph' hêmin] and some things are not up to us. Our opinions are up to us, and our impulses, desires, aversions — in short, whatever is our own doing. Our bodies are not up to us, nor are our possessions, our reputations, or our public offices, or, that is, whatever is not our own doing.”

The 8th-century Indian Buddhist scholar Shantideva of Nalanda University expressed a similar sentiment:

If there’s a remedy when trouble strikes,
What reason is there for dejection?
And if there is no help for it,
What use is there in being glum?

The 11th century Jewish philosopher Solomon ibn Gabirol wrote “And they said: At the head of all understanding — is realizing what is and what cannot be, and the consoling of what is not in our power to change.”

The philosopher W.W. Bartley juxtaposes without comment Niebuhr’s prayer with a Mother Goose rhyme (1695) expressing a similar sentiment:

For every ailment under the sun
There is a remedy, or there is none;
If there be one, try to find it;
If there be none, never mind it.

Friedrich Schiller advocated the first part in 1801:

„Wohl dem Menschen, wenn er gelernt hat, zu ertragen, was er nicht ändern kann, und preiszugeben mit Würde, was er nicht retten kann,” or “Blessed is he, who has learned to bear what he cannot change, and to give up with dignity, what he cannot save.”

The prayer has been variously but incorrectly attributed to, among others, Thomas Aquinas, Cicero, Augustine, Boethius, Marcus Aurelius, Francis of Assisi, Thomas More, and Friedrich Christoph Oetinger (1702–1782).

The Last Gang in Town

Saturday, January 18th, 2014

If you’ve watched any documentaries on the punk-rock scene of the 1970s, then Ian Rubbish’s The Last Gang in Town should feel familiar:

“Heeeeey, policeman, my boot goes in your face!”

(The Clash, by the way, were always the thinking man’s yobs.)

How Netflix Reinvented HR

Saturday, January 18th, 2014

The former chief talent officer at Netflix explains how Netflix reinvented HR:

The first took place in late 2001. Netflix had been growing quickly: We’d reached about 120 employees and had been planning an IPO. But after the dot-com bubble burst and the 9/11 attacks occurred, things changed. It became clear that we needed to put the IPO on hold and lay off a third of our employees. It was brutal. Then, a bit unexpectedly, DVD players became the hot gift that Christmas. By early 2002 our DVD-by-mail subscription business was growing like crazy. Suddenly we had far more work to do, with 30% fewer employees.

One day I was talking with one of our best engineers, an employee I’ll call John. Before the layoffs, he’d managed three engineers, but now he was a one-man department working very long hours. I told John I hoped to hire some help for him soon. His response surprised me. “There’s no rush — I’m happier now,” he said. It turned out that the engineers we’d laid off weren’t spectacular — they were merely adequate. John realized that he’d spent too much time riding herd on them and fixing their mistakes. “I’ve learned that I’d rather work by myself than with subpar performers,” he said. His words echo in my mind whenever I describe the most basic element of Netflix’s talent philosophy: The best thing you can do for employees — a perk better than foosball or free sushi — is hire only “A” players to work alongside them. Excellent colleagues trump everything else.

The second conversation took place in 2002, a few months after our IPO. Laura, our bookkeeper, was bright, hardworking, and creative. She’d been very important to our early growth, having devised a system for accurately tracking movie rentals so that we could pay the correct royalties. But now, as a public company, we needed CPAs and other fully credentialed, deeply experienced accounting professionals — and Laura had only an associate’s degree from a community college. Despite her work ethic, her track record, and the fact that we all really liked her, her skills were no longer adequate. Some of us talked about jury-rigging a new role for her, but we decided that wouldn’t be right.

So I sat down with Laura and explained the situation — and said that in light of her spectacular service, we would give her a spectacular severance package. I’d braced myself for tears or histrionics, but Laura reacted well: She was sad to be leaving but recognized that the generous severance would let her regroup, retrain, and find a new career path. This incident helped us create the other vital element of our talent management philosophy: If we wanted only “A” players on our team, we had to be willing to let go of people whose skills no longer fit, no matter how valuable their contributions had once been. Out of fairness to such people — and, frankly, to help us overcome our discomfort with discharging them — we learned to offer rich severance packages.

Ringo Starr wishes he was a Powerpuff Girl

Friday, January 17th, 2014

The Powerpuff Girls return with a new look, on Monday, January 20 — as this music video featuring Ringo Starr demonstrates:

Not an Ecological Novel

Friday, January 17th, 2014

Dune is not an ecological novel, Norman Spinrad argues:

And though melange is referred to throughout the novel as a “spice” and consumed in small quantities as such, that is not what it really is at all.

What it really is is that which could hardly speak its name in clear in the science fiction of the early 1960s, which explains why the book was such a hard sell to publishers in 1964 and 1965 even with the terminological obfuscation. Which also explains why it became a best-seller after the cultural transformations of 1967 once it was published and why it was one of the engines of those transformations.

Melange is not a fictional “spice.” Melange is a fictional psychedelic drug. Its effects are similar to those of LSD or mescaline or peyote. Only much more powerful.

DUNE, therefore, is not primarily a novel thematically centered on ecology. It is centrally a novel exploring chemically enhanced states of consciousness and their effects not only on individual personality and spirit but on culture.

One of the very first. And, after all these years, still one of the most profound.

Melange, in even small continuous doses, is addictive, turns the sclera of the eyeballs blue, has milder psychedelic effects than LSD, and, like the peyote of the American southwestern desert, an integrated sacrament of the Native American religion, is thoroughly incorporated into the culture and religion of the Fremen.

On the level of the interstellar culture, it is taken in much stronger doses by the Navigators of the Spacing Guild, who use it to attain extreme states of altered consciousness which allow them to pilot starships through a form of hyperspace, turning them into transhuman beings as part of the existential bargain.

The Bene Geserit female adepts use it for more visionary purposes, and dream of creating and/or finding the “Kwisatz Haderach,” a male capable of handling the spice on the highest level, whose consciousness will be freed thereby from conventionally perceived space and time into a kind of Einsteinian four-dimensional viewpoint which will enable him to see “the future” presciently, or, more subtly and profoundly, to surf the geodesics of probability.

Thus Herbert portrays four levels of both the use of psycho-active drugs by a society and the corresponding levels of consciousness. The Fremen incorporate melange as the sacrament of a tribal religion. The Guild Navigators employ it as a pragmatic technological augment. The Bene Gesserit use it in vision quests and mind-melding sessions.

Paul Atreides passes through these three ascending stages on his way to finally employing the drug to achieve the ultimate level, to become the Kwisatz Haderach, the fully Enlightened One, able to view the conventional realm of space and time from the outside, as Einsteinian four-space, a consciousness rendered therefore prescient up to a point, an Enlightenment that turns out to be both a godlike power and a tragic curse.

All this is set in a culture which is anachronistically archaic in a manner which is both rather too familiar and yet interestingly strange.

Stretching disbelief and contorting technological logic by staging swordfights in a space-going technology capable of using atomic weapons and inflicting an improbable monarchical political system upon it for the purpose of setting a pseudo-medieval action-adventure story on alien planets is hardly Frank Herbert’s invention, and these fictional swords-and-spaceships cultures are almost always implicitly Christian and more or less Catholic.

In DUNE too, we have an Emperor and noble vassals and a hierarchical feudal system with a theocratic underpinning. But it is not Catholic or even Christian.

Although the word “Islam” never even appears in the novel and you have to be rather conversant with the real-world referents to get it, the religious template in DUNE is Islamic, not Christian, more Eastern than Western.

The term “Padishah Emperor” certainly points to Herbert’s deliberate decision to do this, since “Shah padi Shah” means “King of Kings” in Farsi, the language of the Islamic Persian Empire.

Nor is it going too far to suppose that the grudge-nursing Fremen, exiled on Arrakis after a long and complex interstellar hegira, are cognates of the minority Shi’ite followers of Ali persecuted and reviled by dominant Sunni cultures.

And the visionary Bene Gesserit have their similarities to the mystic Sufis, Muslims who claim their sect predates Islam, and who emphasize techniques designed to induce direct mystical experience and insight, rather than ritual, rules, or a belief system.

Why Frank Herbert chose Islam as the religious and mystical underpinning of an interstellar culture that is otherwise based on that of medieval, feudal, Catholic Europe, is perhaps beyond the scope of literary analysis, a choice made somewhere in the deep subconscious regions from which artistic creation arises.

However, one can speculate…

While Islam is generally grouped with Judaism and Christianity, the monotheistic religions out of which it arose, there is one fundamental difference between Islam and its direct predecessors.

Judaism began as a tribal religion centrally concerned with the relationship between the history of the Jews and their God and its Bible was written by diverse hands over a long period of time. Christianity converted Judaism into a proselytizing universalist religion based on the story of one transhuman figure, Jesus Christ, its Bible was written in a much shorter period of time in four alternate versions (not unlike Lawrence Durrell’s ALEXANDRIA QUARTET), it is basically a biography of Jesus, and its central concerns are sin, redemption, and morality.

Islam too began as a tribal religion, that of the Arabs, and was transformed into a proselytizing universalist religion, and its holy book, the Koran, is also filled with rules and regulations.

But the Koran, unlike either Testament of the Judeo-Christian Bible, was created by one man, Mohammed, over a very short period of time in historical terms; directly dictated to him by Allah, if you are a believer, and certainly in the throes of some powerful mystical and visionary experience even if you are not, since Mohammed was an illiterate who had never created a literary work before.

Thus Islam, unlike Judaism or Christianity, but like Buddhism, has as its core one man’s mystical and visionary awakening experience. And Mohammed, liked Buddha, made no pretense of being the Godhead, merely (if that can be the word)of directly experiencing it.

The transcendent goal of Christianity is individual immortality in a vaguely described but rather concrete heaven, to be achieved by following the rules. Thus it is basically a religion of morality.

The transcendent goal of Buddhism is the achievement of Nirvana, the ecstatic reintegration of the individual spirit with the universal Godhead from which it arose, to be achieved by meditative techniques. Thus Buddhism is an experiential religion, whose goal is achieving a transhuman state of consciousness.

Islam stands somewhere between. The Koran is as full of moral and legalistic prescriptions as the Bible, but it was written by one man in a state of mystically transcendent consciousness.

And the “heaven” of Islam, salaciously misunderstood by many, including many Muslims, is described as a state of continuous orgasm, which, seen on a mystic level, is a state of transcendent consciousness not unlike the Buddhist Nirvana.

Which perhaps explains why the Sufis, an older and thoroughly experiential religion, aimed entirely at achieving such states by ecstatic dancing, drugs, and other such means of transforming consciousness, could become an aspect of Islam and be generally accepted as such by the mainstream thereof.

And why alcohol, a drug not known for its psychedelic effects, is far more acceptable in Christian cultures than marijuana and hashish, which are far more acceptable in traditional Islamic cultures than alcohol.

Which may explain why Frank Herbert chose to employ Islamic mystical and religious referents in a novel whose central themes are the cultural, psychological, and religions relationships between a psychedelic drug and the societies based upon it, and the stepwise visionary transformation of a young boy’s consciousness by the use thereof into the transcendent consciousness of a “Kwisatz Haderach,” a being so enlightened that in the end he can even perceive the ironic tragedy of his own prescience.

Which certainly goes a long way towards explaining why DUNE could not find a major American publisher, inside the science fiction or in the mainstream, in the early 1960s, before there was anything like the Counterculture it helped to create.

And why it eventually became a long-term best-seller after the evolutionary changes in the consciousness of a generation it did so much to catalyze.

Why Youth Could Rise Then and Not Now

Friday, January 17th, 2014

Napoleon and his generals all rose to positions of power while still quite young:

What makes them seem so young for their rank is our own career structures: a bureaucratic education system extended by credential inflation to take up most of the young years; and the bureaucratic organizations (including the military) where large numbers compete for promotion through an elaborate series of ranks. Ironically, our age of meritocracy is more of a gerontocracy that the pre-bureaucratic era. The exception is young business entrepreneurs in Information Technology (because they don’t wait for credentials), although not in finance and management.