A Tale of Two Bell Curves

Monday, March 27th, 2017

Bo and Ben Winegard tell a tale of two Bell Curves:

To paraphrase Mark Twain, an infamous book is one that people castigate but do not read. Perhaps no modern work better fits this description than The Bell Curve by political scientist Charles Murray and the late psychologist Richard J. Herrnstein. Published in 1994, the book is a sprawling (872 pages) but surprisingly entertaining analysis of the increasing importance of cognitive ability in the United States.

[...]

There are two versions of The Bell Curve. The first is a disgusting and bigoted fraud. The second is a judicious but provocative look at intelligence and its increasing importance in the United States. The first is a fiction. And the second is the real Bell Curve. Because many, if not most, of the pundits who assailed The Bell Curve have not bothered to read it, the fictitious Bell Curve has thrived and continues to inspire furious denunciations. We have suggested that almost all of the proposals of The Bell Curve are plausible. Of course, it is possible that some are incorrect. But we will only know which ones if people responsibly engage the real Bell Curve instead of castigating a caricature.

How to Gain New Skills

Friday, March 24th, 2017

In his How to Gain New Skills guide for students, Ulrich Boser (Learn Better) discusses an experiment that took place years ago at a Catholic all-girls school in New York City:

As part of the experiment, the girls were taught how to play darts for the first time, and the two psychologists conducting the study divided the young women into some groups. Let’s call members of the first group “Team Performance,” and they were told that they should learn the game of darts by trying to throw the darts as close to the center of the board as possible. In other words, the researchers informed the women that the best way to win was to rack up some points.

The psychologists also pulled together another group of young women. Let’s call them “Team Learning Method,” and they learned to play darts very differently. The researchers had these girls focus on the process of gaining expertise, and the women started by focusing on how exactly to throw the darts, mastering some basic processes like “keep your arm close to your body.” Then, after the women showed some proficiency, they were encouraged to aim at the bull’s eye, slowly shifting from some process goals to some outcome goals like hitting the target.

Finally, there was the control group. Their instructions? The researchers told them to learn to “do their best.” In other words, these young women could take any approach that they wanted to learning darts. Let’s think of this group as “Team Conventional Wisdom.”

To learn more about the experiment, I met up with Anastasia Kitsantas, who ran the study together with psychologist Barry Zimmerman. While the experiment took place some years ago, Kitsantas still has the darts stashed away in her office at George Mason University, and on a rainy afternoon, she pulled out the little yellow missiles from an office cabinet to show them to me, laying the darts out like an important relic from some forgotten South American tribe.

Kitsantas held onto the darts because of the study’s surprisingly large outcomes, and by the end of the experiment, the young women on Team Learning Method dramatically outperformed the others, with scores nearly twice as high as Team Conventional Wisdom. The women also enjoyed the experience much more. “Several of the students asked me to teach them more about darts after the experiment. They kept asking me for weeks,” Kitsantas told me.

The best basketball player in the world is not the tallest

Thursday, March 23rd, 2017

Even a strong predictor of outcome is seldom able to pick out the very top performer, Stephen Hsu notes — e.g., taller people are on average better at basketball, but the best player in the world is not the tallest:

This seems like a trivial point (as are most things, when explained clearly), however, it still eludes the vast majority. For example, in the Atlantic article I linked to in the earlier post Creative Minds, the neuroscientist professor who studies creative genius misunderstands the implications of the Terman study. She repeats the common claim that Terman’s study fails to support the importance of high cognitive ability to “genius”-level achievement: none of the Termites won a Nobel prize, whereas Shockley and Alvarez, who narrowly missed the (verbally loaded) Stanford-Binet cut for the study, each won for work in experimental physics. But luck, drive, creativity, and other factors, all at least somewhat independent of intelligence, influence success in science. Combine this with the fact that there are exponentially more people a bit below the Terman cut than above it, and Terman’s results do little more than confirm that cognitive ability is positively but not perfectly correlated with creative output.

Strong Predictor Graph

In the SMPY study probability of having published a literary work or earned a patent was increasing with ability even within the top 1%. The “IQ over 120 doesn’t matter” meme falls apart if one measures individual likelihood of success, as opposed to the total number of individuals at, e.g., IQ 120 vs IQ 145, who have achieved some milestone. The base population of the former is 100 times that of the latter!

Cloaks, Daggers, and Dice

Thursday, March 16th, 2017

South by Southwest included a talk called Cloaks, Daggers, and Dice, which examined how the CIA uses games:

In “Collection,” Clopper’s first CIA game, teams of analysts work together to solve international crises against a ticking clock. His second title, “Collection Deck,” is a Pokémon-like card game in which where each card represents either an intelligence collection strategy or a hurdle like red tape or bureaucracy.

[...]

Also speaking on the panel was Volko Ruhnke, who is an intelligence educator at the CIA and a freelance game designer. Ruhnke said he is particularly interested in one type of game: a simulation tabletop game to train analysts and help with analytic tasks. It could help forecast complex situations by forcing players to handle multiple scenarios simultaneously.
Ruhnke himself created a commercial board game to simulate the Afghanistan conflict and walk players through military, political, and economic issues in the region. It gives players “a much more dynamic understanding of the issues of modern Afghanistan,” Ruhnke said, adding that a similar game could be of use internally at the CIA as well.

Volko Ruhnke is famous — in the wargaming community — for designing the card-driven wargames Labyrinth: The War on Terror, 2001–? and Wilderness War. He was also the original designer of GMT Games’ COIN series, which includes Cuba Libre, A Distant Plain, and Liberty or Death: American Insurrection.

Homeowners’ Quest for the Best Schools

Tuesday, March 14th, 2017

Parents will move to a new neighborhood and pay a huge premium to live near a good school:

For some home buyers, there is no factor more important than the public schools their children will attend. They analyze student-body performance on standardized tests, school rankings, what percentage of alumni go on to four-year colleges and which schools send students to Ivy League or top-tier state universities. They then uproot their lives to move within these districts’ boundaries, where homes can cost hundreds of thousands of dollars more than nearby homes zoned to different schools.

In La Cañada Flintridge, Calif., a city in Los Angeles county, the acclaimed La Cañada Unified School District determines the real-estate market, agents say. “I’m very busy in March, when the private-school rejection letters go out,” said Anne Sanborn, a real-estate agent with Sotheby’s International Realty in Pasadena. When parents find out their kids haven’t been accepted at elite private schools, they start house hunting in La Cañada, Ms. Sanborn said.

Ms. Sanborn added that “there is a mass exodus from La Cañada when their kids graduate high school,” as families sell their homes and seek neighborhoods closer to downtown Los Angeles or Pasadena.

Parents don’t seem to care which way the causality runs:

Online tools that measure student performance have made it easier for home buyers and agents to assess schools across the country.

For example, GreatSchools, an Oakland, Calif., nonprofit, rates schools based primarily on how well students perform on statewide assessments and has provided rankings to real-estate websites Zillow, Trulia, Move and Realtor, said Weezie Hough, director of strategic partnerships.

In an analysis of 1.6 million home listings in the U.S. through the first six months of 2016, Realtor.com found that houses in public-school districts with GreatSchools ratings of 9 or 10, the highest scores possible, were priced, on average, 77% higher than homes in nearby districts with scores of 6 or lower. Additionally, homes located in top districts sell four days faster — at 58 days — than the national median of 62 days, the analysis found.

Dismal Voucher Results Surprise Researchers

Sunday, March 12th, 2017

Three new studies show surprisingly bad results from school vouchers:

The first results came in late 2015. Researchers examined an Indiana voucher program that had quickly grown to serve tens of thousands of students under Mike Pence, then the state’s governor. “In mathematics,” they found, “voucher students who transfer to private schools experienced significant losses in achievement.” They also saw no improvement in reading.

The next results came a few months later, in February, when researchers published a major study of Louisiana’s voucher program. Students in the program were predominantly black and from low-income families, and they came from public schools that had received poor ratings from the state department of education, based on test scores. For private schools receiving more applicants than they could enroll, the law required that they admit students via lottery, which allowed the researchers to compare lottery winners with those who stayed in public school.

They found large negative results in both reading and math. Public elementary school students who started at the 50th percentile in math and then used a voucher to transfer to a private school dropped to the 26th percentile in a single year. Results were somewhat better in the second year, but were still well below the starting point.

This is very unusual. When people try to improve education, sometimes they succeed and sometimes they fail. The successes usually register as modest improvements, while the failures generally have no effect at all. It’s rare to see efforts to improve test scores having the opposite result. Martin West, a professor at the Harvard Graduate School of Education, calls the negative effects in Louisiana “as large as any I’ve seen in the literature” — not just compared with other voucher studies, but in the history of American education research.

There’s always the chance that a single study, no matter how well designed, is an outlier. Studies of older voucher programs in Milwaukee and elsewhere have generally produced mixed results, sometimes finding modest improvements in test scores, but only for some subjects and student groups. Until about a year ago, however, few if any studies had shown vouchers causing test scores to decline drastically.

In June, a third voucher study was released by the Thomas B. Fordham Institute, a conservative think tank and proponent of school choice. The study, which was financed by the pro-voucher Walton Family Foundation, focused on a large voucher program in Ohio. “Students who use vouchers to attend private schools have fared worse academically compared to their closely matched peers attending public schools,” the researchers found. Once again, results were worse in math.

If the voucher programs are new, and all the existing private schools are aimed at (slightly) better-than-average students, perhaps the schools are just a terrible fit.

Can critical thinking actually be taught?

Monday, February 27th, 2017

Can critical thinking actually be taught?

Decades of cognitive research point to a disappointing answer: not really. People who have sought to teach critical thinking have assumed that it is a skill, like riding a bicycle, and that, like other skills, once you learn it, you can apply it in any situation. Research from cognitive science shows that thinking is not that sort of skill. The processes of thinking are intertwined with the content of thought (that is, domain knowledge). Thus, if you remind a student to “look at an issue from multiple perspectives” often enough, he will learn that he ought to do so, but if he doesn’t know much about an issue, he can’t think about it from multiple perspectives. You can teach students maxims about how they ought to think, but without background knowledge and practice, they probably will not be able to implement the advice they memorize. Just as it makes no sense to try to teach factual content without giving students opportunities to practice using it, it also makes no sense to try to teach critical thinking devoid of factual content.

We should spend less

Thursday, February 16th, 2017

Arnold Kling shares what he believes about education:

1. The U.S. leads the world in health care spending per person, but not in health care outcomes. Many people look at that and say that health care costs too much in the U.S., and we should be able to get the same our better outcomes by sending less. Maybe that is correct, maybe not. That is not the point here. But —

2. the U.S. leads the world in K-12 education spending per student, but not in student outcomes. Yet nobody says that education costs too much and that we should spend less. Except —

3. me. I believe that we spend way too much on K-12 education.

4. We spend as much as we do on education in part because it is a sacred cow. We want to show that we care about children. (Yes, “showing that you care” is also Robin Hanson’s explanation for health care spending.)

[...]

8. I do not expect educational outcomes to be any better under a voucher system. That is because I believe in the Null Hypothesis, which is that educational interventions do not make a difference.

9. However, a competitive market in education would drive down costs, so that the U.S. would get the same outcomes with much less spending.

You always have to have a plan B

Thursday, February 9th, 2017

Everybody fails, but not everybody responds to failure the same way, Mike Riggs notes, as he interviews Megan McArdle about The Up Side of Down:

Mike: You use the writing profession as an example of this.

Megan: You have to accept that being bad is part of learning to write. Most people who end up approaching professional writer status were always better at it than other kids. Then they get into the professional landscape and realize everyone else in the industry was also better at it than the other kids. This can be very traumatic for a lot of writers, and I’ve seen some of them just freeze. They don’t turn stuff in because as long as they haven’t turned it in, it’s not bad yet.

How do you hack that thinking? You say to yourself, “Look, I can rewrite garbage, I can’t rewrite nothing.”

Mike: It’s the iteration paradox. You miss 100% of the shots you don’t take, but you also miss a ton of the shots you do take when you’re first starting out. You have to do a thing over and over to get good at it, while somehow dealing with the fact that it’s really embarrassing and discomfiting to try hard at something and still be bad.

Megan: And the only way around that is to accept that failure is an essential part of the process.

You are not supposed to sit down and be Proust on your first pass. Proust wasn’t even Proust on the first pass. That means you have to see doing something badly as better than not doing anything at all. I won’t get fired for handing in 1,000 bad words. I will definitely get fired for not handing in anything.

After that, the next step is learning to recognize where and why you’re bad without rolling around on the floor, saying, “This is terrible, I’m obviously the world’s worst writer.” And you do that by looking at your bad work as a dipstick that measures where you can improve rather than one that measures your innate talents.

Mike: This speaks to the idea that learning how to do something new is good for you even if it doesn’t necessarily turn into a career.

Megan: We learn by doing stuff not well. That’s how people learn to play tennis. You don’t become good at it by creating a really elaborate theory of tennis ball physics, or else MIT would win Wimbledon every year. You hit a ball, you try to guess where it will go. It doesn’t go where you expect and then on the 100th time you finally hit it right. By hitting it wrong all those times, you learn to hit it right.

If you’ve never done anything you weren’t good at, you can’t learn the valuable skill of sucking at something but continuing to do it, which is how people get good at anything. And we have to make ourselves do it because doing something you aren’t good at is usually less rewarding than things that come more easily.

[...]

Mike Riggs: It seems like the best way to hedge against that kind of collapse at the institutional level is to be as diversified as possible at a personal level. Try things that are difficult, save as much as you can, contribute to a 401k. But even that is hard for lots of people.

Megan McArdle: The fact is you can’t assume nothing bad will happen. You could get hit by a truck tomorrow. Your company could go under. We should prepare for failure, which is why I always tell my readers to save 20% of their gross income. As you can imagine, this is not a popular suggestion with my readers.

I also advise people to have a year’s worth of expenses in an emergency fund. This was viewed, even by financial advisors, as quite conservative. But I spent two years being unemployed after getting what was supposed to be the golden ticket to a guaranteed job, which was an MBA from a top-five school. And that taught me there’s no such thing as a golden ticket. You always have to have a plan B. You always have to be thinking about what you’ll do if your company fails. Where will you go next? You should be maintaining connections in that industry, but you should also be living below your means. You should have a smaller mortgage than what you can afford. You should have more savings than you really need.

If you end up dying of cancer at the age of 40, you’ll have over-saved. But if you die of cancer at the age of 40, your biggest regret is not going to be that you didn’t spend more money while you were healthy. Your biggest regret is going to be about relationships and the people you didn’t call, so call your mother.

Introductory psychology textbooks lean left

Wednesday, February 8th, 2017

Introductory psychology textbooks lean left:

Writing in Current Psychology, Christopher Ferguson at Stetson University and his colleagues at Texas A&M International University conclude that intro textbooks often have difficulty covering controversial topics with care, and that whether intentionally or not, they are frequently presenting students with a liberal-leaning, over-simplified perspective, as well propagating or failing to challenge myths and urban legends.

[...]

Ferguson and his team examined textbook coverage of seven areas of research consisting of findings which might be considered particularly appealing or unappealing to textbook authors with liberal leanings, and/or which could be prone to alarmist interpretation. This included research on whether media violence incites aggression; the stereotype threat (the notion that performance differences between groups are exaggerated by the fear of conforming to stereotypes); the narcissism epidemic (the idea that today’s youth are more narcissistic than youth in the past); that smacking/spanking children leads to aggression and other negative outcomes; that there are multiple intelligences; that human behaviour is explained by evolutionary theories related to mate selection and sexual competition (in this case, the authors assumed liberal authors would prefer not to cover this research); and controversy around antidepressant medication.

The researchers looked to see if textbook authors presented the evidence as more definitive than it is in these areas, or only presented one side of the arguments. They found that there was biased treatment of media violence and stereotype threat by half or more of the books, and of multiple intelligences and spanking by a third. A quarter of books failed to deal with controversy around antidepressants. Evolutionary theories were neglected by a fifth of the books and presented in biased fashion by one quarter. “We believe that these errors are consistent with an indoctrination, however intentional, into certain beliefs or hypotheses that may be ‘dear’ to a socio-politically homogenous psychological community,” Ferguson and his colleagues said.

They also looked at textbook treatment of various psychology myths and urban legends, including the frequently exaggerated story of the murder of Kitty Genovese, which is often cited as a perfect example of the “bystander effect”: our reduced likelihood of intervening to help when in the company of a greater number of other people who could help. Nearly half the books perpetuated the myth that 33 witnesses watched the killing of Genovese without doing anything to help her. Meanwhile, nearly three quarters of the books failed to challenge the popular misconception that we only use ten per cent of our brains, or that listening to Mozart makes us smarter. And 70 per cent of the books gave the French neurologist Paul Broca undue credit for localising speech function in the brain: the researchers say that the theory of the cortical localisation of speech was first put forward by Ernest Auburtin. “It is surprising to see so few textbooks addressing common misconceptions about psychology,” they said.

[...]

After all, in recent years, we’ve also covered research by Richard Griggs at Florida State University that’s found biased textbook treatment of Milgram’s classic studies on obedience, outdated accounts of the story of Phineas Gage, biased coverage of Asch’s studies of conformity, and of Zimbardo’s Stanford Prison Experiment. Psychology students: if you’re looking for a rounded and accurate introduction to the field , you could consider supplementing your textbook reading with regular visits to our Research Digest blog. Or maybe you do that already.

People just give up trying to improve

Thursday, February 2nd, 2017

Anders Ericsson — of deliberate practice fame — began his career helping to push the boundaries of working memory:

Most people can repeat back a seven-digit phone number, but not a ten-digit one. He recruited Steve Faloon, an average Carnegie Mellon University student, and they set about systematically working to get better. After about 200 hours of effort, Faloon could repeat back 82 digits, by far a world record at the time. Faloon wasn’t destined for such greatness. Rather, Ericsson’s takeaway is that performance has no inherent limit. “Instead, I’ve found that people more often just give up and stop trying to improve,” he writes. Work constantly at the edge of your ability, though, and your brain changes in a way that makes better performance possible.

They need to learn about the world

Sunday, January 29th, 2017

I haven’t seen Captain Fantastic, but Bryan Caplan’s favorite scene from the movie amused me:

Subtle it’s not, but for me, awesome always beats subtle. The stage: Homeschooling dad Captain Fantastic and his six kids are visiting his mundane sister and her two kids (Justin and Jackson). The sister lets her brother know she’s not too happy with his child-rearing…

Sister: They’re children! They need to go to school. They need to learn about the world.

Captain: [shouting] Justin. Jackson? Would you please come down here for a second?

Jackson: What?

Captain: How old are you now, Jackson?

Jackson: Thirteen.

Captain: Can you tell me what the Bill of Rights is?

Jackson: Um, what something costs, I guess.

Captain: That’s a good guess. Justin, you’re in high school?

Justin: Yeah.

Captain: Do you like your school?

Justin: It’s whatever.

Captain: Do you know what the Bill of Rights is?

Justin: It’s a government thing, right? Like, rights that people have in America and stuff.

Captain: Yep. [shouting] Hey, Zaja?

Zaja: [Captain's 2nd-youngest kid] Yes?

Captain: Would you please come down here a moment, sweetie? I wanted to ask you a quick question. Zaja’s just turned eight, by the way. The Bill of Rights.

Zaja: Amendment one: Congress shall make no law respecting an establishment of religion, or prohibiting the free exercise thereof; Or abridging the freedom of…

Captain: Stop. Regurgitating memorized amendments isn’t what I’m asking for. Just tell me something about it in your own words.

Zaja: Without the Bill of Rights we’d be more like China. Here, at least, we don’t have warrantless searches. We have free speech. Citizens are protected from cruel and unusual punishments…

Sister: That’s enough.

The very bottom 14% have very simple skills

Saturday, January 28th, 2017

If someone you know doubts intelligence differences, James Thompson says, show them the functional literacy data that Linda Gottfredson references:

That is right. Only 4% of the white population can do all the tasks in the list.

NALS Levels

21% get to the 4th level but cannot do carpet cost type problems, and at the very bottom 14% have very simple skills, which do not include locating an intersection on a street map. For many of you reading this, the finding will seem incredible. It is incredible. Human differences are hard to believe, but they are matters to be demonstrated, beliefs notwithstanding.

The mistake isn’t not guessing right

Friday, January 27th, 2017

I don’t follow association football (soccer), but a (fairly) recent Guardian piece noted that Shrewsbury’s Mat Sadler was about to face his old under-17 teammate Wayne Rooney, of Manchester United, until Rooney got injured, and this was especially interesting because so few players from that young men’s team made it in the big leagues:

Only five of the 18 members of an England squad who finished third in that tournament in Denmark are still playing professional football, with several slipping into the non-league scene, such as the former Nottingham Forest midfielder Ross Gardner, who now turns out for West Auckland Town and works for British Gas, while others have walked away from the game altogether.

[...]

Sadler’s story started at Birmingham City, where the left-back made his Premier League debut at the age of 17 and was extremely well-regarded, so much so that when the Football Association’s technical department organised a “Player Audit” in 2003, his name was one of 25 considered as “certainties” for full England honours.

Fascinated by a list he was never aware of until now, Sadler scans through the names of which only seven — Jermaine Jenas, Michael Carrick, Aaron Lennon, Glen Johnson, Michael Dawson, David Bentley and James Milner — vindicated the FA’s judgment. “There are a few that did get there but more that didn’t. It’s nice company to keep, though,” Sadler says, smiling. “I might frame that.”

Doug Lemov points out that present skill is easier to spot than future skill — or talent:

We think we see the future but we don’t. Learning curves, physiological growth curves, attitudes, health, commitment, psychology — they are all too unpredictable. The mistake isn’t not guessing right. It’s betting too heavily on the guess.

Grouping athletes or students by achievement level only works if the grouping is fluid, if it’s constantly changing and responding to progress.

Allocating dwellings, schools, and health facilities without regard to social class had no effect

Thursday, January 26th, 2017

The city of Warsaw was razed at the end of World War II and rebuilt under a socialist government. As this 1978 Science paper notes, the socialist government’s policy of allocating dwellings, schools, and health facilities without regard to social class had no effect on the association of social and family factors with cognitive development:

Of the 14,238 children born in 1963 and living in Warsaw, 96 percent were given the Raven’s Progressive Matrices Test and an arithmetic and a vocabulary test in March to June of 1974. Information was collected on the families of the children, and on characteristics of schools and city districts. Parental occupation and education were used to form a family factor, and the district data were collapsed into two factors, one relating to social marginality, and the other to distance from city center. Analysis showed that the initial assumption of even distribution of family, school, and district attributes was reasonable. Mental performance was unrelated either to school or district factors; it was related to parental occupation and education in a strong and regular gradient. It is concluded that an egalitarian social policy executed over a generation failed to override the association of social and family factors with cognitive development that is characteristic of more traditional industrial societies.