Tyler Cowen kicked off his talk with Bryan Caplan by citing Caplan’s own statement that “no single paper is that good”:
What I meant by that is that if you look at any individual piece, in social science specifically, it’s very hard to see that a reasonable person would fundamentally change their mind based upon any one of them.
People often have an idea of, there’s the really good papers where you should have a mind quake, and you never see the world again in the same way after that. For me, all of them fail to measure up to that standard. I think the way that you really learn something is by reading a vast empirical literature.
The direct cause of this was… I think Noah Smith had a challenge: “Name the two or three papers on each topic that are really convincing.” I was thinking about that and said, “Honestly, I can’t think of any papers like that unless you’re going to cheat and count a literature view as being that kind of a paper.” Just realizing that the way that you actually achieve social science knowledge isn’t by finding the one crucial — might be a natural experiment that shows exactly how the world works — but by assembling a wide variety of evidence and then muddling through.
And that’s the problem across a really massive swathe of our society: What I call the “Blockbuster Mentality”.
Every movie or TV show has to be the biggest, the best, the most popular… Nobody wants to set out to make a small, intimate entertainment that serves as a workaday small-time work; every book has to be a bestseller.
It’s not just the entertainment industry, either–The military procurement realm is horribly prone to this, as well. It’s not enough to design, test, and trial a small incremental improvement to currently issued weapons, they’ve got to achieve “100% improvements in lethality”, whatever the hell that’s supposed to be.
Real progress in any field is not based off of one single world-shaking paradigm-shifting single thing. Instead, it’s built up out of a thousand little things that add up in significance until the world finally does change. Problem is, everyone is looking for that blockbuster, and we’ve ceased doing the minor things of true significance, because the payoffs are too low…
And, oddly enough, Elon Musk is actually demonstrating this with SpaceX–Look at the iterative evolutionary approach he’s taking, vs. the insane literal “moon shot” that NASA is taking with their project–Which ain’t even left the ground once, yet…
I’d actually like to see new movie, TV show, food or beverage product, or particularly flavored diet Coke variety advertised with that magic phrase, “now with 100% improvements in lethality”.
I nominate 3/4 of the new diet Coke flavors for that title right now. The mango version tastes tantalizingly like artificial mango for a few sips and then goes full cough medicine. Good times.
This mindset that anything worthy has to be a “paradigm-shift” does seem to have infected everything, and it’s odd to see Musk, correctly I think, identified here as now being a countervailing example.
Anyone here read Cdr Salamander’s naval blog? He and his audience have worked over the navy’s LCS pretty hard for years, admittedly to the point that most related comments sections read like one is watching an argument among old combatants rehashed many times, but I enjoy every time someone uses the term “transformationalists” as a term of abuse. That mentality does seem to have wrought havoc in the armed services of many nations, but the US in the lead. Many of the things foretold by network centric warfare/network enabled operations/4GW/RIMA et al. have come to pass in that incremental way Kirk cites, but exclusive emphasis on any one of these paradigm shifters has led down many blind alleys for 25 years.
Graham,
There’s a guy writing about software development who talks about a lot of the same stuff, and I think that the points he makes there about doing incremental evolutionary design make just as much sense in software development as they do in rocket science… I wish I could remember who the hell it is, and where I read it, but the sad fact iis that I read it in passing, and only remember nodding my head along with him, going “Uh-huh… Yeah… That’s just like it is in the military, too…”.
I think I may have saved that piece somewhere, but it’s on a disused hard drive, and I’m not even sure where. Gentleman made some very cogent points, and while I don’t think he meant to, I saw some real parallels to what he was saying to what I’ve seen in life.
1. In economics,
Coase’s “Theory of the Firm,” ca. 1937.
2. in African studies / political science,
Jackson and Rosberg’s “Why Africa’s weak states persist: The real and the juridical in statehood.” ca. 1981?
I can think of a few papers and/ book chapters that stretched my mind to the point where it never went to the old position. So here is a third:
3. James Q. Wilson: “Against the legalization of drugs” ca. 1981. Perhaps originally in commentary, but I’m not sure.
= – = – = – =
But in terms of papers generally, I’m not a good reader of papers.
Kirk,
Interesting to see that argument applied to software design.
As a user, I have always appreciated it- I have to do a rage check whenever Microsoft decides to do away with an accustomed feature or make something a 2 step process which long was 1. And so on. Incremental rules.
I don’t know about the coding side. I wonder what if any relationship there is between incremental change in features and the oft-cited problem of programs built on decades of jury-rigged code. I’d like to assume one can have the former while cleaning up the latter a bit.
A lie often told becomes the truth…?
Here’s one I think is very important.
https://en.wikipedia.org/wiki/Domesticated_red_fox
Kirk,”…The military procurement realm is horribly prone to this, as well. It’s not enough to design, test, and trial a small incremental improvement to currently issued weapons, they’ve got to achieve “100% improvements in lethality”…”
I think we’re falling behind because of this.