David Foster explains How Not To Do Market Research:
At a used bookstore, I picked up a book called Rethinking Systems Analysis and Design, by Gerald Weinberg. It’s a pretty old book — 1982 — but contains much that is still of value — a lot of it with applicability beyond the information technology field.In his discussion of how to analyze situations, Weinberg tells the story of a suburban commuter railroad. Several suburbanites had written the railroad asking for a minor change in the train schedule. They wanted to be able to go into town in mid-afternoon to spend the evening with spouses or friends. There was a train that already passed the station at 2:30 but did not stop — all they wanted was to have a scheduled stop added for their station. Here is the response they got:
Dear Committee: Than you for your interest in Central Railroad operations. We take seriously our commitment to providing responsive service…In response to your petition, our customer service representative visited the Suburbantown station on three separate days, each time at 2:30 in the afternoon. Although he observed with great care, on none of the three occasions were there any passengers waiting for a southbound train.
We can only conclude that there is no real demand for a southbound stop at 2:30, and must therefore regretfully decline your petition.
This seems unbelievable, but Weinberg claims that it is actually true. And he has more stories along the same lines:
- A systems analyst in a consumer products company heard that some marketing reps in another building might need terminals to access the marketing database (this was before the days of readily-available PCs). He circulated a questionnaire with the question:
How much use do you presently make of the marketing database?Since making use of the marketing database required a 6-block walk to another building, the usage was zero. The analyst concluded that no terminals for the reps was needed.
- Engineers at a computer manufacturing company were asked to improve a new version of the company’s CPU by adding an efficient method for subroutine calls. After two months, the engineers responded that they had studied a sample of existing programs, and hardly any of them used subroutines in situations where efficiency really mattered. The request was declined as “frivolous.”