An ingenious analyst can often discover exciting and brilliant but “obvious” things

Friday, November 3rd, 2017

Most important of all, Techniques of Systems Analysis says, a study that attempts to influence policy should have a convincing comparison of all the relevant alternatives:

The kind of curves that we draw in our study are sometimes not directly to the point. Usually either with the aid of such curves or by some simple-minded consideration one succeeds in designing what he thinks is a reasonable system. The task is then to show (if possible) that, under any reasonable assumptions, the system designed by the analyst is to be preferred to any alternative systems which are being considered.

[...]

It often happens in practice that people think they disagree on recommendations because they disagree on details of performance, when, in fact, one of the contenders could accept the other contender’s assumptions on performance and still prove his case; that is, the analyst can use an a fortiori argument. It is essential for the Systems Analyst to see if he can do this.

[...]

In other words, in making preliminary expository comparisons, we bend over backwards to hurt our system and to help the alternative system. If it turns out after we have done this that we can still say we prefer our system then we are in a superb position to make recommendations.

One reason the above program can so often be carried through successfully is that many of the most successful and exciting analyses have about them a large element of “the Emperor has no clothes.” (For those who have had no childhood: the phrase refers to Andersen’s story, “The Emperor’s New Clothes.”) In such cases it is not surprising that one can go t extraordinary lengths in accepting the assumptions of one’s opponents and still prove his point. In other cases people may think of the problem as being a choice between two alternatives when a clever analyst may be able to make recommendations that in effectiv put him in the position of being able to “eat his cake and have it too.” Possibly because our moralistic culture tends to overlook and deny this possibility an ingenious analyst can often discover exciting and brilliant but “obvious” things.

Sometimes we cannot go all the way in satisfying the B, C, or D enthusiasts. If we concede their presumably exaggerated performance claims for the gadgets they like and slo concede their presumably undue pessimism for the gadgets we recommend, then in fact one or more of their systems may look better. Under these circumstances we have two alternatives. We can see how far we can go along with the opposition and conduct a so-called “break-even” analysis. In such an analysis we find just what assumptions we have to make about the important values in order to make the performance of the two systems the same. We can then simply ask people to judge whether these assumptions are unduly optimistic or pessimistic. Or we can try to make a more convincing case on what are reasonable assumptions. The best approach is generally to use both of the above measures.

[...]

More than any other single thing, the skilled use of a-fortiori and break-even analysis separate the professionals from the amateurs. A good analyst should clearly separate the parameters (assumptions) into two parts; those to which he can afford to give quite pessimistic values and those to which he has to give “reasonable” or breakeven values. The analyst can then point out that all one needs to believe in order to accept his recommendations is these few crucial assumptions. If the audience accepts that assumed values of these crucial parameters as being reasonable or at least in the break-even region, then they must accept the recommendations. If the whole briefing is built around this idea, it is very surprising how even extremely unpalatable conclusions can be brought home.

In order to carry through the above program most analyses should (conceptually) be done in two stages: a first stage to find out what one wants to recommend, and a second stage that generates the kind of information that makes the recommendations convincing even to a hostile and disbelieving, but intelligent audience.

Leave a Reply