ATP 2-33.4, page 120

Appendix A A-16 ATP 2-33.4 18 August 2014 Example of Analytical Effectiveness of Analysis of Competing Hypotheses The analytical effectiveness of Analysis of Competing Hypotheses (ACH) becomes apparent when considering the Indian nuclear weapons testing in 1998. According to Admiral Jeremiah, the intelligence community had reported: “There was no indication the Indians would test in the near term.” Such a conclusion by the community would fail to distinguish an unproven hypothesis from a disproved hypothesis. The intelligence communities’ conclusion that because the Indians were not testing nuclear weapons now does not disprove the hypothesis India may test nuclear weapons in the future. If the ACH procedure had been used, one of the hypotheses would certainly have been that India is planning to test in the near term but will conceal preparations for the testing to forestall international pressure to halt such preparations. Careful consideration of this alternative hypothesis would have required evaluating India’s motive, opportunity, and means for concealing its intention until it was too late for the United States and others to intervene. It would also have required assessing the ability of U.S. intelligence to see through Indian denial and deception if it were being employed. It is hard to imagine that this would not have elevated awareness of the possibility of successful Indian deception. A-85. A principal lesson in the above scenario is whenever an intelligence analyst is tempted to write the phrase, "there is no evidence that ...," the analyst should ask this question: If this hypothesis is true, can I realistically expect to see evidence of it? In other words, if India were planning nuclear tests while deliberately concealing its intentions, could the analyst realistically expect to see evidence of test planning? The ACH procedure leads the analyst to identify and face these kinds of questions. A-86. Once an analyst has gained practice in applying ACH, it is quite possible to integrate the basic concepts of this procedure into the normal analytical thought process. In that case, the entire eight-step procedure may be unnecessary except on highly controversial issues. A-87. There is no guarantee that ACH or any other procedure will produce a correct answer. The result still depends on fallible intuitive judgment applied to incomplete and ambiguous information. ACH does, however, guarantee an appropriate process of analysis. This procedure leads the analyst through a rational, systematic process that avoids some common analytical pitfalls. It increases the odds of getting the right answer, and it leaves an audit trail showing the evidence used in the analysis and how this evidence was interpreted. If others disagree with the analyst’s judgment, the matrix can be used to highlight the precise area of disagreement. Subsequent discussion can then focus productively on the ultimate source of the differences. A-88. A common experience is that ACH attributes greater likelihood to alternative hypotheses than would conventional analysis. People become less confident of what they thought they knew. In focusing more attention on alternative explanations, the procedure brings out the full uncertainty inherent in any situation that is poor in data but rich in possibilities. Although such uncertainty is frustrating, it may be an accurate reflection of the true situation. The ACH procedure has the offsetting advantage of focusing attention on the few items of critical evidence that cause the uncertainty or which, if they were available, would alleviate it. This can guide future collection, research, and analysis to resolve the uncertainty and produce a more accurate judgment. APPLYING THEORY A-89. Applying theory begins with the formulation of a theory based on the evaluation of other examples of the same phenomenon. This technique is based on the supposition that when a given set of conditions arise, certain other conditions will follow. One of the advantages of this methodology when applied to intelligence analysis is that it economizes thought. By identifying the key elements of a problem, applying theory enables an analyst to sort through a mass of less significant detail. Applying theory enables the analyst to see beyond transient developments, to recognize which trends are superficial and which are