ATP 2-33.4, page 118

Appendix A A-14 ATP 2-33.4 18 August 2014 identify all the reasonable alternatives and evaluate them against each rather than evaluate their plausibility one at a time. A-82. The way most analysts begin analysis is to pick out what they suspect intuitively is the most likely answer, then look at the available information from the point of view of whether or not it supports this answer. If the evidence seems to support the favorite hypothesis, analysts become confidant in their decision and look no further. If it does not, they either reject the evidence as misleading or develop another hypothesis and go through the same procedure again. Intelligence analysts call this a “satisficing” strategy. Satisficing means picking the first solution that seems satisfactory, rather than going through all the possibilities to identify the very best solution. There may be several seemingly satisfactory solutions, but there is only one best solution. The principal concern here is that if analysts focus mainly on trying to confirm one hypothesis they think is probably true, they can easily be led astray when there is so much evidence to support their point of view. They fail to recognize that most of this evidence is also consistent with other explanations or conclusions, and these other alternatives have not been explored. The Method A-83. Simultaneous evaluation of multiple, competing hypotheses is difficult to do. To retain three to five or even seven hypotheses in working memory and note how each item of information fits into each hypothesis is beyond the mental capabilities of most people. It takes far greater mental agility than listing evidence supporting a single hypothesis that was pre-judged as the most likely answer. It can be accomplished, though, with the help of the simple procedures discussed here:  Step 1. Identify the possible hypotheses to be considered. Use a group of analysts with different perspectives to brainstorm the possibilities. Psychological research into how people go about generating hypotheses shows that people are actually rather poor at thinking of all the possibilities. If individuals do not even generate the correct hypothesis for consideration, obviously they will not get the correct answer.  Step 2. Make a list of significant evidence and arguments for and against each hypothesis. In assembling the list of relevant evidence and arguments, these terms should be interpreted broadly. They refer to all the factors that have an impact on judgments about the hypotheses. Do not limit yourself to concrete evidence in the current intelligence reporting. Also include your own assumptions or logical deductions about another people or groups or country's intentions, goals, or standard procedures. These assumptions may generate strong preconceptions as to which hypothesis is most likely. Such assumptions often drive the final judgment, so it is important to include them in the list of evidence.  Step 3. Prepare a matrix with hypotheses across the top and evidence down the side. Analyze the diagnosticity of the evidence and arguments; that is, identify which items are most helpful in judging the relative likelihood of alternative hypotheses. Step 3 is perhaps the most important element of this analytical procedure. It is also the step that differs most from the natural, intuitive approach to analysis, and, therefore, the step an analyst is most likely to overlook or misunderstand. The procedure for step 3 is to take the hypotheses from step 1 and the evidence and arguments from step 2 and put this information into a matrix format, with the hypotheses across the top and evidence and arguments down the side. This gives an overview of all the significant components of the analytical problem.  Step 4. Refine the matrix. Reconsider the hypotheses and delete evidence and arguments that have no diagnostic value. The exact wording of the hypotheses is obviously critical to the conclusions one can draw from the analysis. By this point, you will have seen how the evidence breaks out under each hypothesis, and it will often be appropriate to reconsider and reword the hypotheses. For example:  Are there hypotheses that need to be added or finer distinctions that need to be made in order to consider all the significant alternatives?  If there is little or no evidence that helps distinguish between two hypotheses, should they be combined into one?  Step 5. Draw tentative conclusions about the relative likelihood of each hypothesis. Proceed by trying to disprove hypotheses rather than prove them. In step 3, you worked across the