Analysis and Estimates:

It is important to keep in mind that the intelligence communities of Germany, France, Russia and Israel were as convinced as those of the UK and USA that Iraq had appreciable CBW holdings; the main dispute with Germany, France and Russia was about the appropriateness of military action as a response to those holdings. Independent think tanks such as the International Institute for Strategic Studies had reached similar conclusions. There is an old saying that absence of evidence is not evidence of absence, and Ephraim Halevy has argued, what analysis believed they were looking at was not evidence of absence but, rather, the tip of a well hidden iceberg (Halevy, 2004).

The assessment failure we appear to see in Iraq is not one without precedent. Estimating a country's strengths in a particular type of weapon is notoriously difficult, and the history of intelligence in the last century is littered with examples of damaging overestimates of enemy strength, as well as underestimates. Prior to the Second World War, the UK badly overestimated the strength of the German Luftwaffe and German aircraft production capability. At the turn of the 1960s the 'missile gap' controversy was driven by a radical overestimate of Soviet ballistic missile strength while in the latter years of the Cold War arms reduction talks were hampered by the fact that estimates of Soviet chemical and biological weapons (CBW) stockpiles were wrong by a factor of ten (and sometimes more). Of course, in the case of both Germany and the missile gap, the Nazis and Kruschev's Soviets went to considerable pains to deceive allied intelligence into those estimates as a form of deterrent or delaying tactics. German officials consciously invited an RAF officer, F. W. Winterbotham, to witness air trials and demonstrations. Winterbotham was, at the time, head of SIS's Air Section. Whether or not his Nazi hosts knew this is unclear, but they certainly saw and used him as a conduit through which to influence British perceptions of their warfighting capability. Likewise, with various menacing gestures from Kruschev' 'We will bury you' to series of atmospheric nuclear tests, the Soviet Union tried hard to create an impression of invulnerability. That image was one that hawks in the US defence community were only too happy promote within the Washington beltway to make the case of increased resourcing of their own military programmes. By comparison, the USSR of the 1980s tried desperately and unsuccessfully to refute allied worst case estimates that were the product mainly of sincere error and politically-coloured assumptions about what the USSR was likely to do. So estimating weapons capabilities is a notoriously treacherous and uncertain branch of intelligence estimates. And like Kruschev, Saddam Hussein had real, geopolitical incentives to bluff about the credibility of his WMD deterrent.

Part of the tragedy and irony of the Iraq situation is that many of those involved in assessing Iraqi WMD were alumni of the Soviet estimates. These included figures like Dr. David Kelly who had been amongst the inspectors who went around Soviet facilities after 1991 and saw what the actual situation had been. As a result, they should have been well familiar with the errors made assessing Russian CBW stockpiles. However, it appears that the same kinds of mistakes were made about Iraq as had been made over Germany. The ISC inquiry into intelligence about Iraqi WMD revealed that JIC reports during September 2002 warned that they were dealing with radically incomplete information about a highly unpredictable Saddam Hussein and that, therefore, much of the assessment was forced to rely on 'judgement' rather than evidence. The internal, intelligence community assessments were explicit that while they had evidence of Iraq pursuing programmes they did not have any sense of what scale those programmes were operating on or whether or not there existed substantial stockpiles of weapons or 'weaponisable' chemical and biological agents. Thus, much as the overestimate of German and Soviet capabilities was based on ultimately political (and in a narrow sense ideological) assumptions about how Nazi Germany or the Soviet Union were inclined to act, so similar assumptions appear to have shaped estimates of Saddam Hussein's WMD programmes.

The use of the 45 minute claim is significant in terms of how we look at the assessment process also. As noted above, it was employed without supporting additional sources of information. It was used only once and in a minimally significant role. In earlier drafts of the JIC assessment of 9 September (upon which the September dossier was based) SIS and the Defence Intelligence Staff originally qualified the report as probably referring to battlefield chemical weapons held at forward facilities. This was an inference because apparently the bare wording of the source report did not state this explicitly. However, as the wording of the report was tightened up, this inference was dropped precisely because it was viewed as too inferential. As we shall see below, the unqualified version of the 45 minute report was carried over into the published dossier. The problem at hand regarding analysis is somewhat different. The problem in analytical terms is that the 45 minute report was recognised by the assessors in the JIC as fragmentary information, it was unsupported by additional evidence, and it was supplied by a solitary human source who, while apparently of proven value in the past, appears to have been a sub-agent of a dissident group that will certainly have had its own agenda for cooperating with SIS.

It has been suggested by some intelligence insiders that a noticeable change in JIC drafting procedures in the last decade has been an increased tendency for the members to make sure that their individual contributions to assessments are noted in finished JIC reports (private information). In part this appears to be a sort of understated competition for credibility, an attitude that says 'if there are points to be scored let's make sure they go on the right scoreboard'. But in part it also has the value of allowing consumers to see what proportion of their information is coming from human and technical sources, and it permits the analysts to show how and why they weight the evidence in hand to lead to the conclusions reached. The determination of the JIC's chairman John Scarlett, a former senior SIS officer, to include it in the September 9 assessment despite its isolation and uncertainty, necessarily invites questions about the real motivation for the report's inclusion, that is, in the service of comprehensive reporting or departmental bureaucratic manoeuvring.

As noted above, the process of making intelligence estimates is always a difficult one as it is almost always a decision made under conditions of incomplete information. The pioneer of 'bounded rationality' theory, economist Herbert Simon, has argued that all organisations make their decisions under conditions of incomplete information almost all of the time, but in intelligence the boundedness of that rationality is more pronounced than in most other spheres of human activity. As noted above, intelligence is not about acquiring complete information; the purpose of intelligence is to acquire fragments of information where otherwise there would be none. One almost never has the intelligence equivalent of a smoking gun, as many commentators have observed. The question in assessing the fragments one acquires is what lies behind them. Those fragments are employed as indicators of an otherwise concealed process, like the clustering of iron filings in a magnetic field or the surface wake of a dived submarine. One must always ask 'is this all there is?' or 'is this the 10% tip of the iceberg?' The step from fragmentary indicators to what they indicate is always going to be a judgement call. We have already seen that the US and UK have both overestimated enemy capabilities in the past, but they have also made comparably serious underestimates. During the 1970s, CIA estimates of Soviet ICBM developments proved conservatively low because CIA made its guess on perceived resource limitations in the Soviet economy and industrial capacity (Graham 1979). It later transpired that the USSR had consciously expanded its strategic nuclear arsenal at the expense of other aspects of the nation's industrial development. CIA underestimated the political will of the Soviet leadership (and the political bargaining strength of the Red Army which controlled the USSR's strategic rocket forces). As a result political judgement about an opponent can be vital to making accurate estimates rather than serving only as a body of bias and preconceived notions.

What we see, therefore, in the assessment process of Iraqi WMD is a process in which the JIC collectively seems to have developed a consensus judgement and a confidence in that consensus judgement that may not have been adequately supported by the raw intelligence upon which the assessment was based. Even when Dr. Brian Jones and some of his colleagues dissented about aspects of the drafting of the September Dossier, their concerns involved matters of detailed phraseology rather than the essential conclusions being reached. And, as the ISC inquiry reported, once they had raised those doubts with senior officials, those doubts were considered and then put aside in a fashion entirely consistent with national assessment procedures. It appears evident that even before the JIC was tasked to 'cherry-pick' its own intelligence database to help the government's campaign of public political persuasion its membership had concluded that a substantial threat did indeed exist. That conclusion, it now appears, was probably weakly supported by the raw intelligence available, and as pointed out above, at some point political judgement of the target is a necessary and inevitable step in drawing the fragmentary indicators together into a meaningful pattern. The point where difficulty arises is when that judgement involves force the fragments into a pattern they do not really fit. That the JIC built its conclusions upon a particular judgement about Saddam Hussein's regime is not a failure as such. At a certain level, such judgement represents a gamble that must be taken. The question that Lord Butler must ask is whether it the gamble taken was truly justified by the indicators available.

Another significant consideration is not only the actual estimates made but the degree of confidence with which they are made. At various points in recent years, the US intelligence community has tried to quantify this question of relative uncertainty by assigning percentages to particular estimates. This has since been proved a spurious form of precision, but it does embody a crucial factor in the assessment of intelligence. At certain times, indicators may be found that can only point to a single possibility, but these are rare. Most intelligence pictures are liable to multiple, alternative interpretations. For the most part, estimates are made with varying degrees of uncertainty. One might ask how accurate or inaccurate an estimate that is out by a factor of ten but expressed with ninety percent uncertainty really is. This is a typical problem for the consumers of intelligence. But that problem carries with it an additional question: at what level of certainty is it appropriate to take action? One might make a diplomatic initiative without too grave a risk or cost if one is in error. But if one proposes to take military action, what kind of uncertainty threshold is necessary to warrant violating national sovereignty, shedding blood and sacrificing lives. This is the crucial question to ask because, whatever the agreed assessments made by the JIC concerning Iraq, they were consistently couched in terms of very high insecurity. It is questionable where any UK government afford to take the risk of going to war on the basis of such uncertainty, let alone whether it could credibly justify that action to its public, to Parliament or the international community.