[ad_1]

Art of Accident Investigation

Introduction

Most accident investigation authorities impose time and resource constraints, and we can all think of cases where the resulting pressure on investigators has caused poor quality reports. At the same time, our work is increasingly subject to critical review, both in the Press and in the Courts. We need a way to argue effectively for the time and resources we need, so that we can do our jobs properly. If at the same time we can work in such a way that our work can stand up to critical inspection, so much the better. One way to argue against constraints on investigations is to have a recognised formal methodology. If the investigation is only half done, we can then show that this is so, and demonstrate the consequences of producing a half baked report. A further advantage of using a scientific approach to investigations is the possibility of built in quality control: by following well-established procedures, researchers can be confident of producing sound work. I’m sure we can all think of investigations where some element of quality has been lacking. Inadequate data gathering, incorrect manipulation of data and unsound logic in analysis are examples of the ways things can go wrong. Larkin’s review of the Cali report (Gerdsmeier et al., 1997) has demonstrated that not even the highly respected NTSB National Transportation Safety Board  is immune to faulty logic. Deficiencies in reports matter, because the real aim of investigations is to reduce accident rates by making effective safety recommendations. If we don’t solve the accident properly, our recommendations are likely to be ineffective. But even if we are satisfied that our investigation is sound, it will still be a failure if the recommendations are not adopted. Recommendations involve change: they are disruptive, make work, and cost money. Those with the power to implement them will therefore seek ways to avoid doing so. If the report is open to challenge on any ground – not all the data was gathered, the logic is defective, alternative propositions were not canvassed – then it will be unpersuasive, and inaction is likely to prevail. Frank Taylor’s recent paper (Taylor, 1998) shows that such inaction is the norm: it would be fair to say that failed investigations are also the norm, because those who could act are not persuaded of the need to do so.

Scientific Methods

Scientific methods may offer the possibility of better quality investigations, with more rigorously argued analysis and more persuasive recommendations in consequence, but you could argue that we already make much use of scientific disciplines. We use aerodynamics, stress analysis, materials science, meteorology, aviation medicine and psychology. Even sociology has now become respectable! Why should we need yet more science? There is another way of looking at science, and that is by the type of methodology used. Each of the methodologies – survey, archival, historical, experimental and case study – may be appropriate to any of the disciplines in particular circumstances: it depends on what you are trying to find out. For example, a medical researcher investigating a rare disease might examine the case of an individual patient; he would probably want information from previous cases, and so would study them in archives, and he might try promising new drugs – experimental research. Let us see how each of these methodologies might apply to accident investigation. Surveys are seldom used, though they could have a place in discovering management factors which may have contributed to an accident. Archival research is sometimes used, when investigators search back through the files seeking some particular information from large numbers of accidents. When O’Hare et al. read through many reports to try to establish what . Such analysis led to the discovery of the MU-2 body icing phenomenon.

The MU-2 was a small, twin turbine-engine aircraft used for commuter airline and charter work. Over the years there had been a series of accidents (at least 19) which were characterised by loss of control in bad weather. One possibility mooted was that water was getting into the autopilot computer, and producing unanticipated control inputs, but the accidents were never satisfactorily resolved. After two accidents in rapid succession, BASI performed a special study in which twelve such accidents were reviewed, together with a large number of incident reports. It was concluded that accumulation of body ice, for which there was no provision for removal, could produce a very rapid reduction in airspeed, and also a stalling speed much higher than normal. The circumstances of almost all the reports, for which sufficient information was available, were compatible with this conclusion. It was recommended that clearance for flight in known icing conditions be withdrawn. But again, this is not the main line of work.Experimental work, too, has a part in accident investigation. The Comet pressure tank testingwas experimental: The Comet was the first jet airliner to enter service. Exactly one year after the first service flight, one broke up in a thunderstorm; the accident report concluded that turbulence caused the airframe to be overstressed. Shortly afterwards, another broke up in clear air near the island of Elba, in the Mediterranean Sea. Recovery of the wreckage was difficult, because the depth of water was at the limit for the equipment of the day. The BOAC Comet fleet was closely inspected, and many improvements were made, but almost immediately after service was resumed, a third aircraft was lost, over very deep water. The fleet was again grounded. Efforts to recover the wreckage of the second aircraft were increased, and the wreckage recovered suggested that there might be a problem with metal fatigue. An aircraft from the fleet was impressed for experimental purposes. A special tank was built within which the pressure hull of the aircraft could be contained. The wings were left protruding, and attached to these were levers which could impose loads representative of those experienced in flight. The tank was filled with water, so that if failure occurred when the interior of the fuselage was pressurised, the resulting explosion would be damped. The cyclic loads could be imposed at a rate such that the equivalent of thousands of flight hours could be achieved in a short time. Fatigue failure of the cabin did occur in the experimental aircraft. The damage patterns were sufficiently similar to those found among the wreckage from the Elba Comet that it was concluded that there was a design defect (the use of square cut-outs in a pressure vessel) which made the aircraft vulnerable to fatigue. This defect was rectified, and the aircraft saw many years of trouble-free service (RAE, 1954).Experimental methodology is also used when investigators replicate a flight to see forthemselves whether there was a visual illusion, or what handling problems the pilot may haveencountered.All of these methodologies therefore have a place in the training of accident investigators.However, each contributes to only a small part of the task. It remains to consider whethercase study research describes the work of accident investigators. Let us start by examiningwhat investigators do.

The Accident Investigators’ Tasks

We start with the on-site investigation. We make an initial survey and photograph the scene. The wreckage trail is plotted, and ground scars documented. Then we examine the wreckage in greater detail – are the extremities present? Can control system continuity be established? Were other systems apparently normal? If possible, we document the cockpit control positions, and the positions of control surfaces at impact, and so on. In the case of an accident to a large airliner, the protocols to be followed are detailed in the Manual of Accident Investigation (ICAO, 1970). Witness interviewing, document retrieval and detailed examination of the wreckage follow. In all of this phase, the work can be characterised as data gathering and documentation. There are protocols for each part: how to plot the wreckage trail, witness interviewing techniques and so on. While this phase is in progress, the news media will be demanding to know what caused the accident, and whom to blame. Of course, we tell them that the analysis cannot be started until all the data have been gathered. Of course this is untrue, but it serves to get them off our backs. In the first place, it is untrue because that is not the way the human mind works. The mind seeks to join bits of information together to make sense of them. At quite an early stage, some parts of the puzzle will become clear. It will be possible to characterise the impact as steep or shallow angle, and high or low energy. If the aircraft started to break up before impact, this will soon be known. The answers to these and other questions will give rise to possible sequences of events, and so guide the search for supporting or rebutting information. At the same time, we must be aware of the danger of the ‘glimpse of the blindingly obvious’. The thing that ‘obviously’ caused the accident may, in reality, have had nothing to do with it. Basic data should still continue to be gathered, and alternative explanations sought. Another reason that analysis starts before all the data is available is that an unguided search of the mass of documentation associated with an aircraft and its crew is likely to be fruitless. We need to be guided by some positive line of inquiry. If a mechanical problem seems likely, the airframe logbooks may have useful information. If the pilot may have been fatigued, crew flight and duty time records are likely to be relevant. In other words, the search for data is guided by some theoretical propositions, which the data may support or rebut. (There is a distinction between hypotheses and propositions. Hypotheses will be tested statistically, by examining a sufficient number of samples to support or reject the hypothesis with given confidence. A proposition, on the other hand, is a possibility arising from some theory, which we may or may not find that our data fits). Let me give you an example:

I was examining the wreckage of an aircraft which had broken up in mid-air, and found pre-existing damage that spanned a break in the spar. The damage was such that it would have reduced the strength of the spar in bending, and the spar had, in fact, bent before finally fracturing. A proposition to be examined was that the weakened spar had bent under loading arising from air turbulence, and the bend led to aerodynamic effects which led to the break-up of the aircraft. This proposition was rebutted, by showing that the loads actually experienced by the structure were the opposite of those which would have resulted from the bending of the spar. An alternative explanation therefore had to be formulated, and data sought to support or reject it in turn. The aircraft had in fact developed flutter, and the spar broke at that particular point because it had already been weakened there. (TAIC, 1992).Data analysis seems to be the phase that causes us the greatest difficulty. Little guidance isavailable. The ICAO report format (ICAO, 1994) is silent on the matter, as it is primarilyconcerned with how to write the report rather than how to go about the analysis. Certainly,the results are not uniformly acceptable, e.g. the critique of the Cali report (Gerdsmeier et al.,1997). The Munich accident in 1958, in which an Airspeed Ambassador failed to becomeairborne after an attempted take-off from a slush-covered runway, caused controversy foryears (RAE, 1964; Stewart, 1986). Even today, the conflicting Erebus reports (OAAI, 1980;Mahon, 1981) are debated. It would be fair to say that while we analyse data and writereports, we do not always do these tasks very well. Guidance in these areas would certainlybe useful.To summarise the aircraft accident investigators’ tasks, we first seek to describe the accident,and then answer the questions how and why it happened. Finally, we try to persuade others totake action to avert future accidents.

Case Study Research

Dr Yin, formerly of the RAND Corporation, defined a case study as “an empirical inquiry that investigates a contemporary phenomenon within its real-life context” (Yin, 1994, p. 13). The field of inquiry is very broad. A medical study of an individual is a case study; so is an examination of the effects of Government policy. Case study research is the appropriate methodology “when ‘how’ or ‘why’ questions are being posed, when the investigator has little or no control over behavioural events, and when the focus is on a contemporary phenomenon within some real-life context” (ibid. p. 1). The definition and criteria embrace the field of accident investigation. In a case study, the researcher gathers data from a number of sources, such as documentation, interviews, direct observation and physical artefacts. When possible, corroborative evidence is sought. A database is kept, so that the evidence is available for subsequent review. The parallel with accident investigation procedures is evident. Prior to analysis, the data may be manipulated in a number of ways. Evidence may be placed in a matrix of categories, and graphical displays such as flow charts may be used. Accident investigators place witness evidence in a matrix, so that apparent inconsistencies may be elucidated, and flow charts have been advocated in accident investigation (e.g. Benner (1994); Johnson (1994); Zotov, (1996); Ladkin, (1999)). The researcher’s initial objective may be descriptive, or to examine theoretical propositions in the light of the evidence. There are four dominant analytical techniques in case study research:

 Pattern matching

 Explanation building

 Time series analysis

 Program logic models.

Pattern matching involves comparing an empirically based pattern with one or more predicted patterns. Explanation building is a special case of pattern matching. The final explanation is not fully stipulated at the beginning of the study. The data is analysed by building an explanation about the case, i.e. by stipulating a set of causal links. This should be a primary function of an aircraft accident investigation: see, for example, Benner, (1994); Ladkin, (1999). Explanation building is iterative, and is best done with multiple cases such as the analysis of the series of MU-2 accidents (BASI, 1992). An initial proposition is compared with the findings of the case, and the proposition is revised as required. Then, if another case is available, its details are compared with the revised version, and if necessary the proposition is further revised. This process can be repeated with the facts of a second, third and subsequent cases. It is important to entertain rival explanations (see, for example, the criticism of the Mil-8 accident report(TAIC, 1993; Zotov, 1995)). Yin cautions that this approach requires intelligent investigators, but as accident investigators are intelligent, that would be no handicap! Time series analysis may be used on its own, or in conjunction with other techniques. A match is sought between data points and some theoretically significant trend specified a priori, or a rival trend, or any trend based on a threat to internal validity (i.e. some other causal effect than the one we are considering). For example, when the Police claim that introduction of a new speed enforcement regime has reduced road accidents, one should test to see if the road accident rate had been interrupted at the time of introduction, or whether the trend remained the same as before. Chronologies are a special form of time series analysis. Arraying events into a chronology

allows the analysis of causal events over time, since cause must precede effect. This technique allows for the consideration of many variables. (The flow-charting method known as Multilinear Event Sequencing (Benner, 1994) is an example of this technique). The aim is to compare the chronology with that predicted by some explanatory theory. The theory specifies the conditions. Examples are:

Control surface flutter as a consequence of high speed: some events must occur before others, and the reverse is impossible.

 Helicopter crash after main rotor stall: some events must always be followed by others.

 Time for a wing spar to burn through after engine firewall penetration by fire: some events can only follow others after a (specified) time-lapse.

 In-flight fire patterns compared with post-crash fires: some time periods may contain classes of events different from those of other time periods.

“If the actual events of a case study… have followed one predicted sequence of events and not those of a compelling rival, the single-case study can [be] the initial basis for causal inferences” (Yin, 1994, p. 117). Program logic models are a combination of pattern matching and time series analysis. They can be used where a policy was intended to produce some outcome, and the intervention produced intermediate outcomes which came together to produce the final result.

Advantages of Formal Methodology

There is a clear parallel between air accident investigation, and case study research. The investigator and the researcher do the same things, to achieve the same ends: See Fig. 2. It would be reasonable to regard accident investigation as a particular field of case study research. But, if air accident investigators are already doing case study research, what benefit is there in giving investigation another name? The best investigators already do, intuitively, what case study researchers do. They gather data, form descriptions, postulate alternative explanations, and prove or disprove these. An important advantage of having a formal methodology is that it can be taught, and so the many may be brought up to the standard of the best. Also, it permits ‘the best’ to be standardised, and ‘the rest’ to be evaluated against those criteria. A further advantage is that quality control can be built in. At the design stage of the study (for investigation purposes, at the completion of the on-site phase), research questions can be formulated, and the design can be examined with standard tests for validity:

 Are correct operational measures being used for the concepts being studied, as opposed to subjective judgements? (Construct validity). For example, it is important to distinguish between what the pilot was trying to do, and what we think he ought to have been doing.

 Have spurious effects been avoided, e.g. by consideration of rival explanations? (Internal validity)

Has the domain to which the findings can be generalised, been established? (External validity). For example, we may be able to generalise findings relating to a runway overrun at one aerodrome, to a number of aerodromes.

Can operations (e.g. data collection) be repeated with the same results? (Reliability). Procedures need to be documented. Again, the best investigators undoubtedly do this, but by making it a formal step, quality control will be intrinsic. Case study methodology is particularly suited to dealing with clusters of accidents, such as the MU-2 accidents analysed by BASI. In the past, it appears that the underlying causal factors have generally been discovered by individual intuition. There will always be a place for intuition, but it would be desirable to have formal methods which could be applied by all.

CASE STUDY ACCIDENT INVESTIGATION

Function Appropriate methodology when ‘how’ or ‘why’ questions to be answered Describe the accident, and discover how and why it happened Data gathering phase Observation Interviews Documentation review Artefacts On-site investigation Witness interviews Documentation review Detailed examination of the wreckage Data manipulation Evidence matrix Graphical displays, e.g. flow charts Witness matrix Logic flow diagrams, e.g. Multilinear Event Sequencing Theoretical basis Theoretical propositions to be supported or rebutted by evidence Theoretical propositions guide search for further data to support or rebut them Techniques Explanation Building Time series analysis (e.g.) Why-Because Analysis Multilinear Event Sequencing

Accident Investigation as a field of Case Study Research

Investigators besides, such formal methods should enable underlying causes to be discovered as soon as possible. There may be insufficient evidence from any one accident to discover the core problem which led to it, but as Benner has argued (personal correspondence) it is unethical to wait while a string of accidents occurs before they are analysed successfully. Finally, the use of formal methodology could make a contribution in the area of persuasion. Writing reports and safety recommendations is not easy. If the investigation has been unstructured, it may be difficult to write in such a way that the need for corrective action is seen to be compelling. Conversely, a structured investigation, whose design has been validated a priori, should lead to the development of a report which is not only logically sound, but which is generalised. That is to say, because the investigation leads to the testing and substantiation of theoretical propositions, it is possible to generalise from the particular accident to other occurrences of the same type. This should help to counter the frequently heard responses like “It was a one-off”, or “It hasn’t happened here yet”.

Teaching formal methodology

Potential accident investigators are required to have a wide aviation background. Minima of an Air Transport Pilot Licence and 3000 hours as pilot in command are commonly required for pilots. Engineers are usually required to be at least a Chartered Engineer with flying experience. A military background is often preferred, since there is no better training in commanding the large numbers of personnel and the highly expensive resources that may be needed on an accident site (Zotov, 1997). Trainee investigators, then, are very knowledgeable about aviation, but may have had little formal scientific training. If we conclude that accident investigation can be regarded as an area of case study research, it might well be that we should teach the methodology of case study research first, before imparting the specialist skills required. It is accepted, among professions such as law and medicine, that academic and technical theory should be integrated into practical training (Hunt, 1997), and more recently this concept has been applied to the training of pilots. It should also have a place in the training of investigators. We can then impart methods of investigation, rather than teach facts; when the time comes, the investigator will then be able to apply the knowledge that is available. In teaching case study research methods, we would naturally use applications and illustrative examples drawn from past investigations. Thus, when we discuss data manipulation, we could demonstrate the use of data arrays by taking real witness evidence and placing it in a matrix to show the usefulness of this technique. Instead of the witness matrix being one of a grab-bag of ideas, it then takes its place among the various useful manipulations available. Likewise, when discussing the various quality control measures available, the deficiencies of existing accident reports can be examined using the concepts of validity and reliability. If, for example, internal validity is always considered prior to data analysis, the error of failing to consider alternative explanations (as in the Mil-8 report (TAIC, 1993)) should never arise.

Conclusion

It has been shown that air accident investigations have many of the characteristics of case study research, and in the author’s view, such investigation could be regarded as a field of case study research. What investigations currently lack is a formal methodology. Applying standard research methodology should demonstrate the need for sufficient time and resources to complete the investigation properly and also improve the overall quality of investigations. By making the reports and recommendations more persuasive, it should enable the investigations to better serve their purpose of reducing the aircraft accident rate.

[ad_2]

Source by Jamal Barki