The CRM Advocate

Issue 96.2

Published Quarterly for the Professional Air Crew Trainer

April, 1996

Table of Contents

CRM for CRM: Cockpit Relevant Movies for Crew Resource Management 

Marvin Karlins, Ph.D., Professor of Management, University of South Florida
Captain Freddie Koh, Assistant Director of Flight Operations, Singapore Airlines
Captain Len McCully, Assistant Director of Flight Operations, Singapore Airlines
Captain C. T. Chan, Deputy Chief Pilot - A340 Fleet, Singapore Airlines


Aviation-relevant films (videos) can be useful pedagogical tools in the CRM context if pilots perceive them to be interesting, informative and relevant to the topic(s) under consideration. Films which meet these criteria are identified and reviewed, along with some suggestions for enhancing their training value. By alerting the aviation community to this select group of films it hoped that (a) their use in CRM programs will increase; and (b) airlines will save time and money by not "reinventing the wheel:" developing "in-house" productions when similar products are already commercially available.

If it is true that a picture is worth a thousand words, then imagine how many hours of CRM lectures could be condensed with a few well-chosen video presentations! In fact, the use of films in CRM programs produces significant benefits beyond time conservation: used appropriately, these films can encourage process (creative) thinking (Schroder, Karlins & Phares, 1973), while providing a more stimulating, realistic portrayal of the subject-matter under discussion (Karlins, Koh, McCully & Chan, 1996). Further, the use of movies seems particularly suited to the learning needs of adult audiences (Bedker, 1995); the kind of audiences found in CRM seminars.

Unfortunately, not all films enhance learning: many are so aesthetically impoverished, boring and/or "off target" they actually impede training, particularly when used to educate professional audiences about complex topics like effective flight deck management. What, then, constitutes a "good" CRM film? Based on the opinions of Singapore Airlines Pilots (expressed in seminar evaluation surveys administered at the end of a four-day CRM program), such a film would be:

  1. interesting (e.g., good animation, cinematography, acting, dialogue);
  2. informative (subject-matter deemed worthwhile to learn);
  3. accurate (events and problems are correctly portrayed and identified);
  4. relevant to the topic(s) under consideration (there is a clear linkage between the film and what is being discussed in the seminar); and
  5. thought-provoking (encourages post-film discussion and insights helpful to the enhancement of flight deck performance and job satisfaction).

Armed with the knowledge of what aviators wanted in a CRM training film, the authors undertook an exhaustive search to locate suitable cinematic products for use in future seminars. Films were solicited from airlines, commercial film production companies, consulting firms, US governmental agencies and television stations. From these sources, 243 films were identified and reviewed (full-length feature movies were not included in the review process). Based on this initial examination, 56 "finalists" were selected as

worthy of further consideration. A second review pared this number down to 20 "winners" -- the films we have recommended in this article. Please note that although the films are numbered for identification purposes, these numbers should not be construed as a means of ranking the selections in any way (e.g., film #1 is not "better" than film #10) All the films that made the final list are, in the authors' opinion, of superior quality and should enhance the learning process as long as they fit in with the particular CRM program or topic for which they were intended.

Recommended Films for CRM Programs:

(1) The Dryden Accident.
[Original copyright 1992, Australian Airlines.
Currently owned by Qantas Airways Limited.
Address: Flight Training Centre, Qantas Jet
Base (Ml 48), Mascot NSW 2020, Australia.
Attn: CRM Coordinator.1
Running time: 18 min. 40 sec.

A compelling look at the events leading up to the crash of Air Ontario Flight 363 on March 10, 1989. The film is particularly valuable in the CRV context, as it emphasizes what can happen when there is a lack of teamwork and effective communication between cockpit and cabin crew. Also important is the film's focus on the chain of events that culminated in the tragedy. This allows the viewer the opportunity to see how an accident develops and, hopefully, how to recognize and deal with such conditions should they develop on one of his/her flights. Finally, the documentary describes number of factors (e.g., flight delay, equipment failure, company pressure, flight time limitations, desire to get home) that may have stressed the pilot and affected his judgment. Aviators viewing the video will most likely recognize some or all of those factors as having occurred during their own flights. Keeping "The Dryden Accident" in mind is a good way for pilots to remember the potential dangers associated with such factors and the importance of keeping them in proper perspective (safety first!) should they occur in future line operations.

(2) What the Pilot Didn't Know.
[ABC News, 20-20, October 30, 1992.
Address: ABC News, Film and Tape Practices
47 West 66th Street
New York, NY 10023-6290.J
Running time: 13 minutes.

To intensify the impact and reinforce the message of The Dry den Accident, follow it up with this documentary. The program describes the final moments of USAir Flight 405 which crashed on take-off from La Guardia Airport on March 22, 1992. One factor that makes the USAir crash particularly disturbing is its striking similarity to the Dryden tragedy ... right down to the type of aircraft flown, the cause of the crash and the kind of interaction between cabin staff and cockpit crew before the accident. After viewing The Dryden Accident and What the Pilot didn't Know the view cannot help but be reminded of two critical aviation axioms: (1) good crew coordination and teamwork is vital in maximizing flight safety; and (2) those who do not learn from the lessons of history are bound to repeat them.

There is another ABC news broadcast devoted to USAir Flight 405 which was aired May 14, 1992 on Prime-Time Live. This program focuses on the human dimension in the crash, interweaving personal accounts of crash survivors with a recreation of the accident scenario (produced by combining film simulation with actual newsreel footage of the accident). A computer graphic portraying the takeoff roll is also included (fifteen seconds). The film is very realistic and provides a definite human interest impact to the Air Ontario and USAir tragedies... if such an emphasis is desired by the CRM coordinator. In addition, there are segments of the film (e.g., the computer simulation of the runway accident) that might be useful for pilot audiences. In general, however, this tape is probably better-suited for viewing by the general public, preferably not as part of an in-flight entertainment package.

(3) Delta 191.
[Z-Axis. Address: 116 Inverness Drive East,
Suite 110, Englewood, CO 80112.]
Running Time: Tape #1(54 mm. 30 sec.),
Tape #2 (38 mm. 40 sec.).

Z-Axis is a corporation that produces state-of-the-art computer animation to re-create events for use in courtroom trials. One such event was the crash of Delta 191 at Dallas-Fort Worth International Airport on August 2, 1985. So effective was this new computer-driven re-creation of the crash that it made the lead story in the December 1989 edition of the American Bar Association Journal (Marcotte, 1989). As a prime in the study of how a weather-related accident unfolds, this product has no equal. CRM coordinators and training pilots might want to study both tapes in their entirety (90+ minutes), although

for training purposes a much shorter presentation suggested. To capture the impact of the tapes -- and also the lessons to be learned -- the authors' recommend using the following sections from Tape #1 only: (a) Text (first 2:30 minutes of tape). An animated presentation of the descent profile (from 2,100 fi to impact) with cockpit and ATC communication (scripted and voice recording). (b) All data (beginning at minute 8:14 and running through minute 11:26). Instrument read-outs are provided: wind vector, knots, control column, AOA EPR and airspeed. CRM participants have audibly gasped as they watch the instruments change when the aircraft encounters the violent microburst moments before the crash. Other sections of the two tapes can be utilized at the discretion of the CRM coordinator, based on what (and how much) information he/she wishes to impart concerning the accident. Some coordinators might also want to provide seminar participants with some background understanding of the Delta 191 crash before showing the Z-Axis simulations. If so, the authors recommend use of the film footage beginning at minute 41:45 of Why Planes Crash (see selection #6).

A final note: Because the US Department of Justice paid for the production of this film, the authors were informed that this video is in the public domain, thus free of charge. Please contact the senior author should you desire a copy of this tape.

(4) Northwest 255.
[Z-Axis Corporation.]
Running time: 26 mm. 50 sec.

This crash-on-takeoff from Detroit on August 16, 1987 illustrates a breakdown in crew coordination and departure from SOP~ The computer simulation traces the departure of the aircraft from the gate to the runway (14 minutes), with scripted and voice dialogue recording. From 15:00 minutes forward, the take-off roll and ensuing crash are examined from several different perspectives, including a split screen technique that affords the viewer the opportunity to simultaneously view a normal takeoff with the ill-fated attempt (mins. 25:41 through 26:48).

One aspect of Northwest 255 that makes it unique is a view of the accident as seen from inside the passenger cabin. Using computer animation, the cabin swings back and forth before turning completely over just before final impact. Also interesting: a second perspective where the interior of the cabin is superimposed on the runway, giving the viewer the opportunity to "see" the takeoff from a runway perspective while also glimpsing the inside of the cabin as it eventually flips upside down (minutes 19:34 - 24:12).

Z-Axis has produced other computer simulations of aircraft accidents which are worthy of note, including Avianca Flight 52 (1 hr. 15 mm.) and Continental 1713 (14:45 mm.). Additional accident simulations will become available once litigation involving the airlines in question are completed.

(5) The Wrong Stuff.
[BBC Videos for Education and
Training (1986).
Address: Woodlands, 80 Wood Lane, London
W12 OTT, England.]
Running time: 50 minutes.

The title of this film is meant to remind the viewer that what is the 'right stuff' for a pilot in the single-seat, high performance fighter aircraft is the "wrong stuff" when transferred onto the flight deck of the multi-crew commercial carrier. This documentary focuses on the relationship between a aviator's personality and aircraft accidents, using the crash of Air Florida 90 to illustrate the impact of such a relationship on air safety. The role of the "domineering" captain and the "non-assertive" co-pilot is also examined ... along with a program (United Airline's CRM ) designed to deal with personality and role conflicts in the cockpit. Certain segments of this program can be used in a "stand-alone" fashion to illustrate critical CRM concepts. Some examples: (1) The importance of teamwork the NASA experiments on crew variations in teamwork are presented between minutes 9:08 and 12:50; a second example of teamwork, involving a three-man crew working together effectively to correct a "loss of all generators" problem, is highlighted during minutes 28:03-31:50. (2) The impact of personal behavioral styles in the cockpit., These styles are examined in the context of the Air Florida crash (between minutes 17:34-28:00). (3) The value of a pilot as a creative decision-maker (why he won't be replaced by a computer): Excellent example provided by pioneering aviation expert John Lauber (minutes 46:38-49:58).

(6) Why Planes Crash.
[Produced in 1987 by WGBH Educational
Foundation and distributed by Coronet Films
& Videos. Address: P.O. Box 2649,
Columbus, Ohio, 43216.]

This film, aired one year after release of The Wrong Stuff, provides the viewer with a second overview of human factor issues in aviation safety. The two documentaries overlap in some areas; yet, there are enough differences to make each film worthwhile. For example, this program also has a good reenactment and discussion of the Air Florida 90 crash (minutes 4:19-12:45) and the CRM coordinator might want to consider using both renditions (or segments of them) in seminar discussions of the accident. Other segments of this documentary can also be utilized in emphasizing CRM-related topics of importance: (I) The danger of functional fixation: Loss of situational awareness is illustrated in a simulated reenactment of the final minutes of Eastern Flight 401.. where crew preoccupation with a nose-wheel indicator light resulted in a fatal crash in the Florida Everglades (minutes 22:10-25:32). (2) Automation complacency. Professor Ban Wiener, a leading authority on this topic, warns of the dangers of automation using Korean Airlines Flight 007 and Air China9s Dynasty Flight 006 to underscore his concerns (minutes 36:32-40:20). (3) Decision-making: An excellent and thorough examination of the events leading up to the crash of Delta 191 is presented in minutes 4~ :45-53:25. This film sequence is particularly powerful when used in conjunction with the Delta 191 production by 7-Axis (see film selection #3).

In previous seminars, the authors have used the Delta 191 segment of Why Planes Crash to kick off CRM seminars on decision-making. Followed-up later with the 7-Axis presentation, it encourages open discussion and insights into the problem-solving process. Included at the end of the Delta 191 segment in Why Planes Crash is one of the most powerful quotes that can be used to start or conclude a CRM program. It is worth repeating here. Whether any pilot alive could have flown through a wind sheer of that intensity will never be known. But in an enterprise as complicated as our air transportation system, some accidents will inevitably occur. Pilot error is once again the weakest link in a long chain of human errors. Perhaps that flight could have been saved with better weather information, or if someone, somewhere on the ground had made a better decision than they did. But ultimately, it is the crew that flies the plane, and must bear the responsibility for decisions which no one else is in a position to make. Improved communications ... better management of tasks ... teamwork.... cockpit resource management by whatever name. All these may not rescue any given flight, but mightn't they improve the odds on every flight?"

(7) United Flight 232.
[United Airlines
Address: Flight Training Center, Stapleton International Airport
Denver, CO 80207
attn: C/LR Coordinator (1990).]
Running time: 35 minutes (short version),
47 minutes (long version).

A great "CRM save" story. United Airlines Flight 232 (DC-i 0) was able to crash-land at Sioux City, Iowa on July 19, 1989 after experiencing the "impossible"... a loss of all hydraulics. What unfolds in this dramatic video is how the crew responded to this unprecedented crisis, drawing on skills learned during CRM training to facilitate the teamwork necessary to keep the crippled aircraft flying. Actual news footage of the accident is presented along with cockpit-ground communications and interviews with the tech crew and flight attendants who were aboard Flight 232 during the emergency. This tape is suitable for use with tech crews and/or tech and cabin crews

together. Because the long version involves additional cabin crew focus, it is recommended for seminars involving both pilots and flight attendants. The shorter, 35 minute production is preferable if only pilots will be included in training.

To underscore the important role of CRM in the UAL 232 crisis the authors recommend a short film excerpt (minutes 26:15-29:35) from the NTSB hearings into the crash. [Tape # VAV Ol IA: Sioux City, Iowa United Airlines, NTSB Board Meeting Held 11/1/90,] The film can be ordered for $20.00 from: General Microfilm, P 0 Box 2360, Wheaton, MD 20915. In the excerpt Dr. John Lauber praises the United CRM program and its critical role in enhancing aviator performance in the cockpit.

(8) Aircrash: The Burning Issue.
[BBC Videos for Education and Training (1987).
Address: Woodlands, 80 Wood Lane
London W12 OTT, England.]
Running time: 50 minutes.

After a brief discussion of some well-known fires aboard commercial aircraft (e.g., the British Airtours crash at Manchester), the film provides a very graphic description of what it's like to escape from a burning airplane, as told by a passenger who experienced the horror first-hand. Also included in this informative film: (a) some spectacular video footage of experimental aircraft being purposely crashed to study the effects of different fuels on post-crash fires; (b) a discussion of scientific efforts to create a less-combustible (volatile) aircraft fuel; (c) the evolving use of flame-retardant materials in the aircraft cabin and (d) the use of smoke hoods (masks) to extend passenger survivability time in the event of an aircraft fire.

Although most CRM instructors will probably not show this entire film to their participants, several segments will be of particular interest to pilots; for example: (a) the experimental crashes to study fuel fires; (b) the controlled study of fire dispersion in an aircraft (minute 25:00+); and (c) the discussion of fires aboard actual flights (e.g., Continental Airlines at LAX (minute 21:00) and Air Canada 797 (minute 42:00). (A more complete study of the Air Canada June 2, 1983 fire is contained in a 12 minute video entitled "Flash Fire: The Story of a Flight.")

(9) To Engineer is Human.
[BBC Videos for Education and Training,
Woodlands, 80 Wood Lane, London W12
OTT, England. (1987)]
Running time: 50 minutes.

This film will give the most jaded engineer a dose of humility...and pride. This is a good documentary for any operational staff involved in the engineering side of aircraft operation. The narrator, an engineer himself explains the process by which safe designs evolve into unsafe designs (how engineers take successful structures and evolve them into failures) and then, learning from failure, make them safe once again.

Although the subject-matter of the entire film is not specifically relevant to pilots, there are a few noteworthy segments; particularly those that involve engineering errors impacting on airline safety. Some examples: (1) design problems with the Comet aircraft in the 1950s (minute 2:17); (2) faulty maintenance leading to the crash of a DC-10 in Chicago (minute 34:30). There are also some thought-provoking comments for those aviators concerned with increasing levels of computer automation in the cockpit. According to the narrator (minute 43:20+), computers don't do a good job of anticipating some types of failures, ("they don't think in the failure mode like human engineer do"), can create a false sense of security, and are responsible for ushering in a new era of engineering error, the so-called "computer-aided catastrophes."

(10) Automation Asks for Monitoring--Are We Suited to This Role?
[KLM Airlines (1995).
Address: KLM Royal Dutch Airlines
Flight Crew Training Department (SPUNT)
Flight Services, P 0 Box 7700, 111 7ZL
Schiphol Airport, The Netherlands.]
Running time: 19 minutes.

This training film presents an easy-to-understand "three-phase" model of behavior (perception phase, judgment phase and execution phase) and then uses actual airline incidents to show how automation can impair a pilot's ability to perform effectively on the flight deck. Good introduction to the potential dangers of cockpit automation.

(11) Automation and Workload.
[American Airlines.
Address: Commercial Flight Operations,
American Airlines Flight Academy,
P 0 Box 619617;
DFW Airport, Texas 75261-9617.]
Running time: 19 minutes.

After a discussion of some common mistakes that occur in the automated cockpit (e.g., programming errors on B-757 flight deck computers) an automation-related accident is simulated and discussed (minutes 11:00-18:33). The simulation depicts Thai Flight 311 which crashed into terrain on approach to Katmandu, Nepal on July31, 1992.

(12) Against The Clock.
[BBC Videos for Education and Training (1994).
Address: Woodlands, 80 Wood Lane
London W12 OTT, England.]
Running time: 50 minutes.

Many factors can influence a pilot's effectiveness on the job (e.g., Karlins, Koh & McCully, 1989). This film examines one such factor, fatigue, a particularly relevant topic for aircrew who, in the words of the commentator, must work in a constant state of jet lag." Two segments of this documentary will be of particular interest to tech crew: (1) commercial pilots flying actual flights are studied to see how well they can perform perceptual-motor tasks under different conditions of cockpit rest patterns (minutes 31:30-36:00). (2) Minute 11:00: A powerful warning about the danger of fatigue in the glass cockpit is sounded by way of analogy: the impact of old vs. new style petrochemical control rooms on the operator1s ability to remain alert (and effectively able to monitor) on the job. This film segment should be required viewing for every person working in flight operations.

(13) "MAC 249, What Are Your Intentions?"
[U.S. Air Force:
Produced by 1352nd AVS/DOPV under contract with Aerospace Audiovisual Service MAC for U.S. Air Force
Running time: 20 mm. 30 sees.

An examination of aircrew coordination and teamwork is presented in this training film which can be easily adapted for use with commercial pilots. The Crew of a C-141 Starlifter is confronted with a partially retracted landing gear problem upon takeoff for a trip to New Zealand. At several points during the flight the aviators must confront and solve problems associated with the broken gear.. and the viewer is asked what he would do as well. Because of the complexity of the problem there are numerous alternatives that must be considered, along with the need for acquiring additional information and keeping mindful of fuel-burn in relation to the targeted airport destination. There are natural breaks in the film that allow the CRM coordinator to discuss with seminar participants what they would do if they were the crew aboard the C-I 41. The results of those discussions can then be compared with the actual decisions reached (and results achieved) by the Starlifter aviators as they attempt to complete their mission successfully.

(14) The Public Mind. Image and Reality in America (with Bill Moyers) #4: The Truth About Lies.
[Originally aired on PBS, 11/29/1989. T
o order write: Films for the Humanities & Sciences
12 Perrine Road, Monmouth Junction
NJ 08852.]

This thoughtful examination of truth and deception in American life contains a 12 minute, 30 second segment concerning the Challenger tragedy which is a must see item for advocates of CRM The presentation focuses on the pressures faced by Morton Thiokol and NASA, pressures which played a crucial role in pushing ahead with the ill-fated and ill-advised launch. The desperate efforts of Morton Thiokol engineer Roger Buisjoly to halt the count-down.. .the fateful conference call between Morton Thiokol, the Kennedy Space Center and the Marshall Space Flight Center.. .the investigative hearings into the tragic affair.. all are powerfully and succinctly presented for the viewer's sobering consideration.

This Challenger segment serves as a wake-up call to those individuals who allow non-flight factors to unduly influence appropriate flight decisions. Perhaps this needless and catastrophic accident will not have been in vain if it serves to remind us that, even in the competitive environment of commercial aviation, operational pressures must never be allowed to compromise the integrity of flight safety.

(15) Groupthink (revised edition).
[CRM Films,
2215 Faraday Avenue,
Carlsbad, CA 92008.] 1995.
Running time: 25 minutes.

This training film has two things in common with #14 above:(1) both films focus on the Challenger disaster; (2) both films examine the pressures that led to the inappropriate decision to launch. But, whereas The Truth About Lies focused on pressures impacting the organization, Groupthink targets group pressures impacting the individual group member. When such a condition exists, it can lead to a flawed decision-making process whereby the members' striving for unanimity [within the group] overrides their motivation to realistically appraise alternative courses of action (Janis, 1995)." In a flight deck context, where "teamwork" and "crew involvement in problem-solving" is encouraged, the Groupthink phenomenon can present a potential decision-making hazard that pilots need to know about. This film provides them with that awareness.

(16) Fall From Grace: Patterns of Human Error.
[BBC Videos for Education and Training (1987).
Address: Woodlands,
80 Wood Lane
London W12 OTT, England.]
Running time: 30 minutes.

The distributors of this film ask a provocative question: "We all make mistakes now and again, but what would happen if we suffered the same kind of lapse when flying a jumbo jet?" The answer is not pleasant, as a powerful re-enactment of the Tenerife tragedy amply demonstrates (minutes 9:00 through 13:05). This documentary suggests that everyday inconsequential errors (trivial mistakes) can lead to major accidents.. .it all depends on where and when they occur (in the kitchen they might result in an improperly brewed cup of tea, in the cockpit they might lead to an improperly configured aircraft). It seems that one danger of learning a skill too well is the individual's tendency to perform that routine on "autopilot" -without proper monitoring -- leading to errors that can pop up when certain kinds of distractions occur. This is a good film for emphasizing the importance of situational awareness and monitoring for effective flight deck management, particularly in the glass cockpit environment.

(17) Crew Resource Management.
[Northwest Airlines (1989).
Address: Department F-5830,
5101 Northwest Drive,
St. Paul, MN 55111-3034]
Running time: 40 minutes.

This training video is designed for both technical and cabin crew. A series of vignettes demonstrates the importance of effective leadership, communication and teamwork for optimizing job satisfaction and performance during line operations. The first vignette (8:55 to 12:00 minutes) shows how the behavior of a commander can facilitate or "destroy" the motivation of a crewmember. The second, and longest vignette (12:03-29:10) underscores the critical nature of cabin-crew tech-crew interaction for safe, efficient flight. The scenario (based on an actual incident) involves a passenger's complaints of "fumes" in the cabin.. and examines how those complaints are handled by the flight attendants and pilots. In the first run-through of the scenario, the cabin-crew and tech-crew work poorly together for a number of reasons, allowing the situation to get out of hand (a serious delay in responding to a developing fire aboard the aircraft). In the replay of the scenario, interaction between flight attendants and pilots is excellent, and the problem is handled in an efficient and timely manner. This vignette, which can be combined with The Dryden Accident, is effective for use in seminars designed to foster better interaction between aircrew on both sides of the cockpit door.

(18) The Mystery of Flight 163.
[Granada Productions, World in Action.
Address: BRITE, London Television Centre
Upper Ground, London SEl-9LT, England.]

On August ~ 9, 1980 the pilots aboard Saudi Flight 163 reported "smoke in the cabin" and made an emergency landing. The touchdown was accomplished without incident, but all 301 passengers and crew aboard the L-1011 were killed by fire and fumes that swept the length of the aircraft before they could escape. The documentary explores two aspects of the tragedy, (a) what caused the catastrophic fire; and (b) crew performance in responding to the crisis. For CRM purposes, the crew performance segment (the first 12 minutes of the film) provides the most relevant material. The reconstruction of the ill-fated flight, detailing how the tech crew handled the emergency, reveals several serious break-downs in crew communication, coordination and decision-making. For that reason, the film segment provides an effective introduction to the CRM seminar; where it can be used to illustrate and/or stimulate discussion of human factor errors on the flight deck.

(19) Fatal Error.
[BBC Videos for Education and Training (1991).
Address: Woodlands 80 Wood Lane,
London W12 OTT, England.]
Running time: 30 minutes.

On January 8, 1989 A British Midland B737-400 crashed with the loss of 47 lives. The film begins with a recreation of the accident sequence in which the pilots, faced with an engine malfunction, mistakenly shut down the wrong (good) one. It appears to be an open and shut case of pilot error. But is it? The film's commentator goes on to suggest that "pilot error" might be too simplistic an explanation and points to other factors that played a role in the accident: (1) poor ergonomic design of cockpit instrumentation which made it difficult to detect which engine was malfunctioning; (2) no B-737-400 simulator available for pilots to train on and a very abbreviated conversion course in preparation to fly the aircraft; (3) a flaw in the engine design, which went undetected, in part, because the engine wasn't tested in the air; and (4) failure to mount a camera in the rear of the plane so the pilots could see which engine was malfunctioning.

Fatal Error is a useful film for showing how a chain of events can lead to an accident. It also provides a useful and interesting lead-in for a discussion of decision-making. CRM participants find it stimulating to examine whether the British Midlands pilots could have made a better decision and if so, how? Was the decision reached too quickly? Was there adequate information gathering and assessment to determine if the course of action was the correct one? Once the decision was reached should it have been evaluated further? In examining these questions, the CRM coordinator might want to incorporate a KLM training film that discusses the Midlands crash in a problem-solving context. [Information Management (1995). Address: KLM Flight Crew Training Department (SPUNT), P 0 Box 7700, 111 7ZL Schiphol Airport, The Netherlands.]

(20) Moment of Crisis: There's A Hole in the Plane.
[ABC News, 20-20, September 22, 1989.
Address: Film & Tape Practices, ABC News,
47 West 66th Street,
New York NY 10023-6290.]

What better way to end a CRM seminar--or an article on CRM films--than with a heart-warming example of how pilots can "get it right" and save lives and airplanes in the process! CRM presenters, in their admirable attempt to enhance flight safety by pointing out pilot errors ("learning from our mistakes"), sometimes fail to emphasize the positive side of the cockpit performance equation: the thousands of examples where skilled, dedicated aviators have saved the day. Moment of Crisis presents the saga of United Airlines Flight 811, a B-747 that experienced an explosive decompression and loss of two engines when, at 22,000 feet, the forward cargo door and part of the passenger cabin blew out, sending nine passengers to their death. The crew was able to turn the plane around and safely land in Hawaii where engineers, astounded by the massive hole in the 747's fuselage, claimed that "...based on the structural damage there was no way the aircraft could have gotten back."

Moment of Crisis ends with moments of joy. Captain David Cronin, the commander who had piloted Flight 811 to safety, flew one last flight before reaching the mandatory retirement age of 60. As he left the cockpit for the final time there was a celebration to honor him. Grateful passengers and crew embraced him and placed garlands of flowers around his neck. It is a fitting tribute to a remarkable man and, in the final analysis, a salute to all the commercial pilots who share in his triumph of making the skies a safer place to be.

Some Final Observations

Films will never take the place of the CRM presenter any more than automated cockpits will take the place of the airline pilot. Rather, these video productions are meant to provide additional learning tools for use in the aviation training and learning environments. To maximize their effectiveness, a few final recommendations are offered: (1) when ordering videos, be sure to specify the tape format used in your country (e.g., the United States utilizes the NTSC format; many Asian countries have PAL). (2) Through-out this article the authors have used time designations (expressed in minutes) to pinpoint a specific topic/scene in a film. Please be aware that these times can vary up to two minutes (although usually much less) for readers using the films due to: (a) playback variations in different video systems; and (b) viewer variations in defining where the film "begins." (3) Some films go out of print or are assigned to new distributors. If a reader should encounter difficulty obtaining any film listed in this article, please contact the senior author for assistance (by phone: 813-974-1746; by fax: 813-684-2224). (4) The NTSB maintains a large library of aviation videotapes and accident reports. Written accident reports are available from the National Technical Information Service ~S), 5285 Port Royal Road, Springfield, VA 22161. Videotapes can be ordered for ~20 each from: General Microfilm, P 0 Box 2360, Wheaton, MD 20915. (5) Transcripts (and sometimes videos) of various television and radio programs are also available. For programs appearing on ABC, PBS, CNN and NPR contact: Journal Graphics, Inc., 1535 Grant Street, Denver, CO 80203 (phone: 800-825-5746 or 800-255-6397 For programs airing on CBS, NBC and CNBC contact: Burrelle's, 75 East Northfield Road, Livingston, NJ 07039 (phone: 800-631-1160).


Several individuals provided encouragement and assistance in the preparation of this manuscript. The authors gratefully acknowledge them here: Patricia Antersijn, KLM Flight Crew Training Department; Captain Graham Beaumont, CRM Coordinator, Qantas Airways Ltd; Captain John L. Bedker 11, President, AnCam Inc; Benjamin A. Berman, Aviation Safety Investigator, Operational Factors Division, NTSB; Jeffrey M. Betz, Z-Axis Corporation; Captain John D. Crawley, Supervisor, General Operational Subjects, Flight Crew Training America West Airlines; Roland P. Desjardins, Director - Commercial Flight Operations, American Airlines Flight Academy; Claire Dinan, BBC Videos for Education and Training; Dr. Elena Edens, Office of the Chief Scientist for Human Factors, Federal Aviation Administration; Dr. Robert L. Helmreich, Professor of Psychology, University of Texas-Austin; Captain Calvin L. Hutchings, C/LR Coordinator, United Airlines; Dr. Richard S. Jensen, Professor of Aviation & Director of Aviation Psychology Laboratory, Ohio State University; Amy Klene, Journal Graphics, Inc; Captain Louis G. Nemeth, Supervisor, AQP Training Development and Validation, USAir; Dr. Maureen Pettitt, School of Aviation Sciences, Western Michigan University; Captain Kurt Shular, Delta Airlines; Felicia D. Stoler, ABC News; and George Sweeney, Manager - Human Factors Development, Northwest Airlines.


Bedker, J. L. (1995). Welcome to the future: A technological approach to complete resource management training. In Proceedings of the 1995 EAC Flight Crew Training Conference. Atlanta, GA: EAC Aviation Events.

Janis, I. (1 995). In Leaders Guide for Groupthink (Rev. ed.), p.7. California: CRM, Inc.

Karlins, M., Koh, F, & McCully, L. (1989). The spousal factor in pilot stress. Aviation, Space and Environmental Medicine, 60(11), 1112-1115.

Karlins, M., Koh, F., McCully, L., & Chan, C. T. Expanding teamwork beyond the cockpit door: An integrative program (11OASIS~') for pilots, cabin crew, station managers/traffic and ground engineers In Telfer, A. & Moore, P. (Eds.). Aviation Training; Pilot, Instructor and Organization. In Press: Australia: Avebury Technical, Ashgate Publishing.

Marcotte, P. (1 989). The Final Minutes of Delta 191. American Bar Association Journal, December. 5 pp.

Scbroder, II., Karlins, M., & Phares, J. (1973). Education for Freedom. New York: John Wiley.  

The Evolution of CRM Skills Development from a Systems Perspective

by: Dr. Maureen A. Pettitt
Mr. Joseph H. Dunlap
Western Michigan University

Systems are an integral aspect of the aviation industry. We fly from Chicago to Denver in aircraft depending upon hydraulic and pneumatic Systems, using Flight Management Systems to navigate successfully through the National Airspace System. Airlines and organizations currently using the FAA's Advanced Qualification Program as a model for modifying their pilot training programs utilize variant of Instructional Systems Design (I SD) in the development of training curricula and instructional media. Each of these, however, is an example of a system within a larger system. Solutions to the industry's challenges too frequently focus on one isolated system rather than on the larger whole, on short-term, economical fixes rather than fundamental solutions. In the meantime, we ponder why our efforts fail to create the safe, efficient and effective industry we strive to achieve.

In his widely-read book, The Fifth Discipline1 Peter Senge describes the disciplines necessary to build learning organizations: mental models, shared vision, team learning, personal mastery, and systems thinking. Interestingly, Senge uses the development of the DC-3 to illustrate the necessity of all five disciplines. He notes that the thirty years between the Wright Brothers' first powered flight and the development of commercial air travel were marked by a myriad of failed experiments. "The early planes were not reliable and cost effective on an appropriate scale... The DC-3, for the first time, brought together five critical component technologies that formed a successful ensemble. They were: the variable-pitch propeller, retractable landing gear, a type of light-weight molded body construction called 'monocoque,' radial air-cooled engine, and wing flaps. To succeed, the DC-3 needed all five; four were not enough." Similarly, Senge believes that these five disciplines represent the new "component technologies" required to build learning organizations.

Systems thinking is the fifth and most essential discipline because it is the integrative discipline, the discipline that fuses the others into a coherent body of theory and practice. The purpose of systems thinking is to move from reacting to events, dealing with problem symptoms to a deeper level of understanding, attending to the fundamental problem. Systems thinking focuses on systemic structures, where the greatest leverage lies for solving problems. It is a discipline for seeing wholes, discerning patterns of behavior and interrelationships, and learning about the circles of causality that characterize our complex environment.

Systems archetypes -- or templates -- are causal loop diagrams utilized by Senge and his colleagues to represent generic systemic structures.

In this article, we will explore the evolution of CRM using one such archetype. The scenario begins with the recognition of problem symptoms. In our example, the problem symptom is a string of air transportation incidents and accidents in the 1970s which could be attributed to human factors errors in crew performance -- specifically, lack of or poor communications, team building, situational awareness, and similar skills. Feeling pressured, industry, government and the research community joined forces to begin a serious examination of aviation human factors and to develop solutions to human factors errors in aviation operations. One such solution was CRM awareness training for airline pilots. A linear representation of these events might look like this:

dgrm1.gif (2901 bytes)

Diagram 1

A systems thinking representation, on the other hand, would look like this:

dgrm2.gif (6039 bytes)

Diagram 2

As awareness training increased, it was expected that human errors would be reduced, stabilizing the problem. As shown in this diagram, a side effect of awareness training was significant improvement in pilot attitudes about CRM-related issues. While a seemingly positive side effect, at some air carriers this improvement in CRM awareness reduced attention to underlying problem (the lack of CRM skills) and the fundamental solution (to develop a pool of pilots with excellent CRM skills through improved selection and training processes). Concern about cost, time, and public image only increased the amount of energy dedicated to the symptomatic solution, awareness training, further reinforcing the resistance to developing a long-term, fundamental solution. The resulting structure can be graphically represented as follows:

dgrm3.gif (9602 bytes)

Diagram 3

This shifting of attention away from the fundamental solution was due not only to the success CRM awareness training; but, equally important, to the fact that there are almost always delays in the long-term change loop. The need for an immediate response to internal and external pressure on the industry to demonstrate immediate improvements in system safety contributed to the focus of energy and resources on the quick fix. In systems thinking parlance, this archetype is referred to as "Shifting the Burden." This structure is composed of "two balancing (stabilizing) processes. Both are trying to adjust or correct the same problem symptom." The top loop is the symptom correcting process and the bottom loop is the problem correcting process. The bottom loop represents a more fundamental response to the problem, one whose effect takes longer to become evident but works far more effectively.

In a report last year to the ATA/FAA AQP Working Group, Bob Helmreich and Bill Taggart noted a "slippage" of CRM attitudes at some carriers.2 Possible explanations for this slippage included: selective compliance by line check airmen and instructors; day-to-day management of CRM by non-operational staff, the length of time since initial awareness training, recurrent training focused on non-cockpit topics, and the lack of clarity and alignment in instructional objectives and assessment measures in the carriers' LOFT programs. These explanations may represent negative side effects of the short-term solutions, side effects that may erode long-term capability of the system to select and/or develop crews with appropriate CRM skills.

Despite occasionally discouraging results, there are internal and external forces at work that will meaningfully alter the structures associated with CRM skills development. Clearly, one of the forces encouraging structural change is the FAA's Advanced Qualification Program (AQP). The AQP methodology requires that the airline use CRM factors and principles in developing proficiency objectives. More importantly, the AQP dictates that CRM factors and principles be incorporated into the tasks and subtasks as skills, knowledge, and attitudes. These factors must be individually identified in the task analysis and flagged for evaluation3. The addition of this variable has altered the systemic structure of our template, as shown in the following diagram.

dgrm4.gif (8380 bytes)

Diagram 4

Such templates serve as useful guides for mapping not only events, but also patterns of behavior and systemic structures. They are not meant to be rigid frameworks into which a specific case must fit. Certainly we have presented, in this example, a simplified version of a very complex issue. Any number of variables would be added to the diagram in order to accurately depict the evolution of CRM -- most likely making our diagram indecipherable in the process. Our purpose here was to offer some alternative ways of thinking about CRM and to illustrate how reliance on a symptomatic solution, however successful, may postpone implementation of a fundamental solution.

The lesson from this exercise in systems thinking could be that top management must commit resources and support for long-term, fundamental solutions to the industry's safety issues As Senge notes, the structure described here frequently has an additional reinforcing (amplifying) process created by side effects of the symptomatic solution that "make it even more difficult to invoke the fundamental solution."4 Clearly the success of the shorter-term, symptomatic solution to human factors errors made it easier for management at some airlines to delay the more difficult, costly, and potentially controversial solution: selecting for and training CRM-related skills in a comprehensive and systematic fashion.


1.Senge, P.M. (1990). The Fifth Discipline. New York: Currency Doubleday.

2.Helmreich, B., & Taggart, B. (1995). Report of the NASA/University of Texas/ FAA Aerospace Crew Research Project to the CRM Focus Group. Reproduced by the FAA/AQP Working Group.

3.U.S. Department of Transportation. Federal Aviation Administration. (1991). Advanced Qualification Program. AC 120-54 (pp.7-3,7-5).

4.Senge, P.M. (1990). p.106 

CRM and Technical Operations

by Barbara Kanki, NASA/Ames Research Center


We are well aware of the way in which CRM is incorporated into flight operations. First, there is initial indoctrination, awareness training which is typically taught in a classroom. Second, there is simulation training (typically embedded in recurrent training) which focuses on the practice of specific CRM behaviors. Third, there is Advanced Qualification Program (AQP) which, among other things, is aimed toward the integration of CRM into technical training. Within AQP, the appropriate qualification standards arc based on task analysis which integrate CRM elements.

In introducing CRM to technical operations, there are also numerous ways to proceed. As one might expect, awareness training is an obvious start point (see CRM Advocate article by Taylor and Robertson, Issue 94.4 - October 1994). In fact, the recently held ~ 0th FAA/AAM Human Factors Meeting in Aviation Maintenance and Inspection featured maintenance performance enhancement and technician resource management (January 1996). However, other approaches to airline maintenance human factors were also represented in the conference agenda including programs which incorporate CRM into task analysis, incident analysis, and other training, safety and quality assurance initiatives.

The following two projects are examples of the "AQP" approach; namely the integration of CRM elements into task analysis. In the first case (Project 1), the objective of the enhanced task analysis is to provide systematic guidelines for re-designing engine change procedure. Although a laborious and innovative re-design process had already been accomplished, the "rules" by which one could extend this process to other engines and for airline applications had not been systematically carried out. In the second case (Project 2) the objective of the enhanced task analysis is to create a systematic on the-job training (OJT) system which is carried out through a participative workforce. In both cases, the enhanced task analysis aim toward creating procedures and training which match the operator environment more closely. In the he spirit of the CRM concept, their goals include the safer, more effective and efficient use of resources.

Project 1 - The effects of team factors on procedure design: Validation of engine-change procedures through team task analysis

The objective of this project is to enhance maintenance procedures through the incorporation of team task analysis. In addition, assessment metrics will be developed so that team performances can be accurately and consistently evaluated. The general approach is to use the B737 CFM56-7 engine change operations as a test bed for developing the generic team task analysis and assessment tools (see Repp, 1995). We will produce documented, systematic guidelines for extending the process to other engine/aircraft series and recommendations which can be modified by airline customers who are interested in redesigning their own engine-change procedures.

This is a NASA-initiated project which relies on Boeing collaboration (Customer Services and Propulsion Systems Divisions). Jointly we will: 1) conduct familiarization observations of engine change activities, 2) develop a model team task analysis system, 3) apply the team task analysis system to the original and enhanced engine change procedures (provided by Boeing) which incorporate team process data and feedback, and 4) validate procedure differences through observations, interviews, videotapes (mostly already existing in Boeing archives). Finally, we will refine existing performance metrics to incorporate team processes and write a technical report describing the team task analysis system and associated measurement system.

Project 2 - Task Analytic Training System (TATS): A Human Factors Approach to Aviation Maintenance and Inspection Training

The Task Analytic Training System (TATS) is a performance-based system that involves full workforce participation in its design, development and implementation. It was originally developed to provide comprehensive, structured, on-the-job training for Boeing inspectors (see Walter, 1990). Through incorporation of basic human factors principles such as decision making, communication, team building, and workload management, either directly or as a function of the techniques involved, the TATS process has proven successful in providing not only better training and procedures, but overall improvement of attitude and morale. TATS produces a trained workforce whose performance can be observed and measured against explicitly defined standards.

This is a Boeing-initiated project with NASA collaboration. The goal of the research is to provide an airline model of TATS implementation in maintenance and inspection operations, training and procedures. We propose to track the implementation of TATS in several participating airlines. The field evaluation will include numerous issues (such as needs identification, job task analysis, writing and verifying training modules, employing tracking mechanisms and performance metrics), as naturally-formed teams decide which individual and team tasks to analyze, evaluate and train. We will assist in adapting the TATS so that team tasks can be included in the task analysis, training evaluation plans. In addition, we will assist in the development of metrics and the verification processes. The product of the project will be a guidelines document for implementing the TATS in airline technical operations for a variety of applications.

Additional information on either of these projects may be obtained by contacting: Barbara Kanki, NASA/Ames Research Center, MS 262-4, Moffet Field, CA 94035 ( or Diane Walter, Boeing Commercial Airplane Group, MS 2J-21, Seattle, WA 98124-2207


Repp, T. (1995). Improving 737 CFM56 engine change times. In Airliner (Oct-Dec 1995), Seattle: Boeing Commercial Airplane Group.

Taylor, J.C. & Robertson, M. (1994). Successful Communications for Maintenance. In The CRM Advocate (Issue 94.4), Charlotte: Resource Options

Walter, D. (1990). An innovative approach to NDT Inspector Training at Boeing. In J.F. Parker, Jr. (Ed.) Human Factors issues in Aircraft maintenance and Inspection, Training Issues. Washington, DC: Federal Aviation Administration.  

A New Tool for Assessing LOFT Debriefings

by Lori McDonnell and Key Dismukes, NASA/Ames Research Center

How much crews learn in LOFT and take back to the line hinges on the effectiveness of the debriefing that follows LOFT. The simulation itself is a busy, intense experience, and thoughtful discussion afterwards is necessary for the crew to sort out and interpret what happened and why. Instructors are expected to lead LOFT debriefings a way that encourages crew members to analyze for themselves their performance in the LOFT. Rather than lecturing the crew in the traditional manner on what they did right and wrong, the instructor is expected to facilitate self-discovery and self-critique by the crew.

NASA is nearing completion of a study that examined LOFT debriefings at five major airlines order to determine the extent to which facilitation used in debriefings, the effectiveness of various techniques, and how well crews are able to analyze their own performance along CRM dimensions. (All individuals and organizations were deidentified in the process of analyzing the data). As a part of that study, we developed a Debriefing Assessment Battery (DAB) so we could systematically evaluate the effectiveness of facilitation and the character of crew participation in debriefings. We summarize here that part of the study, which was recently published as a master's thesis by McDonnell (1995).

The DAB is a subjective rating scale consisting of four categories (Introduction, Questions, Encouragement, and Focus on Crew Analysis and Evaluation) in which the instructor's facilitation is rated, and two categories (Crew Analysis and Evaluation and Depth of Crew Activity) in which the crew's participation is rated. Each category includes four items (questions) which the rater grades on a 7 point scale from poor to outstanding. We used the DAB to evaluate debriefings from audio tapes and performance before eliciting their own perceptions and analysis, not following up on issues raised by crew members, and showing video segments without discussing them.

Not surprisingly, we observed a correspondingly wide range in the depth and focus of crew participation. Also, even when crew participation was high, the discussion revolved around the instructor's questions and comments. Crew members rarely engaged in direct back-and-forth discussion for more than a moment. Thus, the sometimes-expressed concept that crews should debrief themselves, using the instructor as a resource, has not yet been achieved and at this stage of CRM implementation may be overly idealistic.

The report of the full study will provide detailed information on the content of the observed debriefings and an analysis of which techniques seem to be most effective in eliciting crew participation. We also explore the implications of these findings for programs to train instructors to be effective facilitators. Readers who wish to be notified when the report is published may contact the authors at NASA Ames Research Center, Mailstop 262-4, Moffett Field, CA 94035.

Reference: McDonnell, L. K. (1995). Facilitation techniques as predictors of crew participation in LOFT debriefings. Unpublished master's thesis. San Jose University. Also under review as a NASA Contractor Report.

The material contained in The CRM Advocate back issues is the property of the contributing editors. No duplication of any kind is authorized without the express written permission of the editor. All rights reserved. For training and information purposes only. The intent of the editors is shared information, through controlled distribution to the benefit of the safety of flight.


Home ] Up ] 93.1 ] 94.1 ] 94.2 ] 94.3 ] 94.4 ] 95.1 ] 95.2 ] 95.3 ] 95.4 ] 96.1 ] [ 96.2 ]