Table of Contents

• The View from Austin, by: Bill Taggart, NASA/UT/FAA Crew Research Project
• Key Ingredients of High-Quality LOFT: A Recipe for Effective Pilot Training, by: J. Randolph Law, Robert L. Helmreich, & John A. Wilhelm, NASA/UT/FAA Aerospace Crew Research Project
• Report: NASA/FAA/DoD Communications Workshop, by: Dr. Barbara G. Kanki, NASA Ames Research Center
• Intervention Strategies For The Management Of Human Error, by: Earl L. Wiener, Abstract by Peter DeNucci
• Editorial: Make all pilots -- check pilots, by: Lou Nemeth
• InterScience America offers CRM Training, by: David P. Baker, Robert W. Sweezey, & Robert E. Llaneras

by:
Bill Taggart
NASA/UT/FAA Crew Research Project

This is the first of a column to offer various "news" and "views" from our perspective at the NASA/UT/FAA research project here in Austin. At the outset, I should probably mention that these comments are my own and do not represent what our various sponsors would say.

About three weeks ago, I received a telephone call from a regional airline manager who was charged with the responsibility of implementing a CRM course at the airline. The individual was very glib, and indicated that the CEO was against the training and refused to do anything until it was a required FAR- Nonetheless, the carrier was proceeding with CRM implementation because it was required in their dealings with the Department of Defense. This person now had about three weeks to design and implement a human factors training program, but the only resource available to him was the original version of the CRM advisory circular, dated 1989.

As we normally do, I assembled a care package of various articles and reports, along with the current version of the advisory circular. I stressed how most carriers today were moving away from gimmicks and games like "Lost in Lubbock" and were concentrating on solid skills-based training. In this regard, the behavioral markers have been used as a 'jumping-off point for developing CRM training and assessment systems that are specialized and fit a particular operating culture.

I also enclosed a reprint of a talk by Dr. John Lauber on the subject of fostering a safety culture in organizations. This is an article that strongly emphasizes the role of top management in developing attitudes, practices, and behaviors that buttress and support standardization and a commitment to safety. Such organizations tend to be open and receptive to ideas and thinking, welcoming outside critique, and always looking for ways to improve.

In the US, there are several airlines that have made the transformation from closed xenophobic cultures that were defensive, self-centered, and skeptical, to ones that have substantially changed the way in which they conduct flight crew training and checking. It is pleasing to us here in Austin that implementation of CRM, LOFT, data collection, and other initiatives have contributed to these turnarounds.

Another factor that is becoming ever more important in the decision to implement CRM and LOFT involves both the legal community and insurance underwriters. Airlines that have implemented comprehensive human factors programs have been able to argue for lower hull-loss premiums. The picture is different for those carriers without a high-quality integrated CRM/LOFT programs. When faced with insurance losses where human factors were part of the picture, some have experienced significant increases in the cost of coverage. The influence of the risk-management folks will continue to grow. If the operations managers are slow to come around, then others will force their hand. I have even heard one 747-400 fleet manager (non-US) argue that the time may come when auto-lands are the norm and standard.

The growing influence of the legal community comes in a different manner, although the impact can still be expressed in monetary terms. The safety and operational benefits of CRM are now indisputable, again my opinion but supported with much data, and this fact along with supporting information has been made widely available throughout the aviation community. Any operator, even one with a good safety record, that resists CRM, or implements it on a superficial "square-filler " basis is incurring, in my judgment, a level of potential liability and exposure that constitutes unacceptable risk for the managers and shareholders of the airline.

As a final note, I would like to throw in a few words about the topic of Selective Non-Compliance. Dr. Lauber's comments in the safety culture article stress the need for top management to be on-board and supportive. Unfortunately, we have seen examples where senior management is truly committed but the system breaks down at the chief pilot or fleet manager level. For some inexplicable reason, these managers stay put, while the organization tries to bring in reinforcements to provide human factors and CRM support in order to compensate for problems with the boss's viewpoints. In my view, the real problem is one of leadership and of holding chief pilots and fleet managers accountable for their actions or lack of them. Although the last thing we would like to see is putting every chief pilot and fleet manager in a pink jumpsuit, and holding hands with their line pilots during touchy-feely new-age crew awareness sessions, I do think that there is room for improvement in making sure that all managers are on the same page of the same book. When line pilots see that selective non-compliance is no longer tolerated at the management level, it will be clearer that such behavior will not be tolerated from others as well.

by:
J. Randolph Law, Robert L. Helmreich, & John A. Wilhelm
NASA/UT/FAA Aerospace Crew Research Project
University of Texas at Austin

Abstract

Although Line Oriented Flight Training (LOFT) is used by several major airlines, it is one of the least understood aspects of pilot training. Most LOFT training centers allocate substantial resources for scenario development with little attention paid to instructor training. Using 91 LOFT observations and multiple linear regression techniques, the present study found that, in addition to scenarios, instructors have a significant affect on the effectiveness of LOFT. No matter how well-scripted the scenario, an unskilled instructor can significantly reduce training quality. In addition to this general finding, the current study addresses specific strategies to enhance scenario design and details the skills LOFT instructors need to maximize the positive impact they can have on pilots in training.

Introduction

Line Oriented Flight Training (LOFT) may be one of the most widely used but least studied methods of pilot training. Although several major airlines use LOFT as part of their overall pilot training program the quality of LOFT training provided across airlines, and between fleets within airlines, can vary considerably. LOFT training centers have traditionally concentrated on scenario development to improve the quality of LOFTS. Although well-scripted scenarios are undoubtedly an important part of high-quality pilot training, how instructors conduct a LOFT session is equally important. For example, a study by Wilhelm (1991) found that several dimensions of the LOFT environment greatly affected pilots' perceptions of the usefulness of the training (e.g., how much of what they learned will they use on the line?). In the eyes of pilots receiving LOFT, the two most important factors affecting LOFT effectiveness were quality of the scenarios and quality of LOFT instructor's debriefing/videotape skills. Although these findings made sense, theoretically and practically, they were questioned by some skeptics who asserted that pilots undergoing training are neither accurate judges of LOFT quality nor are they objective in their evaluations. Addressing these criticisms, the present investigation seeks to find relationships between specific scenario design and instructor skill factors as they relate to LOFT quality as judged by independent observers -- finding a more objective recipe for effective pilot training.

Background

Lauber & Foushee (I 98 1) described LOFT as "The use of a training simulator and a highly structured script or scenario to simulate the total line operational environment for the purposes of training flight crews." They also believed that LOFT should be realistic and should involve the exercise of teamwork skills. This last point is important given the fact that at least 65% of air carrier accidents and incidents over the past 20 years were primarily caused by deficits in teamwork, or Crew Resource Management (CRM) skills (Nagel, 1988). Any training designed to ameliorate the primary cause of most aviation accidents (i.e., LOFT) is worth investigating very thoroughly.

There are a variety of factors which should be theoretically related to the quality of LOFT. Lauber & Foushee (1981) addressed which issues they deemed important to effective LOFT. Regarding scenarios, Lauber & Foushee believed that each should have specific objectives, should take into account the special needs of the organization that will use them should not be overly complex, but should be scripted as precisely as possible so as to attain their original objectives. Regarding instructors, Lauber and Foushee likened their role to that of a moderator. Ideally, the instructor would be "transparent" during implementation of the scenario in the simulator, and would facilitate the crew to critique themselves at the end. The most active role of the instructor might be taken during the initial crew briefing where the basic ground rules of the LOFT would be explained before orienting them to the specific scenario(s). The Lauber and Foushee guidelines were so well-written and compelling that after some initial LOFT observations in 1988 (seven years later), no research data had been collected to verify the accuracy of their basic tenets. A call to research was made by Helmreich, Wilhelm & Gregorich (I 98 8) to further explore the philosophy underlying LOFT and how to maximize LOFT's effectiveness as a setting for pilot training.

Answering Some Questions:

In order to address the questions generated by the preceding discussion, the LOS Evaluator Survey (LES; Wilhelm, Butler, & Connelly, 1992) was developed so that expert observers familiar with LOFT could rate various dimensions of the training experience including scenario, simulator, and instructor dimensions. Given the range of information gathered by this instrument, several key questions regarding LOS could be addressed:

I. Scenario Design

1. Which aspects of scenarios determine their value for crew coordination and technical training?

II. Instructor Skills

2. What activities are associated with high-quality LOS briefings?

3. What skills and behaviors are needed for effective scenario implementation?

4. What actions of the LOS instructor are related to superior debriefings?

III. LOS Quality

5. Which components of the LOS are linked to overall quality of the training experience?

The remainder of this paper focuses on information gathered from observing 91 commercial aviation flight crews in LOFT sessions. Expert observers used the LES to gather the data needed to address the questions posed above.

Method

The LES

The LOS Evaluator Survey (LES) is a 40-item instrument designed to measure several dimensions related to Line Oriented Simulation (LOS) training (Wilhelm et al., 1992). There are six main sections to the LES, each of which has several Likert-type items addressing specific information relevant to LOFT: (1) Scenario, (2) Instructor Briefing, (3) Instructor Administration of the simulation, (4) Instructor Debriefing, (5) Crew Evaluations, and (6) Overall Evaluations.

Subjects: Participants were 91 flight crews, and their LOS Instructors, from three major US air carriers which conducted CRM seminars and LOFT between 1991 and 1993. Each flight crew and instructor were participating in regular company training and agreed to allow trained observers to evaluate the quality of the LOS, the scenario, and the performance of the flight crew in training.

Observers: All raters were professional pilots and/or researchers trained to evaluate the quality of LOS administration and the quality of flight crew performance on a variety of human factors dimensions. All observers received the same training on how to rate the dimensions covered on the LES.

Procedures: Observers met with flight crews and instructors before LOS briefings began. Observers explained their purpose (to observe for research purposes only), and the confidentiality of their observations and ratings was emphasized. Observers did not "participate" in the conduct of the LOS from this point forward. After the LOS, observers completed the LES. Most LOS sessions lasted between 6 and 8 hours, which included the pre-flight briefing and post-flight debriefing by instructors.

Analytic Approach: Of several possible methods, we chose multiple linear regression to evaluate the questions outlined above. This technique can tell us which specific scenario elements or instructor behaviors are most related to the global outcomes of interest independent of the other predictors. For example, multiple linear regression can tell us which instructor behaviors performed during the pre-simulator briefing best "explain" summary judgments regarding instructors' overall briefing quality?

Results & Discussion

After reducing the data (i.e., factor analyses), regression techniques were used to address each of the original questions.

I. Scenario Design

Question #1. Which aspects of scenarios are linked to their value for crew coordination and technical training?

A. Crew coordination value of the scenario

The seven specific scenario items in the scenario section do a good job of predicting " Scenario's value for crew coordination training" (R² =.493). Beta weights and semi-partial correlations (sr_xy) for each of the items suggest that (1) workload, (2) realism of the problem and (3) creative problem solving required are all important features of scenarios designed for crew coordination training. Mechanical condition of the simulator was also significantly related to the outcome (see Table 1). Semi-partial correlations and Beta weights for the four items mentioned indicate that each of these components adds something to the overall scenario quality independent of the other components.

Table 1
Regression Summary for Predicting
"Scenarios Value for Crew Coordination Training"

B. Technical value of the scenario

The seven specific scenario items predict "Scenario's value for technical training" only moderately well (R² = .216). Beta weights and semi-partial correlations for each of the items suggest that only workload and checklist activity were important factors in designing scenarios for technical training. However, the relatively low predictiveness of the model suggests that there are many aspects of scenario design not measured by the LES that arc important when designing scenarios for technical training (see Table 2).

Table 2
Regression Summary for Predicting
"Scenario's Value for Technical Training"

In summary, it appears that raising workload levels of many scenarios would be the most useful change to current scenario design practices. Additionally, maximizing the realism of the problems presented and the amount of creative problem-solving required would improve scenario quality, though probably not for technical training. However, the moderate to low predictiveness of these models suggests that we are not effectively tapping all the important dimensions that constitute scenario quality -- especially scenario technical quality.

II. Instructor Skills

Question #2. What activities are associated with high-quality LOS briefings?

The five behavioral briefing items did an excellent job of predicting "Overall quality of the briefing" (R² = .864). Beta weights and semi-partial correlations for each of the items suggest that, although all behaviors were significantly associated with briefing quality, (1) orienting the crew toward LOS training, (2) integrating the crew into the discussion, and (3) reviewing CRM concepts explain differences in overall briefing quality above and beyond what any of the other behaviors explain (see Table 3). It is worth noting that orientation to LOS training" and "integration of the crew into the briefing" had by far the strongest independent associations with overall briefing quality.

Table 3
Regression Summary for Predicting
"Overall Quality of the Briefing"

Question #3. What skills and behaviors are needed for effective scenario implementation?

Information in Table 4 suggests that the four specific simulation items predict "Overall simulation evaluation" ratings very well (R² =.762). Although each behavior is reliably related to the outcome, actual operation of the simulator made the largest independent contribution towards explaining ratings of the overall simulation quality (sr_xy =.30).

Table 4
Regression Summary for Predicting
"Overall Simulation Quality"

Question #4. What actions of the LOS instructor are related to superior debriefings?

All five items chosen for the model are very predictive of ratings for "Overall effectiveness of the debriefing" (Adjusted R²=.800). "Summarization of key points at the end of the debriefing" and "keeping focus of the crew on self-appraisal" are particularly potent predictors of debriefing quality independent of the other variables (see Table 5).

Table 5
Regression Summary for Predicting
"Overall Effectiveness of the Debriefing"

II. Instructor Skills Summarized

The three models presented above demonstrate that nearly all the measured LOS instructor behaviors were significantly related to overall ratings of briefings, scenario implementation, and debriefings. Each model also accounted for the majority of the variance in the respective dependent variables (R² ranged from .76 to .86). Although nearly every measured instructor dimension was related to its respective global rating (e.g., "adherence to published script" con-elated highly with "overall simulation quality"), there were some instructor factors that had powerful independent relationships with these outcomes. For example, orientation to LOS" and "integration of the crew into the brief' were strongly and independently associated with high quality instructor briefings. "Simulator operation and "FA roles and communications" were uniquely associated with effective scenario implementation. And "summarization of key points" and "keeping focus of crew on self-appraisal" each made substantial independent contributions towards predicting instructor debriefing quality. The main point of these findings is that, even though most instructor skills are important, a few instructor skills can be regarded as most important, and should be given special attention during instructor training.

III. Predicting LOS Quality

The last three models presented looked for specific instructor behaviors associated with different aspects of LOS administration. We now turn to a model predicting the four ratings in the "Overall Evaluations" part of the LES that tap different aspects of LOS training quality. The following section explicates a model designed to predict these global assessments LOS quality.

Question #5. Which components of the LOS are linked to Overall quality of the training experience?

Although all of the predictors were moderately to highly correlated with the outcome (ps <.05), only three of the variables were independently related to overall LOS quality (see Table 6). With the overall model being very predictive (R² =.740), the quality of the instructor's debriefing had the highest independent relationship with quality of the overall training experience (sr_xy =. 28). Scenario quality and instructor LOS administration quality were also independently associated with overall quality of the training experience (sr_xy = .26 and. 19 respectively).

Table 6
Regression Summary for Predicting
Overall Quality of Training Experience

III. Predicting LOS Quality Summarized

The main finding of the LOS Quality model is that the quality of instructor skills, especially debriefing skills, have reliable, strong relationships with the overall quality of the LOS training even after accounting for associations with other strong predictors. In addition to the instructor dimensions, scenario quality was independently related to the various facets of LOS effectiveness. Furthermore, this model was highly predictive, suggesting that many of the factors influencing LOS quality were tapped by the LOS Evaluator Survey.

Summary/Recommendations

Findings from this study demonstrate that there is consensus between pilots receiving training and experts observing training on what factors are most related to overall LOFT effectiveness -- skilled instructors and well-designed scenarios are necessary ingredients for high quality LOFT. Nearly all the factors mentioned by Lauber & Foushee (1981) and Butler (1993) had substantial impact on the scenario and instructor quality ratings.

Given these findings, air carriers may reconsider a scenario-dominated approach to LOFT. Without question, effective scenario design is a very important element to consider when training pilots. However, the information now available suggests that scenario implementation (by the instructor) and the surrounding context of the whole training experience (i.e., instructor briefings and debriefings) are at least as important as the scenarios when it comes to delivering high quality LOFTS. Results from the present study are also suggestive of strategies for enhancing scenario design practices and developing effective LOFT instructor training programs. These strategies can be gleaned from examining the most influential specific ingredients that make up the entire recipe for effective 'lot training in the simulator environment.

First, scenarios from the three airlines in this study often did not to have enough workload generated, and the amount of creative problem solving required was usually quite low. Consistent with these findings, decreased scenario workload levels strongly related to decreases in scenario CRM and technical value. If the scenarios in our study were representative of the aviation industry (a question to ponder), increases in workload and allowing for higher degrees of creative problem solving should greatly improve many LOFT scenarios currently in use.

Second, numerous instructor qualities were strongly related to the quality of briefings, scenario implementation, and debriefings. The breadth of skills necessary to successfully train a crew in LOFT is truly phenomenal. Instructor training touching on nearly all of the skills treasured by the LES may be necessary for developing superior LOFT instructors. But how can instructors be expected to do everything equally well? Realistically, the focus of instructor training needs to concentrate on the most important areas relevant to how instructors impact the crews they train.

Fortunately, our study found that certain instructor behaviors were particularly important to trainers' effectiveness as LOFT instructors. Although all the skills measured by the LES should be addressed in instructor training, the following skills should be given special attention and extra time during the instructor training process:

Briefings

1. Orientation of crews to LOS training
2. Integration of crews into the briefing
3. Review of CRM components

Scenario Implementation

1. Operation of the simulator
2. FA roles and communication
3. Adherence to published script

Debriefings

1. Summarization of key points at the end of training
2. Keep focus of crew on self-appraisal
3. Mix of positive and negative critique

by:
Dr. Barbara G. Kanki,
NASA Ames Research Center

NASA Ames Research Center, FAA/Civil Aeromedical Institute, and the Department of the Air Force, Armstrong Laboratory held a joint workshop on Metrics and Methods of Voice Communication in San Antonio, Texas, May 13 - 14, 1994.

The purpose of the workshop was to bring together researchers who share a common interest in language research to discuss their experiences with methods, tools, approaches, etc., which are currently available for furthering our understanding of voice communications. The workshop was organized around three main topics:

1. discourse processes,
2. acoustic processes, and
3. software applications for aiding communication data collection and analysis.

Most of the research reported pertained to aviation operations, specifically pilot and/or controller communication. However, participants represented a variety of perspectives (e.g., investigation, training, cockpit and procedures design). Approaches ranged from laboratory and simulation methods to field and case studies.

As language researchers, our goals were varied and collectively covered a wide range of human performance issues. For example, some of us direct our research toward developing or enhancing training programs; others assess the impact of new technology on task performance; still others investigate human performance issues underlying accidents and incidents in the aviation system. Given each of our particular objectives, we select from a wide assortment of data collection, transcription, and analytic techniques to suit our varied needs. At the same time, we face common issues and problems, and share methods, and metrics. Analysis of voice communications is often labor intensive and soffietimes unreliable. By sharing our experiences with various techniques, workshop participants gained new insights, and left with new ideas, methods and application tools with which to experiment.

Discourse Processes

The keynote speaker for discourse processes was Dr. Herbert Clark from Stanford University who spoke on "Managing Joint Problems in Speaking". Through research, he has addressed the ways in which speakers and hearers create shared understandings in spite of ambiguities inherent in language. He described numerous examples of verbal and nonverbal patterns of communication which convey information about speakers' intentions and thereby help the hearer to correctly understand.

The issues raised by Clark were especially relevant to pilot - controller communication discussed by the next two presenters. The first presentation outlined the most prevalent pilot - controller communication problems in today's aviation system as reported in the Aviation Safety Reporting System (ASRS) database. It was also discussed how this database can be used as a resource for researchers. The second presentation was a description of work in progress at Volpe Transportation System Center in which various forms of pilot controller phraseology are being evaluated in order to identify formats that optimize the information transfer process and minimize communication errors.

We next considered pilot to pilot communication on the flightdeck. Several research projects from NASA Ames Research Center were presented which highlighted flightdeck design and training issues with respect to the effect of automation on crew coordination and communication. These studies asked the following: 1) what design recommendations can be made for better supporting effective crew coordination in advanced technology aircraft? and 2) how can we train crews that will optimize crew communication in these aircraft? In addition, a number of methodological issues were addressed including how to reliably code and integrate multiple levels of speech events, methods for collecting and managing communication data, and the analysis of speech communication in the context of nonverbal information.

Acoustic Processes

Discussions of acoustic processes were initiated by keynote speaker Dr. David Pisoni (and associates, Drs. L. Nygaard and A. Bradlow) from Indiana University. While considering both speech production and perception sides of communication, they discussed indexical characteristics of speech, such as gender, dialect, speaking rate, physical state, etc. and how these properties contribute to how we understand spoken language. Applications of acoustic research led naturally into discussions of its relevance to the work of the FAA in developing voice recognition technologies for ATC training. As the accessibility and reliability of acoustical analysis has improved, the benefits for aviation forensics and the investigative purposes of the National Transportation Safety Board (NTSB) have also increased.

Integrating Discourse & Acoustic Research

The obvious next topic for this consortium of researchers was to consider the link between discourse and acoustic processes, and the next discussions (led by the NTSB participants) described the approach taken by Russian investigators/researchers who have studied the effects of stress on voice communications from three perspectives: 1) the acoustical properties of speech, 2) the cognitive or logical properties of the speech content, and 3) the interactive components of communication between speakers and hearers (e.g., the way in which speakers communicate and relate to each other). When all three perspectives are integrated into a single speech profile, three distinct levels of stress are identified, ranging from increased vigilance to extreme panic, from no decrement in performance to totally dysfunctional behavior.

The work of the Armstrong Laboratory at Brooks AFB was also presented and represented interests in both discourse and acoustic processes. While military operations are uniquely different from commercial transport operations, they share the same types of communication issues that underlie flight safety and team situational awareness. Their work provided yet another source of research experience and communication databases.

Software Applications

The third main area covered in this communication workshop was the presentation and demonstration of software packages that aid in the transcription, database management, coding, graphical display and statistical analysis of communication data. Several researchers described specific packages (OCS Tools, A.C.T, McShapa, and ATSAT), demonstrated how they are used, and discussed their strengths and weaknesses. Presenters made materials and some actual software available to workshop participants.

Further Information

A proceedings of the workshop is in process of being written, and will be made available later this year to anyone interested. If you are interested in more information about the workshop or would like a copy of the proceedings, call or write B. Kanki, NASA Ames Research Center, MS 2623, Moffett Field, CA 94035, (415) 604.5785, BGK@eos.arc.nasa

by:
Earl L. Wiener
Abstract by Peter DeNucci

Editor's note -- The following article is an abstract of a 112 page document by Dr. Earl Wiener, NASA Contractor Report 4547. The full report is available by contacting Dr. Wiener at the University of Miami at Coral Gables, Department of management Science, P. 0. Box 248237, Coral Gables, FL 33124. Tele: 305.284.6595.

Dr. Wiener examines the role of human error in aircraft accidents and incidents, and the methods of managing these occurrences. He uses the term "management" here to imply not only error elimination at its source, the human operator, but also a means of preventing errors, detecting errors when they occur, and preventing errors from adversely affecting the system once they do occur.

These methods of error elimination, or control, are referred to as "interventions." The plan by which interventions are formulated, tested, and implemented is an "Intervention strategy". Traditional methods (basic human factors in product design, training and procedures) and modem methods (computer techniques) are examined.

The study also looks at the possible sources of error experienced by humans operating complex systems, and the resources that can be brought to bear on these problems. The goal being to match methods (resources) with demands (potential errors), to determine whether an effective error intervention strategy can be derived.

Dr. Wiener also introduces six Lines of Defense against human errors. Lines of Defense are seen as a series of imperfect, cascaded filters, each of which may stop an error, or allow it to pass. For any human error or class of errors, there may be a unique set and order of the Lines of Defense. The purpose of an intervention strategy is to strengthen one of more of these lines. They are as follows: (Wiener 1987c)

1. Execution of normal procedures and airmanship, backed up by human vigilance, by the responsible crew member(s).
2. Detection of abnormal condition by other crew members.
3. Secondary indications or displays (sounds, vibrations, etc.).
4. Warning and alerting devices.
5. Detection of error conditions by persons external to the cockpit (ATC, other crews, ground personnel, cabin crew, etc.).
6. Machines that take action of their own (alpha floor protection, auto slats, stick pushers, etc.).

A set of fifteen guidelines for the design and evaluation of intervention strategies is also introduced.

Here is a summary of these guidelines:

1. Is the proposed intervention strategy necessary? Is there a well-defined problem it can prevent?
2. Will flight crews carry out and follow the procedure?
3. Resist politically inspired interventions.
4. Does it interfere with other systems, diminish safety elsewhere, or create a problem for the flight crew.
5. If the intervention involves displays, is the information easily interpretable.
6. Does your hardware or software design conform to accepted standards of human factors?
7. All interventions should be examined for any adverse effect on air traffic control.
8. Flight crews should not be placed in punitive situations or have an added risk of violation.
9. It is economically feasible to management?
10. The intervention strategy should strive to be common to all models within a fleet.
11. Be sure it is the easiest, least costly way to accomplish the intervention.
12. Examine all paperwork associated with an intervention strategy. Does this paperwork actually aid the crew, or does it place unnecessary burdens on the crew?
13. It should be acceptable to pilots or affected personnel.
14. Intervention should not be at odds with other mandated items.
15. Above all, the intervention strategy should be effective. It must be demonstrated to achieve the safety gain for which it was designed.

Summary Of Intervention Strategies As Related To CRM

Training offers flight management the opportunity to intervene in a broad class of problems. The strategy is based on the belief that the class of problems is more easily attacked as a training problem than through discipline, standardization, procedures changes, or the like. A good example is CRM. CRM training offers a remedy for a broad, perhaps poorly defined, class of problems, whose origins are inadequate or inappropriate communication in the cockpit (Wiener, Kanki, and Helmreich, 1993). The intervention comes in the form of a training program for all pilots. At some carriers the training is extended to other personnel, such as maintenance, cabin crews, and flight management. It is not remedial training for a handful of personnel who have been singled out as requiring intervention, nor it is psychotherapy. CRM training is a broad scale approach to social communication-based behaviors and attitudes. It attempts to change cockpit behavior, not personalities. (Foushee and Helmreich, 1988).

CRM training might be an interesting place to apply this list of guidelines, interesting because many of the questions raised by these guidelines could not be easily answered. Furthermore, the value of intervention must by taken on faith, and a few good examples. CRM training at United Airlines, one of the pioneers in the field, was recognized by the captains in two fatal accidents, a B-747 door separation in flight near Hawaii (NTSB 1990a); and DC- 10 crash in Sioux City, following total loss of hydraulics, (NTSB 1990b) as a major factor in their success in saving as many lives as they did. Such examples are difficult to come by, since it is usually problems and failures that get reported, not positive outcomes.

In closing, Dr. Wiener leaves the reader with an assignment. "Design a practical intervention strategy to prevent once and for all, an inadvertent no flap/no slat takeoff. Attack the problem from any angle; use any existing methods and technologies, or any methods that could reasonably be brought into existence. Then test your solution against the guidelines."

by:
Lou Nemeth

One thing organizations do is require compliance. They tell employees what to do and expect conformity. The older the organization gets (in many cases) the better it gets at requiring compliance. After all a model of how we may do things in our company is better than no model at all, right?

Compliance in fact may be a good thing -- just imagine if we all tried to fly jet transports without rules. Too much compliance, however, without employee involvement (commitment) will lead to complacency. One analogy may be the teenagers rule to be home by I 1:00 pm You may choose to demand that he be home every night by a certain time. Sooner or later, one way or another, he must learn on his own, to become a well adjusted adult, that time must be allocated for rest to be effective with daytime responsibilities. The alternative -- a lack of commitment to the rule will lead to complacency as the teenager adjusts to adult life.

Let's look at it from another angle.

In the model above, the event trigger equals something in the organization that creates a concern, say an incident. It's a snowballing dynamic in that the more incidents there are the greater the need, and the more often the need, for change. In this case the change will be a new item on the checklist or a new procedure. The forces of compliance and continent, however, are a balancing act. Too much compliance and there may be a backlash from the troops or worse disregard for the new procedure.

In the model below, both dynamics are of an increasing nature with significant benefit to the whole.

The question then begs to be asked -- What is the application of this to how we operate our aircraft? How may we balance compliance and commitment among thousands of pilots?

1. Find out what the weakness' are.
2. Ask the pilots to be prepared to come to training (ground or flight) as the briefer (the traditional check pilot role). Ask any check pilot when in their careers they were the most proficient -- i.e., when they were preparing to teach.
3. Measure the effectiveness by continually collecting data of what the weakness' are.

This will:

1. Lead to a higher level of learning through self discovery.
2. Balance compliance with commitment.

The challenge:

Getting the traditional check pilots to do it. Of course, we could require compliance.

* Thanks to Pratt & Whitney for this formula in their teams' training course.

by:
David P. Baker, Robert W. Sweezey, & Robert E. Llaneras

Crew Resource Management (CRM) training has been a concern for many years in aviation. While there has been no clear consensus regarding the best approach to CRM training, a pending revision of a Federal Aviation Administration (FAA) Advisory Circular (FAA, in press) delineates three phases of CRM training: awareness phase, practice and feedback phase, and continuous reinforcement. In response to the need for CRM training, InterScience America (ISA) has developed a comprehensive behaviorally-based approach that has been implemented for both fixed and rotary winged aircraft. This approach conforms to FAA guidelines with respect to developing aircrew awareness and providing opportunities for aircrews to practice and receive feedback on CRM skills. Several characteristics of this CRM training are listed below:

• The training is aircraft specific. CRM training is guided, tailored, and coordinated with aircrew members from a specific aircraft.
• The training is behaviorally based. CRM training is based directly upon (i.e., one-to-one compatibility with) a task analysis of critical CRM behaviors in a specific aircraft. These behaviors are used to develop instructional objectives that guide the training.
• The training is organized around unique skills. CRM behaviors, derived from the task analysis, are organized into unique skill dimensions, and these dimensions are the basis for the instructional content.
• The training is modular. Skill dimensions are individually developed into stand-alone instructional modules, which can be utilized individually, together in any combination, and/or integrated into existing training, as dictate by the instructional requirements of a specific situation.
• The training modules include opportunities for practice and feedback. Exercises, targeting specific CRM behaviors, are presented in each module via videotape exercises, role play exercises, and/or analysis of air mishaps. Additional opportunities to practice CRM skills are provided through low and high fidelity simulation.
• The training emphasizes evaluation to ensure training transfer. CRM training includes numerous performance-based evaluation exercises that are used during the conduct of the course. These exercises provide the basis for subsequent validation of the training itself
• The training provides separate training for CRM instructors. Training is provided to CRM instructors with respect to CRM facilitation skills, CRM exercise and scenario development, and CRM assessment and feedback techniques.

To date, this training has been implemented in six Navy (i.e., H-46, H-53, A-6, F-14, T-44, and MH-53) and two Marine Corps (CH-46 and CH-53) communities. In addition, the training was evaluated by an outside evaluation agency (in a CH-53 community) using multiple measures of effectiveness.

These included:

1. Reaction of trainees and instructors to instruction and course content,
2. Performance on attitude and knowledge questionnaire, and
3. Direct observations of teanwork performance in flight simulators.

In the evaluation, a series of 85 performance criteria were developed for use in assessing aircrews flying a simulator mission. Prior to the mission, half the crews received the CRM training program and half did not. In general, aircrews who had received the training demonstrated higher-levels of CRM in the scenario. The evaluation results also produced a number of lessons learned, and areas for improvement in course development were identified. This involved improving and modifying exercises and scenarios and improving various aspects of the instructor materials.

Finally, in a recent real life event, a U.S. Navy helicopter crew, flying a mission from an aircraft in the North Atlantic, found themselves in an "extremis" situation involving unanticipated fog, disorientation, and an inexperienced co-pilot, among other things. The crew had recently completed CRM training course, and credited it with having a great deal to do with the crew's performance in the situation. Subsequently, the pilot of that aircraft wrote an article entitled: "Gone in the Goo" for Approach Magazine and in that article made the following statements regarding the value of the aircrew's pervious CRM training: "I believe that the crewmen's initiative [in dealing with the situation] was a direct result of their [CRM] training". "We're here to tell the tale because of good crew coordination".

The material contained in The CRM Advocate back issues is the property of the contributing editors. No duplication of any kind is authorized without the express written permission of the editor. All rights reserved. For training and information purposes only. The intent of the editors is shared information, through controlled distribution to the benefit of the safety of flight.

[ Home ] [ Up ] [ 93.1 ] [ 94.1 ] [ 94.2 ] [ 94.3 ] [ 94.4 ] [ 95.1 ] [ 95.2 ] [ 95.3 ] [ 95.4 ] [ 96.1 ] [ 96.2 ]