The CRM Advocate

Issue 94.2

Published Quarterly for the Professional Air Crew Trainer

April, 1994


Table of Contents


 Exploring Crew Resource Management in Long-Haul Operations: Initial Design of Survey Instruments and Plans for Data Collection

by: Leslie Partridge and William R. Taggart

The dynamics of the enroute shift change are the focus of the NASA/UT/FAA long-haul crew effectiveness questionnaire. The project goal is to implement a long-haul survey instrument that will identify crew transition performance markers. With a great deal of data collected on general crew coordination issues, researchers and airlines worldwide are now wanting to focus on specific areas of concern. One such area is the impact of the enroute shift change that occurs with augmented long-haul flight crews. Consider this ASRS report:

The Captain was relieved by an International Relief Officer shortly after accepting a flight plan reroute from Air Traffic Control While the relief pilot settled in, the First Officer began setting up a step climb to a higher altitude. On the way out of the cockpit for his rest break, the Captain reminded the First Officer to insert the reroute into the Flight Management Computer (FMC). The First Officer mistakenly inserted the wrong routing, he was one digit off, into the FMC. Some time later, ATC questioned the position of the flight. After the First Officer and the relief pilot verified the true position with ATC, the flight was cleared back on course (ASRS S.R. 2869, 1993).

Questionnaire Development

The current research into ultra long-haul flight operations began at the request of several air carriers concerned with identifying crew member attitudes and behaviors in this previously unexamined area of crew performance. As part of initial data gathering, NASA/UT/FAA requested a database search from the Aviation Safety Reporting System (ASRS). Concurrent with this step was a data request letter I distributed to selected long-haul airlines and air safety organizations world wide that have participated in NASA/UT/FAA sponsored research in the past. During the same period, an open-ended answer type long-haul transition survey was distributed to members of a military airlift squadron.

The ASRS database search yielded 675 reports referencing long-haul flight crew performance incidents. These were separated into two sections: reports referencing long-haul flight crew transition and augmented crew performance/behavior incidents (77 reports); and other long-haul human factors-related incidents (598 reports).

Responses were received from 15 of the 27 airlines and air safety organizations contacted in the initial ultra long-haul data request and from the military airlift squadron. Most of these respondents agreed that this is an area ripe for investigation and expressed strong interest in the results of the project. Responses from organizations and airlines have ranged from the sharing of company procedures to recommendations and/or suggested procedures for use in the crew transition briefing (Figure 1).

Some airlines have specific policies on the handling of the enroute crew others do not, but most are interested in having the process studied and standardized. Regulatory organizations such as the Federal Aviation Administration (FAA) and the International Civil Aviation Organization (ICAO) do not have any written guidelines on the crew transition. In a long-haul-related project, the FAA is preparing guidelines for airlines on the subject of crew rest in flight. The management of rest periods by crews and the activities that take place when crew members return to the flight deck after taking that short-but-refreshing nap are the subject of our investigation. A common theme in the feedback from all respondents is that there is a gap between what takes place during the enroute shift change and what could be done to optimize the transition procedure and enhance safety. Figure 1 shows current and suggested procedures for ultra long haul transition briefings. The transition briefing procedures currently being used, even in organizations with no formal shift change guidance, include topics such as Position and Altitude, Actual versus Estimated Time and Fuel, Variations from Computer Flight Plan, Significant Weather, Communications, and Expected Clearances. The two shift change topics most suggested are Transition Duration, and Specific Change Point.

Transition Duration being either spending a predetermined amount of time during the shift change doing position observation and then briefing, or at least going through a crew transition briefing checklist. Specific Change Point refers to making a pre-arranged schedule for the inflight shift change, whether it is a single position duty rotation or a change of the entire flight crew enroute, as is done by some ultra-long-haul carriers. Since the formation of the crew as a team typically begins in the pre-flight briefing, and the enroute shift change occurs at a time when the crew members are at least a few hours into a long duty day, the establishment of the Specific Change Point (where, ideally, the crew uses a formal Transition Duration procedure) sets clear crew duty guidelines that augment safety, communications, and cooperation.

Survey Implementation And Use Of Results

The current project involves working with selected international airlines toward the goal of developing useful crew transition process guidelines for the aviation industry. We have recently conducted structured interviews from the jumpseat position of an ultra long haul carrier. Based on the interview results, we will use relevant issues exposed in the structured interview phase to revise the content of the Augmented Flight Crew Transition Attitudes Questionnaire (ACQ), (Partridge, Taggart, & Wilhelm, 1993), and the Line/LOS Checklist for Augmented Crews (LLC/AC), (Helmreich, Wilhelm, Kello, Taggart, & Butler, 1991. The survey instruments, thus refined, will then be used in a written follow-up survey of the participating crews. With human factors problems being the largest contributor to aviation accidents, clearly researchers must identify the behavioral markers of the people working in the system and determine which behaviors or types of behaviors contribute to safe and efficient operations. Through the use of the ACQ and the LLC/AC, line check airmen and training departments will be able to identify opportunities for enhanced training and standardization.


 FOQA And CRM - The Next Level Of Safety

by: Coby Johnson, USAir

Accident and incident statistics have proven without question that more than seventy percent of the hull losses and loss of life in the air transportation industry are a direct result of human error.

Further analysis yields the incredible fact that these human errors very rarely are a result of deficient "stick and rudder" motor skills but are in fact due to a failure of a host of cognitive activities. This is where it gets complicated.....

In the human factors community we have almost a perverse need to quantify and dissect these cognitive imperfections into convenient categories, groups, and areas. We often hear references to rather vague descriptions of "interpersonal disfunctionary behavior," a lack (or lapse) of what we conveniently refer to as "judgment," situational awareness problems, "Controlled Flight Into Terrain (CFIT),...one of my favorites (is this an oxymoron?), and so on and so on.

As CRM principles and methodologies evolve in the next several years we must take great care that we don't become bogged down by subjective precepts, definitions and behavioral models that lack empirical validation, realism, clarity or the functional capability to be employed in training scenarios. Achieving a consensus on these issues may in fact be the greatest challenge of all.... recently I heard a speech by one of our most acclaimed human factors notables and I well recall his humorous, yet sobering analogy: "Getting two HF guys to agree on a judgment definition is like trying to stuff a cat in a trashbag!"

A reliable, scientific, evaluative methodology central to the future of CRM development traces its beginning to the NASA/UT/FAA CRM marker system. To date its been the best attempt made at the very complicated task of quantifying cognitive behavior. Our experience with the system though, suggests a very real need to focus on alternative evaluative strategies that more effectively filter subjective observer inputs, resolve standardization/training issues and better attend to the technical resolution of cognitive dysfunction. These alternative strategies may be available to us already in a human performance monitoring and evaluation system that has been in place at many airlines around the world for over three decades. It's called... "FOQA."

FOQA - What?

"FOQA" stands for a Flight Operational Quality Assurance program. Essentially, the system consists of a specially programmed black box called a "Quick Access Recorder (QAR)" which is integrated into a component of an aircraft's avionics system known as the digital data bus. The QAR, which uses magnetic tape or, more recently, optical disk technology, records pre-programmed flight events or "parameters" for analysis later by a data recovery and analysis ground station. Current Digital Flight Data Recorders (DFDRS) used on US Carriers' fleets meet the FAR requirement of recording between 11 to 17 flight parameters (depending on the type certification date of the respective aircraft). DFDRs are required by the FAA to provide essential flight path raw data (heading, airspeed, vertical velocity, etc.) to investigators in the event of an accident.

The digital data (up to 4000 flight parameters) recording, processing and analyzing technologies and capabilities available under a FOQA program provide airline management with a much more extensive and sophisticated flight data collection and analysis capability. In-flight FOQA data is currently being used by over a hundred airlines around the world to assess flight crew performance, training programs, operating procedures, aircraft performance, aircraft operations/utilization/design, aircraft maintenance monitoring, ATC procedures and Airport surface irregularities and design. All airlines polled in the recently published Flight Safety Foundation FOQA report agreed that the FOQA data used for engine monitoring, fuel bum, and manufacturer warranty protection more than paid for the installation and maintenance of a FOQA program on their fleets.

Where DFDR parameter data is static and essentially unobtainable (due to regulatory requirements), FOQA parameters are flexible, easily programmed, and available to the operator almost on a real time basis. Airline Safety system analysts routinely use FOQA data to track safety related trends. For example, one airline programmed the ground analysis station to detect and report aircraft approach paths that were too fast, too steep, and resulted in touchdowns that were too long. Senior airline management shared the findings with the pilot group and pursued an initiative through their flight crew training and evaluation programs which emphasized the absolute necessity of flying stabilized approaches. The results were startling... more than a 50% reduction in "rushed" approaches were reported in the first three months after the program's inception.

Airline Training Managers have developed strategies where FOQA data provides an accurate quantifiable means for validating technical training programs. Analyzing actual crew performance in line operations and comparing that performance with established procedures and standards reveals the successful performance of a training program as well as systemic or individual "exceedances."

An "Exceedance" is defined as an airborne operational event which is conducted outside specific limits programmed for that event. For example, an "exceedance" would be recorded by the system when the aircrew flies an approach at VREF + 25 Knots for > 3 seconds between 50 feet Above Ground Level (AGL) and touchdown. Airlines differ in their internal procedures for handling "exceedance" information and events.... they run the gamut from "Report to the Chief Pilot at your earliest convenience" (not always career enhancing); to mailing a copy of the report to the Captain's home with no action required; to recreating the event with the actual flight crew in a full flight simulator. One of the most effective strategies used by many airlines is to provide the "exceedance" information to a member of the pilot's association who debriefs and, when necessary, counsels the crew. Our British colleagues refer to this latter course of action as a "formal rebuke"... I like that.

So there it is..."FOQA," an HF scientist's database dream. Real-time analysis capability of real pilots in real airplanes in the real world. If you want to study one specific area, operating procedure, environmental influences or any combination of anything pilots do with their flying machines FOQA can provide almost all the answers. It's only limited by the programmer's analytical imagination. Most importantly, of course are the safety and training program validation benefits FOQA brings to the airline. Earlier I referred to the excess of 100 airlines around the world that have taken advantage of this program to varying degrees. All believe their FOQA programs have probably prevented accidents and foster a safer line flying environment than they would have without it. FOQA is currently not in place at any US air carrier. Why?

LAWYERS

We have a great proclivity in this country to sue one another. Its as American as apple pie and baseball. We promulgated the Privacy Act to protect our citizens and simultaneously enacted the Freedom of Information Act (FOIA) and "Discovery" litigation so we can beat them over their heads with big juicy lawsuits. I knew a lawyer once who displayed a large sign on his desk that proclaimed "Sue, the Bastards." When I asked him who the "Bastards" are, he looked at me over his spectacles and said... "Why everybody, of course." Database security is the critical issue for FOQA acceptance in this country.

Airline concerns are focused on increased accident liability and possible punitive actions brought by the FAA for rule infractions revealed by FOQA data. Pilot concerns are focused on possible punitive actions by airline management and the FAA. Airlines and pilots are concerned that FOQA data may become public through the FOIA process or otherwise be made available through legal "discovery" methods.

The FAA has established precedence in the protection of flight data through its current rules pertaining to Cockpit Voice Recorders (CVRs) and DFDRs. Recently the FAA has promulgated rulemaking to protect FOQA databases. The FAA is committed to the FOQA concept and will do whatever is required to ensure both the Airlines and the pilot organizations are satisfied with the database protection before requesting any data analysis information.

FOQA As A CRM Validation Tool

The whole issue of evaluating resource management training and effectiveness is complex and multifaceted. Attempting to objectively measure and evaluate the effectiveness of CRM training of complex human cognitive functions such as "interpersonal behavior" and "assertiveness" may be a noble goal but is wide open to criticism and skepticism. Perhaps it's time for a "paradigm shift" in evaluative methodologies for the CRM community.

FOQA provides the capability to understand exactly what the airplane is doing by evaluating a precise set of parameters at any given time. Evaluation of FOQA data can quite effectively relate when checklists are accomplished, how flight profiles during the various flight phases compare with standard operating procedures and practices., and identify trends that require attention by training, evaluation, or flight standards personnel. Any "exceedance" of preset limits is immediately captured and reported by the system for subsequent analysis and determination of the cause of the exceedance. For example, if the wing flap limit speed is exceeded during an instrument approach, the flight data may be analyzed prior to the exceedance event in an attempt to determine the cause. By overlaying approach chart data with actual FOQA flight data it can be determined at what point the crew began slowing, descending, and configuring the airplane for the approach and landing. Up to this point the analysis is objective and based on factual dynamic flight data.

After this point the reason(s) for the "exceedance" are subjective: unless the flight crew themselves participate in the reenactment of the exceedance. This participation may range from a written response (usually the least effective) to an extremely proactive method used by one European airline where the event is recreated on a large computer screen for the pilots, involved in the "exceedance," to review. The CRT provides a "split screen" presentation with one part showing a three dimensional animated aircraft display, another part showing a flight instrument panel cluster and wind screen (with the actual visibility experienced at the time), and a final part showing the airplane's progress on an instrument approach chart overlay. At any given time all three displays offer an integrated review of what actually occurred, up to and including the exceedance event itself, with the actual airplane flight data. In these scenarios, at this airline, the flight crews get actively involved in the determination of cause which may include a late descent, late accomplishment of a checklist, a situational awareness issue, ATC problems and so on. At any rate, the entire analysis of this event, using this ingenious approach, is almost entirely objective in its scope and validation methodology.

The analytical process just outlined evolved because a government owned airline, in cooperation with the government civil aviation authorities and a relatively small pilot group, were able to forge a system of trust and confidentiality, without fears of discipline or retribution, with the singular goal of maximizing safety. The situation in the US is of course much more complex and fraught with legal and cultural hurdles. But therein lies the challenge to the CRM community. How can we evaluate and analyze FOQA data to best facilitate effective CRM training development and implementation into our pilot training programs? One approach may be what many airlines are already doing in this country... designing and implementing an Advanced Qualification Program (AQP). AQP requires that each pilot task be analyzed through a systematic methodology to determine performance objectives and qualification standards. When CRM factors are applicable to the task they must be identified, properly integrated into the training of the task, and empirically validated on a continuing basis. There is absolutely no better validation of an airline AQP than the analysis of data gathered from line operations through a FOQA program. The capability to use real in-flight data and exceedance information to develop LOFT and LOE scenario is both provocative and dynamic.

The CRM community needs to be cognizant on the benefits to training development and validation that FOQA will bring to the industry. The FAA will be conducting a trial program, probably to begin this year, that will begin to formalize the entire concept. Several airlines in this country are installing QARs and establishing agreements with their respective pilot groups in anticipation of the program's initiation. Flight parameter design, collection and analysis protocols will soon be developed with CRM input. It's essential we reexamine our evaluation methodologies and, if necessary, be prepared to shift our focus and priorities to FOQA as an empirical and objective tool for enabling a dynamic and evolving CRM program. This is key, above all, to establishing and guaranteeing that CRM principles are integrated in future FOQA programs... the next level of safety.


 FAA Sponsored Research Report

by: Eleana S. Edens, Ph.D., FAA HQ

FAA Systems Engineering and Development has sponsored human factors research, including CRM, over the past 7 years. These studies provide data, tools and guidelines to foster the growth of air carrier CRM training and assessment.

Additionally, the research supports internal FAA requirements by affording guidance for AQP initiatives, and supporting FAA Flight Standards and AQP regulatory requirements.

The following is a synopsis of the current FAA CRM research program which is intended to delineate the performing organizations and present a brief overview of the specific research being conducted.

For further details contact: Eleana S. Edens, Ph.D. Federal Aviation Administration, 800 Independence Avenue, ARD-210, Washington, D.C. 20591. (202) 267-7867

University Of Texas: "Design And Evaluation Of Human Factors Training In Aviation".

An update and revision to the Line/LOS checklist is planned. This revised checklist will increase the scope of the information gained, shifting the focus of the previous instrument from employing composite judgment scores to employing specific behavioral indicators in CRM assessment.

In order to define the key elements of LOFT effectiveness a regression model of LOFT components will be developed. Instructor/evaluator training methodology will be redesigned to incorporate recent advances in training techniques.

Carlow International: "Crew Resource Management Tool".

Expansion of the prototype CRM assessment expert system developed during Phase I is underway in Phase 11. This study considers a systematic approach to LOFT development and the CRM assessment process based on observable crew behaviors in LOFT and LOE. This approach employs an instructor centered approach to LOFT development and CRM assessment.

NASA AMES: "Decision Making On The Flight Deck".

The effects of automation on crew shared mental models and team decision making strategies will be investigated. The goal of this research is to identify the differences in crew decision making performance that may be due to automation levels and to develop training guidelines to address these differences. This study may have design implications. Methods to improve pilot decision making in light of natural human strengths and weakness and will also be studied.

Distributed decision making between flight deck, ATC, and airline dispatch departments will be studied and training and procedure guidelines to enhance communication and decision making will be written.

NASA AMES: "Performance Measurement".

Methodologies and tools will be developed to enhance crew performance measurement focusing on implementing several sources of performance data to achieve reliable and valid measurement systems.

NASA AMES: "Non-Verbal Behavior On The Flight Deck".

Plans are being developed to investigate this aspect of crew performance.

Naval Air Warfare Center Training Systems Division.

Guidelines for designing scenarios for Aeronautical Decision Making training will be specified. A model of team situational awareness will be constructed, followed by guidelines for training of pilot situational awareness.

Battelle Memorial Institute: "An Investigation Of Training Issues Concerning The Advanced Qualification Program".

A model AQP targeted to Regional Airlines needs will be developed. Tools to customize a generic AQP to the particular operational circumstances of regional carriers will be available to the air carrier industry.

ERAU And USAir: "Computer Aided Debriefing Station".

A Computer Aided Debriefing Station which will display, in a split screen format, critical flight information along with a replay of the LOFT session will be built and tested.

This current research is in varying stages of completion and technical reports for each project will be available. FAA human factors/human performance research initiatives attempt to address critical, timely aviation safety concerns. Your comments and suggestions are encouraged.


 Previewing As A Technique For Enhancing Situational Awareness

by: Conrad S. Biegalski

Situational Awareness (SA) is a crew attribute which we intuitively understand to have an impact on flight safety. However, not only does the concept lack a common definition, it also defies measurement. For the purposes of this article I will define SA as follows (this definition slightly modifies that expressed by Judge, et al in 1992): The continuous ability of a crew, acting as a single entity, to accurately perceive the relationship of themselves and their aircraft to all factors which could influence successful completion of their mission and to forecast as well as execute tasks based on that perception.

During the time I flew fighters in Vietnam, each mission would be fully planned and discussed. In the military, this type of discussion is referred to as "briefing" the mission. These discussions would include the following items:

  • A briefing by our intelligence experts on the target and the potential threats. This briefing always included a review of terrain charts and often included recent photographs of the target area.
  • A briefing by our weather analysts to explain the weather we could expect to encounter on departure, enroute, and on our return to home base, as well as detailed analysis of the weather for the target area.
  • A briefing by the flight leader on the formation we would fly, tactics we would use in the target area, techniques he selected for ingress and egress, and any other topic which lead deemed appropriate for review.
  • A separate briefing by flight lead on emergency procedures.

In USAF's Air Mobility Command, or "AMC" (previously "MAC", for Military Airlift Command), a series of videotapes are kept at all main operating bases. Most of these videos are of airfields which we transit irregularly or which present unusual terrain features (e.g., Ascension Island) or visual illusions to the pilot (e.g., Colorado Springs landing to the North). Pilots make use of these tapes to preview their landing airport and its approaches if they are unfamiliar with the destination.

In each case, the object is to enhance the SA of a flight crew by providing vicarious experience. In essence, crews are encouraged to "preview" possible events in order to be better prepared for dealing with problems likely to arise during their flight. Examples of these include the presence of hazardous terrain in close proximity to an unfamiliar landing site and the potential occurrence of ground fire, triple A, or enemy fighter attack.

Such previewing is not unknown in commercial aviation. Each commercial flightcrew receives weather and NOTAM information prior to departure. Each crew analyzes the route and the weather to assess fuel needs. In addition, each is encouraged through their company's CRM program to discuss certain other items prior to flight. Once again, this process provides a preview of potential problems and possible solutions to prepare the crew for possible events in their near future.

It is clear that most airlines and flying organizations understand the value of previewing possible events in the preflight arena. However, the question I pose is this: How much formal emphasis is placed in each company's CRM program on the concept that Captains are responsible for initiating at least one and perhaps several "previews" in low workload periods during the conduct of each flight? A parallel question exists: How much, if any, formal emphasis is placed on the fact that previewing may be initiated by any crew member if and when the Captain fails to do so, and that in fact it is the responsibility of the entire crew to ensure that this has been accomplished?

Research demonstrates that the successful Captains talk more during routine portions of the flight than those in which problems needing solution actually exist. Crews led by such Captains tend to make fewer errors than those led by Captains with other communications patterns. Additional research shows that crews led by Captains who either initiate or encourage inflight discussion among crew members experience more success and fewer errors. There are other research indications that the tendency to initiate discussion may vary according to personality type.

So, what exactly is "previewing?" And how shall we teach it? If we are going to formally teach previewing as a technique for enhancing SA, might it be possible to break it down into its component parts so that students could practice it as though following a formula?

As a precursor to conducting a preview, I suggest that Captains and their crews should assess the current workload. If it is not demanding, then they should initiate a team investigation (preview) of the near future in the following manner:

Look forward (into time) using the current flight, scheduled itinerary, and known mechanical status of the aircraft as a basis/framework for the thought process.

Expand this view to include all things (weather, traffic potential maintenance problems, etc.) that could reasonably affect or influence your planned flight/series of flights that day.

Assess each situation as a potential stimulus (a trigger which requires action) and for its potential to disrupt the existing plan by:

Question other crew members as to their:

  • Perception of the potential stimulus. ("Hey Frank, what do you think about that forecast of thunderstorms at XYZ? You ever been in here in the Spring? I haven't.")
  • Their analysis of the dimensions of the stimulus... how serious can this get? ("So, if it gets really crappy at this hour, is that gonna be a problem? Like is it really crowded at this time of the day, or not?")
  • Their analysis of the effect of the stimulus on the original plan. ("Well, once we get more than about 30 minutes past ABC we won't have any good alternates and we'll only have about 20 minutes of extra fuel to hold with. I see that as a real problem! What do you think?")
  • Possible actions the crew could take to either complete the scheduled flight safely or avert undesirable consequences by modifying the existing plan. ("What I think we oughta do is check the XYZ weather in about another two hours. Then we'll make a final decision to continue or divert while we can still make it back to ABC.")

This whole process, of course includes the need to confirm that all who have a need to know or a potentially valuable input are included in the process (expanding the relevant work group to it's greatest reasonable size). In addition, though the sample discussions offered above are quite simplistic, actual discussions need to focus on the interaction of one possible event with another ("With the number two hydraulic system gone we've lost the _____ and if the weather gets really bad this combination could cause trouble with _____).

The reason this works, of course, is that once previewed, events lose their ability to surprise a crew because the crew has prepared at least a basic response to certain adverse situations. Consider the operating manual and its procedures for emergencies and abnormals. Crews study these procedures as a form of previewing a proven, effective response to given conditions which could occur during the conduct of any flight. This suggested system of inflight previewing is no different. Even if the precise series of problems do not occur as previewed, the crew is still in far better position to act with minimum discussion because of prior planning. In other words, through previewing, they have become more situationally aware and thus better able to act quickly and effectively in response to potential problems. The alternative to previewing is to be reactive, rather than proactive... and reactive crews are demonstrated to produce a greater number of errors.


 Report On SAE G-10

Aerospace Behavioral Engineering Technology Committee Meeting

Meeting Date: January 17 through January 21, 1994, Daytona Beach

by: Pete Denucci, USAir

The Realistic Training Subcommittee was the predominant CRM group represented. It was lead by Dr. Richard Adams, from FIT. Others in this group were: Dr. Tom McCloy, FAA Human Factors, Frank Murphy, Chief Engineer Flight Decks, Boeing, Dr. Bill Connor, Captain, Delta Airlines, Regional Airline representatives, ALPA, and NASA.

The FAA is currently involved in a study of Circadian Disrhythmia, or sleep deprivation. They believe fatigue is a contributing factor in aviation accidents. With the help of several university psychologists, they seem confident in getting the documented proof soon. Significant interest was expressed in the line pilot's "average day , regarding rest, nutrition, schedule, etc.

The group summarized fatigue into six areas of study:

  • Awareness Training
  • Alertness Recognition & Management
  • Reinforcement Training
  • Individual Mitigation Actions (i.e. caffeine)
  • Management Pressures & Scheduling
  • Operational Effectiveness (measurement)

Frank Murphy, from Boeing, briefed Boeing's "Human Centered Automation Design Philosophy". He facilitated this through the human factors considerations used on the B-777 cockpit design. The resultant discussion provided three key issues:

  • "Information Management Training" must be taught at the airlines. Architectural protocol should be the thrust of the training.
  • Savings will be achieved by the airlines if "Table Top" FMS Simulators can be used.
  • Negative training will be decreased as a result of these improvements.

Frank Murphy's closing remarks regarding Automation/Integration were:

"Let the Pilot have the final authority as to the operation of the aircraft, and be well trained in the use and limitations of it."

Other Topics Of Interest:

The FAA has developed a computer program that tracks radiation exposure to crew-members. It uses a Heliocentric Potential based on the 11 year solar cycle of galactic radiation. Believe it or not, this will be a user friendly system for airline flight crews.

The Harris Corporation has just sent their first state of the art VSCS ATC system to Seattle for installation in the ATC system. This is part of a $1.5 Billion ATC network. Dick Pitts, the Harris representative, laid out a program in which within 10 years ATC communication would hopefully be as follows:

  • controller types in message
  • message sent to "box" in aircraft
  • aircraft interprets signal and autopilot executes ATC command

Boeing is developing a High Speed Civil Transport, or HSCT. Some of it's characteristics are:

  • mach 2.4
  • 311 feet long
  • 716,000 lbs. GTOW
  • latest technology
  • two pilots
  • simulated visual windscreens

Summary

In conclusion, this meeting provided a friendly environment for the advancement of Human Factors, as they relate to Aviation. The integration of the human being into the ever increasing technological world will be paramount in importance as we venture into the next century.


 After Studying CRM for Years, Where Do We Go From Here?

by: Hugh Huntington

CRM has moved from a United Airlines program to an industry-wide effort. It has seen movement from resistance by most pilots to acceptance by most pilots. Within some airlines, we have seen the concept move from an expert effort to one being taught by line pilots. There is now even some evidence of documented CRM related accident prevention.

If we review the literature thoroughly, we now know that our understanding of CRM is much greater; but the materials to be studied are fundamentally the same as they were ten years ago. An occasional new accident will add new information; but from a case material standpoint, new data resulting from the accident reports is relatively limited.

Many airlines have a thoroughly developed CRM program. Furthermore, most air carrier pilots have completed at least one program. Some programs have even included CRM in basic Flight Attendant training. Some airlines have even been so bold as to teach CRM classes with Pilots and Flight Attendants in the same class.

So What Is Left To Do?

The fundamental assumption of CRM programs is that desirable changes in pilot attitudes will result in parallel desirable changes in cockpit behavior. This is a commonly held assumption by consultants and psychologists relative to individual and organizational behavior changes. Although this assumption is basically valid, experience also shows it has some limitations. First, individuals may be unwilling to change or unreceptive to new information. It is probable that little or no change will occur in the behavior of a person who is overtly motivated to reject new information or unmotivated to improve. Second, trainers and educators in general are not skilled at helping students translate intentions to act differently based on intellectual reasoning into the realization of new behaviors in their workplace.

There are some very practical ways to enhance pilots' continuing education and their understanding of the need for and manifestations of effective CRM behavior.

1. The first step is to review the existing program to see if the values implied in the CRM program are clearly stated. Values might be teamwork, mutual respect, and flight safety. If these are clearly stated, the next step is to articulate specific behaviors which would demonstrate each of the values. For example, if a clearly stated value for CRM is good communication, what would this behavior look like if seen by an impartial observer? Might there be times at which good communication takes on different forms? For example, good communication during normal operations may be quite different from the communication which should be practiced when a Captain is experiencing some incapacitation.

2. The work produced in item one should be reflected in the flight operations manual. An airline with which I am familiar made almost forty modifications to its flight operations manuals to reflect both the attitude and behavior changes expected of its flight crews. In some instances, this was done with minor word changes; in others it required major rewriting of small sections in the manual.

3. Much more work is needed in the development of sophisticated and thorough techniques for reviewing CRM related behaviors during check rides and LOFT scenarios.

4. CRM philosophy is based on a basic value of treating all individuals with respect, regardless of the relative authority positions held. This philosophy is being extended to include the interactions between training or checking staff and pilots or flight attendants. We are beginning to understand that the primary training and education philosophy in this country has been one which encourages the 'expert' to convey information to the less talented. I would suggest that any individual who abuses their position of authority over another equates to the Captain who insists that "I'm the Captain, by God, and don't you forget it". Reworking this value system will take a long time.

The bottom line is that trainers at all levels in the Airline will need to learn not only the theory behind CRM but also how to model CRM behavior in the classroom. This will often mean that they must offer their students less lecture and greater levels of involvement. This is certainly the case for dedicated CRM classes, regardless of their length and frequency. An instructional methodology which emphasizes lecturing airline professionals about CRM is antiethical to CRM principals.

5. Another untapped area of CRM is the further study of leadership styles. We need to understand when and under what circumstances alternative styles of leadership are appropriate. There is a time to be autocratic, and there is a time to be highly participative. Neither is correct all of the time, and both have consequences at a safety performance level. Safe performance from a behavior standpoint is a function of the general atmosphere created by the Captain and crew and by the set of circumstances imposed on them.

6. CRM is so often treated as "Charm School." The absence of words such as "courage" and "risk-taking" appear to be part of the "softness" of charm school. Each of us will change something about our behavior based on the awareness created initial CRM classes. However, each person reaches a point at which new changes in behavior do require courage and risk-taking. At some point, the myth that CRM is "soft" needs to be challenged. The reality is that it is harder to change behavior than to learn new flying skills.

7. Aviation publication is another area with significant unexplored potential for CRM development. Even the NTSB Reporter (which reports on commercial as well as private flying accidents) scarcely touches CRM issues. This became blatantly obvious in a recent review of the USAir incident at Charlotte in which a spoiler cable broke. Not one word was mentioned about the significance of the Captain having set the atmosphere for flight safety in his pre-briefing. A flight attendant who was generally not willing to approach Captains in their cockpits for any reason was willing to do so with this Captain, precisely because a positive atmosphere had been created beforehand. In addition, a non-flying Captain in the back of the airplane also felt comfortable advising the Captain of record of his observation that a spoiler was stuck in the up position.

8. We can see that CRM has tremendous potential for enhancing flight safety when we review general aviation accident data. Many accident chains begin with a poor decision to fly into adverse weather. Commercial aviators are not immune to either the making or consequences of such erroneous decisions. CRM addresses the process in which one must engage in order to arrive at sound decisions, and exercise good judgment based on complete information and defensible logic. It also gives an understanding of the attitudes which can enhance or detract from these processes.

9. Those who are responsible for CRM within the industry may well want to address how strong the commitment to CRM really is. If only one hour per year is dedicated to CRM in recurrent training programs, serious doubt can be raised about the total commitment. Training is an expensive process, and therefore this question may also need to be addressed by the pilot and flight attendant organizations as well. If our unions are serious about flight safety, maybe some compromise needs to be reached relative to the cost of recurrent training.

Serious Acid Test Questions Regarding CRM

1. Can we expect flight crews to operate from their CRM philosophy while in the aircraft in the absence of administrative and management willingness to act according to CRM philosophy?

2. How do we maintain the focus on flight safety and CRM philosophy during the periods of significant union/management stress? Can we have both high stress and high focus on safety?

3. Is CRM nothing more than effective human interaction that, by its very nature, maximizes the potential for full utilization of all resources available which thereby minimizes the potential for error?

4. Can we be fully accountable for our personal interactions with others without deeper reflection and understanding of who we are and how we affect others? A paradox faced by CRM is that the talents that which have traditionally been considered demonstrative of outstanding pilot capability (being data oriented, doing things consistently and professionally, being very orderly and complying with routine and highly refined procedures) may not necessarily be the talents that are required for deep interpersonal reflection. Will we in the long run be able to continue through CRM to significantly enhance the interpersonal effectiveness of flight crews if we continue to reward only the historically valued set of pilot characteristics?


 Automation in Corporate Aviation: Human Factors Issues

by: John Wise, David Abbott, Donald Tilden, Jennifer Dyck, Patrick C. Guide, Lanny Ryan, ERAU

Executive Summary

Automation has the potential to significantly enhance the safety and efficiency of aviation. A series of studies performed by Wiener and his colleagues at NASA Ames identified a number of human factors problems with the use of automation. The major problems involved areas where change to a pre-planned set of actions were required.

Corporate aviation, with its mission of efficiency, needs the advantages that automation promises; but its basic mission requires constant change. In addition, corporate aviation is in a position to take advantage of high technology sooner than the air carriers because they are perceived as important business tools by the corporate decision makers, and thus cost is less of an issue. Therefore, it was hypothesized by this group that corporate aviation was an ideal environment for studies on the effect of automation.

The following is a summary of a two year study that investigated the impact of automation in corporate aviation cockpits.

Approach

The approach followed in this study emulated previous work in the air carrier environment (Wiener, 1989). In general, the study used three data collection techniques to cover all aspects of automation and corporate cockpits. They were:

  • Questionnaires
  • Flight Observations
  • Simulator Observations

Questionnaires were used to sample pilot perceptions of the current implementation of automation in this environment. The forms included space for open-ended comments on both positive and negative perceptions. Flight observations of more than 60 missions allowed investigators to look first hand at the equipment and at how it was used by flight crews in the "real world." Finally, observation of more than 160 hours of simulator training missions allowed investigators to see the use of automation in off-normal and emergency conditions, as well as to determine the manner in which crews in initial and current training were being taught to use it. The body of this report discusses the specifics of these three methodologies and the data collected using each.

Results

There is a lot that is still unknown about the optimal relationship between humans and automation in aviation. However, the research accomplished to date has generated a strong basis for the effective - if not optimal - integration of automation. There is no intent on the part of the authors to imply that unequivocal answers are available for all of the problems identified, but the research described does provide data upon which good initial strategies can be based.

Pilots

In general, the investigators were impressed by the high quality of pilots seen during this study. As a group, they were "good sticks" who were very interested in knowing everything they could about their aircraft and its equipment. Although it can be said that most current corporate pilots have flown without automation for the majority of their careers and are just recently transitionin into automated cockpits, there is already a fairly wide range of experience with automation across the pilot population. The levels of automation sophistication as well as corporate pilot automation experience can be variable even within crews or flight operations. For example, an automation-naive pilot may fly both a basic and a highly automated aircraft daily, or an automation sophisticate may fly a high-end aircraft eighty percent of the time and a moderately automated aircraft only on occasion.

Corporate pilots are fairly unique among pilots of high performance aircraft in that they regularly fly more than one type of aircraft. Such an environment makes the possibility of negative transfer of training a real possibility. This is especially true given the complexity of their aircraft and the fact that most of the differences between aircraft are software based and can only be understood through training and/or experience.

Training

As should be expected in an environment where the technology is rapidly changing, training programs are also in a transition phase. There has been a problem with training organizations being unable to obtain the actual automation hardware. Some training personnel report delays of two years between the release of their aircraft and their first chance to obtain the appropriate avionics. As a result, one can currently find that some simulators used in type-training operate without EFIS (electronic flight information systems) or FMS (flight management systems), and other type-training simulators for similar aircraft with similar levels of automation are fully EFIS/FMS equipped and are available with additional stand-alone part task trainers for FMS specific training. Training organizations recognize this as a problem and are working to find solutions. For example, some advocate developing PC based software that models the operation of the avionics.

Results from the pilot surveys, in-flight and simulation observations, and interviews indicate that great differences exist among corporate pilots in their level of trained proficiency on FMS equipment. These data indicate that FMS training for the aircraft with lower levels of automation has been much less complete than that which is available on the highly automated aircraft.

Good training in the understanding and use of the FMS in a wide variety of flight situations has not been routinely available to corporate pilots. In both actual and simulated flight, the level of trained proficiency has a great impact on benefits gained or consequences suffered from the use of automated systems. The lack of adequate training may lead to FMS errors which may compound operational problems until the pilot finally abandons the attempted use of the FMS and goes back to hand flying.

Software/Hardware Interface

Programming inputs appear to be too complex and frequently require multiple screens. Checking the input for errors made by the pilot-doing-the-programming is difficult, as is cross-checking by the other pilot. Very little of what has been learned about ease of use in the computer environment is utilized in aircraft automation. Interface advances such as Macintosh and Windows are not applied in aviation equipment. A major problem, particularly in the corporate environment, is a lack of standardized keyboard layouts. Perhaps more important is the lack of standardization of the underlying logic of how an FMS is to be programmed. A truly "user friendly" interface standard which allows pilots to rapidly achieve a high level of user proficiency and transfer that proficiency to other systems would reduce many of the problems pilots experience with current equipment.

Beyond the need for standardization there exists a critical need for dramatic improvement in the quality of the interface from a human factors engineering perspective. All of the current FMS interfaces are basically an extension of the old interactive programming languages developed in the late 1960's. They were all developed assuming operation by a computer science professional with formal programmer training and experience. This "programmer' could debug the software while sitting at a desk as the product was developed. Early program languages were not intended for use by the non- programmer interested only in using the application and working in a time critical environment such as aviation. In short, a more effective, user friendly interface is definitely needed.

Awareness of the current operational mode of the system is apparently difficult for most operators to achieve. Only pilots who spend a minimum of 400 hours per year in an FMS aircraft demonstrate a consistent ability to determine mode status. Such experience demands suggest that the feedback to the operator should be improved so that pilots can more easily become aware of system mode and anticipate automated actions.

Basic Coding

Coding of data is a critical issue in effective human-machine interaction. A basic set of standards exists for things such as the color coding of airspeed indicators, but these were developed assuming a steam gauge em cockpit. Standards need to be developed based on high quality empirical data relating coding techniques to human-machine interaction. This total laissez-faire attitude toward coding is most easily noted in the illogical and inconsistent way in which color coding is applied to automated aircraft.

The color coding observed in this research appeared to be more effective as a marketing tool than a decision aid and generally did not follow basic design principles related to human perception of color. Most color codes are decided upon by designer whim and approved by a test pilot unschooled in the principles of human cognition. These decisions are usually based only on personal experience. From a pure human factors point-of-view, much of the color coding that we noted could be expected to inhibit performance rather than improve it. The encoding rules appeared to be based on some cultural stereotypes (red represents danger) and little else. While cultural stereotypes are important, they are far from the most important considerations when using such a salient stimulus as color.

Finally, many of the color coding techniques used were not multimodal. This means that when the color was failed, all of the coding would be lost as well. This situation could be dangerous for any pilot. Even more importantly, color-impaired pilots could suffer major costs in performance ability. It would seem reasonable to avoid discrimination against color-impaired pilots by using redundant codes. This could become very important in the age of the Americans With Disabilities Act, since designs magnifying disabilities could be interpreted as being legally unacceptable.

Knobs And Dials Human Factors

A quick review of the cockpits of the multi-million dollar aircraft we saw reveals a significant number of basic design flaws identified by human factors research programs in the 1940's and 1950's. The adjacent placement of controls that operate dramatically different aspects of the aircraft, such as identical heading and altitude cursor knobs, should be avoided in order to protect against errors such as altitude busts.

A related problem is the design of EADI's. The traditional attitude indicator was shown to be an error-inducing instrument more than twenty years ago. The current interface has been maintained because of the limits in the electromechanical display used, not because of its superior or even adequate human-machine performance. Work needs to be done to allow civilian flight control displays to achieve their potential for providing basic flight information in a totally intuitive format.

The keyboards used to enter data are a significant cause of problems. Not only does the design vary from system to system, but keyboard use often forces simple tasks to become very difficult. For instance, typing in a list of locations to create a flight plan is much more time consuming, distracting, and potentially error inducing than pointing to those placed on an illustrative map. Not only do standards need to be developed for keyboards, different methods for inputting data need to be explored. The military, for example, has used "coolie hat" switches on top of control sticks and yolks to provide continuous cursor control.

Certification

The FAA's National Plan for Human Factors in Aviation has identified the inclusion of human factors engineering criteria in the certification of aircraft and their avionics as a desirable goal.

Operating In The National Airspace System

The impact of automated systems on pilot workload and on reserve workload capacity for non-automation related was observed to be quite dramatic. During high workload situations in the simulator, the level of crew FMS proficiency is associated with the ability of the crew to remain in control rather than falling behind. This effect can be exaggerated by an automation problem frequently criticized by corporate pilots. They claim that the flight profile resulting from the use of autothrottles and VNAV (vertical navigation) does not match the expectations of ATC and thus is not worthless in a controlled environment.

Check Ride Requirements

As it currently stands, a person can get a type rating on a highly automated corporate aircraft with all of the automation turned off. There is currently no requirement for a pilot to demonstrate skill in operating the aircraft the way it is designed to be used, i.e., in the highly automated mode. The authors recognize the need for a requirement that pilots demonstrate the skill to fly an aircraft in a manual mode. However, because the automated mode is the normal mode of operation and because there are parts of the automation that can induce serious errors, it would appear to make sense that they should be required to demonstrate skill in the use of the automated equipment as well.

Conclusions

The pilot-computer interface problems identified in this report can usually be resolved by altering the computer instead. of attempting to alter the human. In many cases, the errors made by pilots are design-induced. That is, if the interface were designed differently, these errors would not occur. Thus, while it often appears easier to alter the human side of equation through training, it is usually most efficient in the long run to alter the computer side of the equation. Based on the research presented here and on the experience of these investigators with similar programs as well as their involvement in the design of advanced cockpits and their experience as pilots and flight instructors, the above recommendations are offered for consideration.


 Letter to the Editors -- from Capt. Bill Traub, Vice President, United Airlines

Congratulations on a superb publication. I just finished reading both issues and found the material to be thought provoking and well presented. You have a great set of contributors.

.... in case you haven't heard, we've graduated our first class in the new 737-300/500 AQP program. In my 30+ years of training this is probably the most significant improvement to airline crew training that I have ever witnessed. It has far reaching implications as the program is totally proficiency based and includes significant CRM training right from day one.

Significant also, on February 1, 1994 we celebrated our 12th anniversary of single visit recurrent training which has always included videotaping of the LOFT sequence. From the article in your newsletter it would appear you're on the verge of taking videotaping to a new level. It sounds like it has great possibilities for the future.

Again, congratulations on an excellent newsletter.
Warm regards, W. H. Traub, United Airlines
Vice President, Flight Standards and Training


The material contained in The CRM Advocate back issues is the property of the contributing editors. No duplication of any kind is authorized without the express written permission of the editor. All rights reserved. For training and information purposes only. The intent of the editors is shared information, through controlled distribution to the benefit of the safety of flight.


 

Home ] Up ] 93.1 ] 94.1 ] [ 94.2 ] 94.3 ] 94.4 ] 95.1 ] 95.2 ] 95.3 ] 95.4 ] 96.1 ] 96.2 ]