Contents
by: Doug Schwartz, Operational Integrity is a term that embraces the six imperatives of every flight crew. To assure that the flight is operated safely; the aircraft and resources are used efficiently; the flight complies with the established schedule, that passengers are provided with the most comfortable flight possible; the flight is operated in accordance with applicable government regulations, and that the flight is operated in accordance with company operating policies (often called operations specifications). Operational Integrity is the responsibility of every flight crew.
The Components Of Operational IntegrityThe performance of a flight crew to achieve Operational Integrity has two components, a technical component and a human factors component. The technical component of crew performance includes knowledge and skills relating to aircraft systems; normal, abnormal and emergency operating procedures; ATC procedures; instrument flight procedures; navigation and charts. This area is well documented, thoroughly trained and rigorously checked. In fact, the airline industry is quite accomplished in the technical component of crew performance. The human factors component of crew performance includes a body of non technical knowledge and skills that influence the technical performance of flight crews. Communication skills; team building; the exercise of command and use of leadership; decision making; management of resources, workload and stress are among the subjects relating to this area. Today, the airline industry is only just beginning to address the requirements of the human factors component of crew performance. It is poorly documented, rarely trained and almost never checked. Technical Versus Human Factors Influence In AccidentsIt is revealing to note that only 30% of crew caused accidents result from technical crew performance failure. However, 70% of crew caused accidents are the result of human factors crew performance failure. The purpose of this paper to put the two components (technical and human factors) into perspective and to suggest that significant operational gains can be achieved through new emphasis on human factors. Success achieved in the area of technical crew performance must be repeated in the area of human factors crew performance if we are to improve system safety. This requires a different view of old issues. As an industry, Perhaps even as a society, we have grown to expect technical solutions to resolve our operating problems. Today however, technical solutions are providing less and less margin of improvement while, at the same time, becoming increasingly expensive. An example of the application of technical solutions to operating problems in the late 1960s and early 1970s was the use of flight simulators. This all but eliminated the risk of accidents during training flights within the airline industry. And, because simulators provided an effective tool for better pilot training, we saw still more improvement in the line safety record. In the 1970s, the introduction of ground proximity warning systems helped improve the accident record by reducing controlled flight into terrain errors. As effective as these systems were, and continue to be, they did not have as much impact as the broad use of simulation. TCAS, terminal collision avoidance systems, is another example of new technical solutions applied to operational problems. TCAS is a valuable resource and a welcomed addition to the cockpit that will help reduce the risk of midair conflicts. However, does the cost justify the benefit? When was the last midair collision involving an airliner? Many cannot even remember that it was in 1989 in the Los Angeles area when an AeroMexico MD80 collided with a light single engine aircraft over Cerritos, California. Is TCAS worth it? Probably yes. But it is an example of increasingly expensive technical solutions applied to operational problems that yield a declining measurable benefit to system safety. Solutions To Be Found In Human Factors Training Of CrewsWe must recognize that human solutions to operating problems will provide more benefit
for less investment than will technical ones. Flight crews require both technical and
human factors knowledge and skills if they are to be as effective as possible. Training in
non-technical knowledge and skills, normally referred to as Cockpit Resource Management
(CRM) training, will influence the technical performance of flight crews and provide an
effective means to improve system safety. This paper was delivered to a human factors seminar hosted by the Soviet Union and sponsored by ICAO.
by: Interestingly, airline aviation has been slow to embrace certain new technologies. The most obvious example is the Electronic Flight Instrumentation System in modem airplanes. Despite all the potential to create displays that are not dependent on the manufacturing technology, EFIS cockpit displays look a lot like the displays they replace. The advantage to be gained from their use is merely an increase in the mean time between failures rather than the creation of more meaningful displays. The guiding philosophy behind this design must certainly be the old adage "if it ain't broke, don't fix it." This hesitance to creatively adapt technologies for operational use has been paralleled by a resistance to the use of innovative technology in the realm of training. That is, until now. It may be agreed that self-discovery is the most meaningful and lasting method of learning. The more pilots garner from involving themselves in training rather than being told what to know, the better retention and understanding they achieve. Professional pilots have a fairly good idea about what is expected of them in simulator training and could probably debrief themselves. Therefore, any tool which facilitates this process could be considered instructive. Instructional technique can be modified to enhance the self-discovery process. There is an existing philosophy of instruction which advocates turning the student into the instructor. It requires students to teach their instructors rather than being taught. (Why didn't I think of that while I was an instructor?) Students must be prepared for lessons because they must teach them. The results of this are astonishing. In addition, technology can be employed in a manner promoting self-discovery. For instance, USAir and many others use video cameras in their simulators when they give their CRM Recurrent Line Oriented Flight Training sessions. The intended consequence of this strategy is for the simulator instructor to become less involved and the students more involved in the debrief The videotape allows pilots to objectively review their own actions rather than seeing themselves through the eyes of an instructor. Evidence for the high level of pilot approval of this technique lies in recurring pilot requests to retain personal copies of the tapes for review. (All carriers who use this technique currently erase them immediately following training to ensure confidentiality of participants). These two tools, reverse debriefing and video playback, have recently been married into a promising technological innovation. This technological solution creates not simply a newer but also a better way to train pilots. A consortium comprised of NASA/UT, Embry-Riddle Aeronautical University, USAir Training, and Resource Options Inc. (a privately held firm in Charlotte, North Carolina) has come together to produce CADS (Computer Aided Debriefing Station). The beta test system for CADS will be running in the first quarter of 1994 at the USAir training center in Charlotte, N. C. CADS takes a simulator data stream and instantly translates it into digital information for replay. After a simulator session, the images are instantly available to instructors and students. Because the information is digital, it can be accessed randomly without the time lag associated with video searching. The CADS display will be a 35 inch monitor with four different views. Three of the four views will be computer animations, the fourth will be digitized video. The first is a recreation of the flight instrumentation. The second presents a plan view of the aircraft in flight. The third view shows a profile view of the aircraft in flight. The final view is a digitized video image of the pilots in the cockpit with the attendant audio taken from the video cameras already installed in USAir's simulators. All four views are available simultaneously on the screen or any view can be enlarged to cover the whole screen. As previously noted, there is a benefit to pilots in being able to view themselves
through their own eyes as objectively as possible. In addition, concurrent viewing of
digitized video along with instrumentation and graphic displays of aircraft attitude
encourages relating technical performance to human factors. Finally, the bottom line
benefit to the airline organization is more effective flight training.
by: Why Line Oriented Flight TrainingThe FAA and Aviation Community have recognized the importance of placing more emphasis on human factors generally, and Crew Resource Management (CRM) in particular. Thus, training to produce and improve crew performance is changing. CRM characteristics such as those described in CRM Advisory Circular 120-5 la are being reexamined and systematized. These CRM skills must be developed, recognized, and integrated with technical skills to allow complete assessment of total crew performance. The introductory CRM training the flight crewmembers have experienced is similar to the foundation of a building; it is an essential structural part, but by itself the foundation has limited operational utility. If CRM training is to be operational effective, it must be built into other training steps and activity ties in a systematic way. The functional environment which provides the opportunity to combine the CRM and technical skills is the Line Oriented Flight Training (LOFT). The LOFT is an environment which is structured to allow and encourage the application of technical and CRM concepts to a situation or situations. Instead of being programmed with a solution, the crew can manage the operational environment to learn its limits, properties, and operational relevance. The LOFT Design Focus Group was chartered by the ATA AQP sub committee and charged with the development of standards for the design and conduct of LOFT under the Advanced Qualification Program (AQP). Though this is the specific focus of our group, the efforts will also benefit those airlines who are now or will soon be developing LOFT programs for recurrent training outside of AQP. The original charter was expanded to provide guidelines and recommendations for change and revision of the Line Operational Simulations (LOS) Advisory Circular 135b. In addition to LOFT design, the LOFT Design Focus Group developed a philosophy embracing all the various elements of LOS including definitions applicable to LOFT design and conduct. This report briefly covers this philosophy and an introduction to LOFT design. The Focus Group working paper has full details of the LOFT design and it's conduct. LOS PhilosophyThe overall objective of LOS is to improve total flight crew performance, thereby preventing incidents and accidents during operational flying. Included under LOS are Line Oriented Flight Training (LOFT), Special Purpose Operational Training (SPOT) and Line Operational Flight Evaluations (LOE). Although the training objectives of each of these events is quite different, the LOS Philosophy provides the general design concepts which apply to all LOS. For specific details of each LOS, type refer to AC 120-35B definitions and guidelines. When we discuss CRM we are looking for techniques which will allow and encourage crews to become better problem solvers and resource managers. This development of interpersonal crew skills increases the safety and total effectiveness of the flight operation. The LOFT context must be structured to allow and encourage CRM behaviors to emerge and the crew to become aware of them, i.e., the scenario must last long enough for crew traits to become evident and should require CRM skills to be displayed in response to circumstances. Similarly, scenario construction should focus on the range of CRM and technical objectives integrated into a training program. In theory, a productive scenario could take place without the aircraft leaving the gate; involving problems with weather, discussions with dispatchers, ground staff, preflight documentation, etc. Except for LOE, which involves crew evaluation, all Line Operational Simulations are training events in which crews can enhance their CRM and technical skills through practice. Thus, the evaluation of LOFT for CRM skill performance would be highly counterproductive. The total crew concept allows crewmembers to use their full resources and creativity to create a complete learning experience. However, LOFT learning should not be artificially stress free; crewmembers should maintain reasonable performance parameters applicable to their phase of training. This would not only allow, but require handling failures and their consequences. If the LOFT instructor identifies crew member performance deficiencies additional training or instructions will be provided. Introduction To CRM LOFT DesignThe focus of LOFT is on how crew members practice and become equipped with a better set of CRM skills to deal with the various types of situations faced in the line operation. It is best if LOFT scenarios are designed so that they are not only operationally relevant and believable, but are also a good test of the cockpit crew's teamwork skills. LOFT training is systematic and is intended to simulate actual problem situations on the line that require good crew skills for effective resolution and decision making. LOFT scenario design is enhanced if-a good strong foundation in CRM concepts and awareness exists within the organization. It is important for the effectiveness of LOFT to create as much realism as possible. LOFT design and evaluation guidelines are being written to maximize benefits by enhancing line realism. LOFT design should include preflight activities and details such as review of trip paperwork and manuals and conduct of communications, etc. These should be carefully presented if they are part of the scenario. These requirements do not preclude the employment of scenarios using short segments beginning or ending in an enroute environment, if the objectives of the LOFT can be met. If the scenario is designed to begin in an enroute environment, enough quiet time should be present for the crew to become acclimated to the flight routine. One misconception that periodically occurs is that LOFT training means that the workload of the crew in the simulator is continuously increased until the crew becomes overloaded. This is not the purpose or intent of LOFT training and can actually help to defeat its effectiveness if designed in this way. LOFT scenarios are best if they are straightforward and if the entry point is relatively simple. For example, choosing a departure airport that requires a good, effective preflight briefing even under the best of conditions, might be one way to begin. An entry point that allows the crew to choose from different options, for example, is very useful. One scenario can have a wide variety of outcomes and choices depending on the decision and course of action that a crew undertakes. Again, the scenario should be realistic, and the situation should be one where the crew lives with whatever problems they have until the situation is either resolved or the airplane (simulator) is back on the ground. LOFT scenarios should be designed to foster an environment where free and open communication is practiced. This encourages the crew to provide necessary information at the appropriate time (for example, initiating checklists and alerting others to developing problems). Furthermore, the LOFT design should encourage active participation in the decision making process and questioning of actions and decisions by all crewmembers ensuring that decisions which are made are clearly communicated and acknowledged. Alternatives To Full Fidelity SimulationLOFT has to date been carried out in sophisticated, type specific flight simulators. This requirement is costly and limiting to the broader benefits to be gained by practical CRM training. Many operators are thus denied the benefits of such training. The focus group analyzed the needs and objectives of, what is called, CRM Practical Training and concluded that some training objectives could be effectively met in training environments less complex than full flight simulators. In order to broaden the application of CRM principles, which has generally required full flight simulation and termed LOFT, it was necessary to define CRM Practical Training. Definition: Objective: Development Process: Elements of the Process:
The intent of the additional step, Step 4, is to identify all resources and information that must be available or limited to create and maintain the frame of mind necessary to facilitate realism. To accomplish this, the following are examples of factors that should be carefully considered for each training objective. Examples of Objectives:
The answers to these questions will result in a set of significant factors that will suggest the lowest level of sophistication required to maintain the appropriate degree of realism. After the significant factors have been determined, focus can move to scenario development where the information is used to produce an effective scenario. Recommendations:
StatusThe LOFT Design Focus Group is currently in the process of completing the working paper
for the AQP committee. The purpose of this document is to be a general guideline for LOFT
design and conduct. We are also nearing completion of the draft of recommended changes and
additions to the LOFT advisory circular.
by: The SAE G-10 operational communications subcommittee was formed in January, 1993 to focus on issues related to the effective verbal communication between pilot and air traffic controller. This group is chaired by Linda Connell of the Aviation Safety Reporting System at NASA Ames Research Center, and is composed of airline representatives, pilot union representatives, FAA, NASA, and other government researchers. A strong effort is being made to get more adequate operator and union representation from the air traffic controller community. During the first meeting (January, 1993), the group members agreed that the subcommittee's first objective would be to define a relatively narrow focus since the topic of pilot/ATC communication is very broad and encompasses numerous related topics. The second objective was to identify the type of mechanism that would present the end results of the group's effort. By the end of the first meeting, it was determined that an ARD "issues" document would be developed within the scope delineated in the introduction / charter statement. The Charter StatementThe second meeting took place in August, 1993 and the first order of business was to review the introduction / charter statement drafted at the January meeting. Incorporating a few minor changes, the statement currently reads as follows: ARD # 50045: Issues Impacting Effective Operational Communication Within the National Airspace System.
The ARD DocumentIn its entirety, the ARD document will contain an overview of past, current, and future efforts in operational communications with updates on progress toward solutions of pilot / ATC communication problems in the National Airspace System. Following SAE guidelines, the ARD will be an "issues" document, updated after one year, with a life span of two years. At that time, it may progress toward becoming an ARP document of recommended practices. Focus On Issues: Not ProblemsIn addition to discussing substantive issues and sharing information related to operational communications issues, the group also discussed how the ARD document should be structured, what it should and shouldn't include, and how to ensure that all relevant perspectives are represented. A concern voiced by many in the group was that pilot/ATC "problems" have been known and discussed for many years. Therefore, it is essential that this document focus on underlying issues and not simply restate the usual list of problem areas. Fundamental guidelines for outlining issues were discussed and include the following: Discussion of issues should:
Issues should be tied to operational errors and other indicators of stress on the system:
Standardization Issue must be resolved:
Issues and UsersIn addition to the topics listed above, a first cut at identifying issues was made during the August meeting. These issues were conceptualized as a list of areas that are "lacking" in some respect; for instance, the lack of cross operational education and training, the lack of standardization and personal accountability, the lack of communication, standard definitions and metrics related to problem areas across the industry. In addition, a list of users was generated that includes the following: pilots (91, 121, 135), and controllers (tower, TRACON, en route). Future PlansThe group will continue to discuss the issues at the next meeting and draft an ARD
document that satisfies the guidelines described above.
by: Southwest region ATC has formed a working group of controllers, managers, and union representatives known as ASSET which is dedicated to studying and reducing loss of separation problems. The ever increasing level of system demand and technological capability suggests a need for continual improvements in efficiency. However, we cannot ignore the very real danger of catastrophe inherent in limitless increases in capacity and decreases in separation standards. Therefore, ASSET would like to explore possible changes in training, scheduling, procedures, and policy which might improve ATC effectiveness without compromising safety. NASA/UT will be collecting confidential information in a structured format from Southwest Region employees in a variety of positions. Much like the current Aviation Safety Reporting System, we will be independently studying trends reported by those with concerns about safety risks that might be preventable. Sadly, the ASRS system has not been heavily utilized by ATC controllers due to the absence of immunity benefits enjoyed by pilots under the system as well as a lack of system understanding. As all operators are aware, an environment which is unforgiving of human fallibility can "set someone up" for charges of neglect or incompetence regardless of the honor of their intentions; this is as true for controllers as it is for pilots. Unlike ASRS, the majority of data to be collected by NASA/UT will address non technical aspects of performing Air Traffic separation duties using a simple yes/no questionnaire. In addition to broadening their focus regarding mishaps to include human factors, the ASSET team has expressed interest in hearing the other side of the story: the operator's view-point. In the past, ATC has frequently been unable to determine what intentions, constraints, or concerns might have influenced pilots during operations which eventually led to charges of "operational error" on the part of the controller. Knowledge of these things might allow Air Traffic Control facilities to take corrective action aimed at incident prevention. The Southwest region has encouraged NASA/UT to obtain and deidentify voluntary reports from operators related to specific cases which have caused concern excluding those which might involve pilot deviation. Their objective is to learn how the goals of pilots and controllers might be in conflict and how a more effective team dynamic might be established both between pilots and controllers and within controller teams to support safer operations. We would greatly appreciate the voluntary participation of pilot groups in this cooperative effort which involves no risk of disclosure to anyone who contributes information. Interested parties should contact Sharon Jones at NASA/UT in Austin, Texas. Information obtained would be used ONLY by NASA/UT to enrichen understanding of particular events. Feedback would then be given to ASSET about general trends just as it has already been given to many of you by NASA/UT in regards to fleet statistics or the effectiveness of training programs. This project offers an opportunity for air carrier pilots and management to have
meaningful input into improvement initiatives in the Air Traffic Control system. It also
suggests the potential for operators in gathering objective information from Air Traffic
tapes and controller responses to questions about working conditions in order to clarify
the circumstances surrounding aircraft mishaps. Finally, after testing in the ATC
environment, the human factors questionnaire under development can be refined for use in
the study of air carrier incidents where crew dynamics play a role.
by: Synergy is certainly one of the bywords of CRM. One of the things that has become obvious to me over several years of facilitating the implementation of CRM at my airline is that our training tapes could use a little synergy. Professor Jim Blanchard, Embry Riddle Aeronautical University and Captain Lou Nemeth, Chief CRM Facilitator for USAir in Charlotte, have gathered their respective expertise and have decided to create CRM films that would be both informational and entertaining. Mike Simmons, CRM facilitator for USAir and Line Pilot, brings considerable video-graphic and dramatic production experience to the project. Captain Marshall Rogers, also a CRM facilitator for USAir, brings a couple of years CRM experience along with a general good sense of humor to the table. These four fellows are marrying subject matter expertise, university level depth, and
years of combined airline experience with film making expertise to create videos that will
both educate and entertain. They hope that by developing creative presentations, the
material will remain indelibly etched in the corporate consciousness of all who view them,
The films will be available beginning in summer 1994.
by: AbstractCRM is expanding beyond the cockpit and beyond national boundaries. It would be premature to apply research and training designed for pilots in one country to other occupational groups and nationalities without first gaining an understanding of the performance norms in the new groups. These norms encompass issues of leadership and followership, communication styles, work values, the use and flexibility of regulations, and the concept of "team". The NASA/UT group Is presently investigating these issues in different work environments and in different countries. Preliminary results and implications for training are presented. CRM Varies with CultureThe NASA/UT research group has been studying aviation human factors for over 15 years. Much of that time has been spent in close collaboration with commercial airlines developing, evaluating and refining Crew Resource Management (CRM) concepts and training. In one sense the group has acted as a clearing house for airline personnel who wish to share training materials and lessons learned on the front line. The anecdotal feedback and the many ongoing evaluation projects have helped us recognize and resolve many problem areas. As CRM is being investigated and applied in ever-widening contexts, it has become increasingly clear to us that there is no one universal pattern of behaviors that can be called CRM. To explain this statement more clearly, I need to go back to the original definition of CRM. The memorably pithy definition of CRM provided for us by John Lauber is "the effective utilization and management of all resources - information, equipment and people - to achieve safe and efficient flight operations". In essence this definition captures the three components of a model of performance in the aviation environment. The broad outcomes are safety and efficiency, and the inputs include individual, group, organizational, regulatory, national and environmental factors. CRM represents the intervening process - the means of utilizing and managing the resources put forward by the inputs in such a way as to optimize the outcomes. If we know the desired outcomes and the inputs we have to work with, then the optimal intervening processes can be designed. Here we are talking about the specific actions and behaviors that represent the "how to" processes of "effective utilization and management" as mentioned in Lauber's definition. In this paper I will present the case for national culture as an input factor, showing how optimal CRM practices will vary as a function of the culture. Before going any further, let me provide a simple working definition for culture. Culture can be defined as the values and practices that we share with others that help define us as a group, especially in relation to other groups. There can be occupational cultures (e.g., what is it about pilots that makes them different from dentists or dispatch?), organizational cultures (QANTAS and Australian), and national cultures. National culture has been a relatively recent addition to our own model of input factors because initially CRM was only practiced in the U.S. and other Westernized countries. Within these relatively homogeneous national cultures, the effect of culture was generally uniform and therefore masked. But as CRM expands beyond the cockpit and across national boundaries, we are now recognizing that the CRM practices which we held to be universally applicable are indeed culturally influenced. I would like to provide a brief sampler of the cross-cultural social psychology that has helped us understand and propose hypotheses about group based work behavior in different countries. For the sake of depth over breadth, the discussion will highlight differences between the U.S. and some Asian countries. (In many cultural aspects Australia is considered very similar to the U.S.) Some examples of training successes and failures will be provided to demonstrate that cultural differences are alive and well and operating within commercial aviation. I will finish with a brief discussion of our plans for further study in the area of national and sub-cultures. The most cited work in cross-cultural psychology is Hofstede's extensive study of IBM employees in over 50 countries. Hofstede was able to identify four dimensions of cultural variation which differentiate and cluster national cultures. The most researched dimension has been Individualism-Collectivism. The U.S.A. and Australia were ranked the top two individualist cultures in Hofstede's study; most Asian and Latin American countries were located at the opposite collectivist end of the dimension. A second dimension is Power Distance, and it is conceptualized as the extent to which the less powerful expect and accept that power is distributed unequally. Power Distance is inversely correlated with Individualism, i.e., countries which are high in Individualism tend to be more egalitarian, while collectivist cultures tend to be more hierarchically structured. The U.S.A. and Australia were ranked 38 and 41 out of 53 countries and regions in Hofstede's study, suggesting moderate to low power distance. The location of a country on the individualism dimension will strongly affect the level and nature of communication and social interactions in the work group. Considered within the aviation context, individualism refers to the level of independence that each crew member prefers. In individualist cultures, pilots aspire to the "right stuff'. Heroically self reliant, "white scarf blowing in the wind", the stereotype of the solo flyer has persisted. Crew members in individualist cultures, prefer individual responsibility and minimal interdependence, with open and direct communication only as needed. This attitude is expressed most vividly in the anecdote of the U.S. captain who told his first officer to "just look out the damn window and I'll tell you when I need you". Crew members in collectivist cultures prefer greater interdependence and group oriented striving, with more indirect and a greater volume of communication. The emphasis on harmony is revealed in a story about a multi-cultural South-East Asian airline. The airline employs pilots from eight nationalities. The only time conflict has erupted (ending in a fist fight) in the cockpit, the two pilots were from Australia and New Zealand, not from any of the collectivist Asian cultures. Preferred forms of leadership and followership also from this distinction, especially in light of the inverse correlation with power distance. Much has been written about leadership / followership, and there does seem to be some convergence in the literature, from the U.S.A., India and Japan for example, that an effective leader is someone who is both task (instrumental) and relationship (expressive) oriented. However the specifics of effective leader behavior may indeed vary cross-culturally. The same behavior may seem invasive and inconsiderate in one culture, but benignly paternalistic and encouraging in another. Power distance was the focus of a 1984 analysis of air crew communication within 151 civilian aircraft crews in six countries. The authors argued that safe cockpits have "warm climates" (defined as a feeling of general good fellowship and helpfulness) and open communication. To that end they noted that power distance was positively correlated with the perceived importance of warmth, and negatively related to open communication. In other words, pilots in high PD countries were more sensitive to the importance of congenial relations in the cockpit, but they perceived and accepted barriers to open communication due to status inequalities. On the other hand, the cultural value of egalitarianism conveyed in low PD countries was strongly related to perceiving no barriers to communication due to status, but the value of a warm cockpit climate tended to be under-rated. This summary of behaviors may seem academic and removed from the real world of CRM and training. But let me take three examples from recent CRM training experiences to illustrate the pervasive influence of national culture in CRM. The first involves a reasonably common occurrence. An Asian based airline contracted with a U.S. based training company to introduce CRM to its pilot group. The first stage was CRM awareness training. The pilots were polite and deferential to the trainers, and acquiesced to all that was proposed. Being a CRM course originally designed in the U.S., assertiveness training was one component of the course content, and this material was also accepted without question. The trainers consulted with senior management, expressing their concerns that the pilots were unresponsive and that training might not transfer to the line. A member of senior management attended the next seminar and formally gave the pilots permission to deviate from their national culture. The next stage of the training was Check Airmen training. The airmen were to be trained in evaluation techniques and methods of feedback and critique. It was at this point that the training completely unraveled, with the usually congenial pilots declaring they could not do what was asked of them. This example is rich in cultural imperialism and cultural mutiny. First the consultants promote a model of optimal CRM behaviors. Masquerading as universally applicable, the behaviors are in reality only "optimal" in their host culture. Tolerance and polite deference for the trainers' odd ideas gives way finally to open disagreement in the face of being asked to provide direct specific feedback (in essence, criticism) to members of their own in-group - the extended family or organization. In what will be a face-saving strategy for both senior management and the consultants. I suspect the remainder of the training will be provided as specified in the contract. Another more positive example of cultural influence is seen in the contrast of two CRM courses developed and conducted in-house at two different airlines - one based in South East Asia, the other based in Ireland. As part of the training evaluation at both airlines, participants were asked what about the course they particularly liked and what could be improved. The Asian participants liked the teamwork and communication modules, they wanted the course extended to four days, allowing for more role plays and discussion, they wanted an expert rather than just a trainer to lecture them on stress, and many asked that the pre-course manual be distributed a couple of weeks prior to the course. The Irish participants thought the teamwork module "waffled", they wanted the course shortened to two days to avoid repetition, and they liked the stress management techniques and the analyses of air crashes based on video reconstruction. There are two explanations for these disparate results, and both are culture bound. The first explanation suggests that participants resonate to, and therefore rate more highly those aspects of training with which they are most comfortable. The aspects of the training that are within the trainees' "culture comfort zone" will be tolerated and accepted to a greater extent than those aspects which are outside the comfort zone. The Asian participants value group harmony and the methods by which that harmony can be achieved. Consequently they endorse the teamwork and communication modules. They also wanted a longer course, which would allow for greater discussion leading to group consensus on issues (and therefore harmony). Incidentally, in assessing group process factors with the Cockpit Management Attitudes Questionnaire, the Asian carriers nearly always score higher on the Communication and Coordination scale than do U.S. or European carriers. The preference for an expert to lecture on stress rather than just a trainer may reflect the value placed on authority in high power distance cultures, and the elevated role of the teacher. The request for early release of the pre-course manual for adequate preparation also reflects the more disciplined educational style by which many of the participants were raised. Rather than our Western notion of adult learning, which requires high participant involvement through interaction and even entertainment, the Oriental model is more lecture oriented, with the teacher as the unquestioned authority. Thus, the delivery and presentation as well as the content of CRM training can be culturally influenced. The reaction of the Irish participants to their course content could not have been more different. The Irish are from an individualist culture similar to the U.S.A. but with lower power distance (ranked 49 out of 53). The group endorsed flattened command structures to an extent we had not seen with any other airline. Also, the pilots were one of the first groups to be administered the new, cross-culturally designed Flight Management Attitudes Questionnaire (FMAQ). With information gained from other items on the FMAQ, we can say most emphatically that the Irish do not need assertiveness training - they will speak up! In line with Redding and Ogilvie we also noted that the willingness to be assertive and pursue open communication may take precedence over a concern for group relations in this low power distance culture. The teamwork and communication modules were considered "waffly" - soft, group-oriented material that seemed to repeat itself. The material was most likely outside the "culture comfort zone" of the Irish pilots, and so they tended to perceive it as long-winded and even trite. On the other hand they enjoyed and wanted more stress management techniques. In an individualist culture where effective performance often equates to performance without assistance, and individuals are monitoring themselves and not others, asking for help is difficult. It is easier to learn the techniques of personal stress management, and continue unassisted, than it is to learn the culturally-incongruent communication and teamwork skills which would allow for greater sharing of tasks and workload distribution. The other content area which the Irish participants enjoyed was not role-playing or direct interaction amongst group members, as mentioned by the South Asian pilots, but rather the analysis of crash videos. These videos invite analytical thinking and blame apportioning in a third-person, personally non-threatening environment. The analytical, reductionist approach to problem solving is very Western in its orientation, and is therefore very comfortable to the Irish pilots. It is easier to debate the causal factors than it is to personally engage in team building skills, and directly interact with others. In a culture where being right is more important than being agreeable, there is little effort to build consensus. I mentioned there were two explanations that fit the results from these two groups. The first was that participants resonate to and are more accepting of training material that is culturally congruent, both in content and delivery. There is of course a simpler explanation. It may be that the two courses were just very different, that the teamwork module presented in the South Asian course was terrific while the Irish teamwork module was just plain lousy. The crash video analyses may have been handled better by the Irish trainers than their Asian counterparts. If this is the case, and I suspect there may be some truth in it, there is a very simple cultural explanation for it. The in-house trainers are members of the same culture as the participants. Just as the participants have their "culture comfort zone", so too do the trainers. I am speculating but I wonder if it is not the case that trainers resonate to and therefore develop better modules for those content areas about which they are most comfortable. The teamwork module developed by the South Asian trainers may indeed be a better module because the trainers were more comfortable with the material. If this is the case then we can predict an additive interaction effect. If the CRM material is culturally congruent, or at least within the "culture comfort zone" of both the trainers and the trainees, then the training will be especially effective. If trainers and trainees are both aware of their "culture comfort zone", then the design, the implementation, and the eventual acceptance of CRM may be a smoother road for all. Cultural differences have been discussed here in broad national terms, but they can also be discussed in relation to organizations and occupations. For example, pilots with a military background may have different values regarding appropriate flight deck behavior from pilots with commercial backgrounds, and flight attendants and other occupational groups may differ from pilots, in the values they hold to be important. We are confident that these cultural differences impact work behavior, and we continue to investigate their influence. Future Directions: Research and TrainingAs noted earlier, we have developed a questionnaire for cross-cultural use. The Flight Management Attitudes Questionnaire (FMAQ) contains items from an earlier questionnaire of cockpit attitudes, the CMAQ, so that longitudinal analysis can continue where appropriate. It also contains previously validated items which capture cross-cultural differences in broad work-based values (taken from Hofstede's Value Survey Module, 1982, by permission), other items suggested by the cross-cultural literature, and more specific items written expressly for the aviation environment. There is also a sub-scale which rates attitudes towards cockpit automation which will be asked of the pilots. Data collection has begun on this project with participant airlines from Europe, the Pacific Region, South and North America, and Asia. As CRM programs expand beyond the cockpit and across national boundaries, it becomes
incumbent upon us to understand the host culture, its environment and its work norms. It
is not acceptable to export the "optimal work processes" from one model of
performance to another. Rather, a detailed examination of all the relevant input factors
and the desired outcomes will in turn determine the design of those optimal processes.
Failure to do so is akin to prescribing a patient's medication, based on another patient's
diagnosis. Sometimes it works, more often it doesn't .....
by: A large number of crews at American and other airlines have fallen into a common trap - the altitude deviation. The typical incident involves two common factors. First, one pilot does not hear an assigned altitude which is misheard and incorrectly inserted into the altitude window by the other pilot. Second, the misheard clearance very often is to ten or eleven thousand feet. With a revision to each Aircraft Operating Manual, American implemented setting verification procedure developed by USAir and ALPA. That group had determined that by far the majority of altitude deviations occurred because the wrong altitude had been set into the alerter window, rather than by a malfunction in the altitude alert or capture systems. Certainly, some were due to malfunctions or to mode changes or errors, but these were a small minority. That led them to develop a procedure where altitude clearances were deliberately verified on insertion into the alerter window, and that policy has been adopted by American with minimal changes. The procedure is typified by this quote from the B-757 manual: "Both pilots should be aware of all traffic communications and clearances. Upon receiving an altitude clearance, the pilot-not-flying will normally set [and arm] the clearance altitude on the AFDS panel and then point to the altitude window until the pilot-flying acknowledges that the setting agrees with the clearance by stating the clearance altitude. If there is any question regarding the altitude clearance, it should be verified with ATC. When approaching within 1000 feet of the clearance altitude, the pilot-not-flying will state, e.g., leaving 23 for 24 [armed] and the pilot flying will verbally acknowledge. [words enclosed in brackets appear only in the DC-9 and DC-10 manuals]". The pilots at American were asked to do three things:
These procedures have now been in effect for 6 months. Have they had any effect on altitude awareness at American? AA looked at two sources of information:
ResultsWhenever a change is implemented to solve a problem, it is always difficult to determine whether the change in procedure caused a change in the environment. There are always alternative explanations. Even if the rate of deviations suddenly drops to zero following the changes in procedure, the drop could be attributed to the procedure, to other things that are going on in the airline, or simply to the process of calling attention to the frequency and danger of altitude deviations. So we are not naive or bold enough to conclude that the procedural change alone has reduced the altitude deviation rate. Consider these statistics on incidents resulting in violations filed against AA pilots for altitude deviations during the past five years (provided by Bruce Bickhaus, APA, see chart).
As you can see, altitude deviations resulting in FAA enforcement action were occurring at a rate of two each month during 1991. This may not seem so serious given the increasing number of flights over the same period of time, but a single altitude deviation could result in an accident, and no one wants to go through the violation process. In any case, like many other airlines, American was seeing the rate of altitude deviations increase. That appears to have changed in 1992, when the rate fell to 1.5 per month, at about the of implementation of the procedure change. If current trends hold, the rate appears to be falling to a rate of somewhere between .75 and 1.0 per month, a level last experienced in 1990, when there were significantly fewer flight operations. That would indeed be a very positive trend. Going beyond the statistics, however we have heard both positive comments and a number of concerns from pilots on the line:
Overall, the results since American implemented the new altitude awareness procedures have been positive. The rate has declined from 1991. American does not know, however, whether these changes resulted from the new procedures themselves, from the articles to crew members, or from the attention given to the topic in training and in discussion among pilots on the line. It may never be known what caused what (in fact we could probably only be certain if the rate suddenly reversed itself, which would mean it was the attention and not the procedure), but that probably is not important. What is known is that the pilots on the line made it happen. The Next StepAmerican and the APA have presented a comprehensive Safety Awareness Program (ASAP) to
the FAA for approval. it is designed to take a proactive approach by using the experience
and observations of line pilots to correct deficiencies information. throughout the
system. The actions taken on the line in response to the focus on altitude deviations has
laid the groundwork for a more cooperative approach. American is working on the details
with FAA Flight Standards. Stayed tuned.
by: Since I was first introduced to CRM in 1980, the concepts and the training programs have changed substantially. We have migrated from the letter C meaning cockpit, to it embracing the entire crew. Training programs have been improved and strengthened over the years, and quality control measures have been used at more than a few carriers. Nonetheless, it seems that some of same mistakes and missteps seem to be cropping up over and over. Since a good aviator learns from their own mistakes as well as those of others, this articles is meant to poke the needle at some common missteps that can be avoided. The top ten ways to kill off a CRM program was hatched at an aviation psychology meeting at Manly Harbor / Sydney, Australia just about one year ago. The original list looked something like this:
Many of these are self explanatory, and they also help to explain the life-cycle that many human factors programs seem to go through. We know that at many carriers, CRM has come into being due to external forces, such as one or more aircraft accidents, or a series of embarrassing incidents. In other cases CRM is implemented as part of an AQP program that is designed to save the carrier mega-dollars when implemented. And there are also numerous cases of programs being started because the thinking was to find ways to make an already safe operation even safer. In these cases, it is usually the senior people in flight operations and standards that become involved in setting up the program. The operational objectives are clear, and key members of the management team are usually involved to ensure that the program stays focused and operational. When this does not happen and operational control is lost, the result is usually a program that is a false start, or one that only lasts for a short period of time before being abandoned or replaced by another. Even when an effective program based on sound research and operational goals is launched, there is frequently a change as the program moves to the next stage in the life-cycle. The operational managers become convinced that the training is on course, and control of the program is frequently passed to one or more of the CRM facilitators whose sole mission in life is to put everyone in the airline, including the dumpster out back through the CRM seminar. Instead of quality, the measuring stick becomes how many warm bodies pass through the training, and also how many different types of programs are in place. The mindset here seems to be that more is better. We know that some CRM instructors and seminar leaders are more effective than others. We also know that line pilots leading seminars on a part time basis seem to do the job best. There are also a few that end up being more on the charismatic side instead of on the operational side when facing a room full of pilots. When the participants remember more about the instructor and how entertaining they were, than about the material to be applied on the line, something is definitely missing. There is more than one example over the past ten years where a pilot made a great instructor in the front of the room, but nobody wanted to fly with them on the line. Instructors and program leaders need to be credible with their peers and work to keep the program content operationally relevant, and standardized for the airline. Number five on the list deals with boxes, grids, and acronyms. Pilots like acronyms and abbreviations that can be used as memory aids. But this approach can also be overdone and sometimes keeps a program rooted in abstract academic concepts, instead of teaching operational skills. One major carrier set out as one goal to "create and effective CRM recurrent program that did not use any boxes, grids, or other cute devices." Ignoring the checking and training pilots can be accomplished in a variety of ways. There have been some programs that simply don't involve this key group in the training. At other places, the standards and checking staff are allowed to practice "selective compliance" when it comes to supporting and implementing CRM principles and skills. Sometimes, the check airmen react negatively because the course content is too academic or is seen as psycho babble. In other cases, they react to what they see as a tilt too far in the direction on human factors, with the needle being swung totally away from the technical side of performance. And still others react negatively to the person in the front of the room or the "style" of training that is being carried out. When the checking and standards folks are not actively involved from the very beginning, without fail a certain percentage will become polarized and negative. Trouble is sure to follow and it is amazing how creative a smart check pilot can be in trying to kill an idea or concept they do not agree with. It seems that every now and then, some well intentioned individual attempts to turn CRM and LOFT into some type of clinical therapy or self help session. This is one of surest and quickest ways to kill off a CRM program. If its not operational, the crew force will not embrace it. They may tolerate it for a short period of time, but change will not occur on the line. The antidote for avoiding these pendulum swings, and to keep the program on-track is straightforward. Operational management must stay involved with the program as active supervisors and participants. It can not be delegated to a few volunteers or others that are not seen as possessing high operational credibility on the line with their peers. In addition, the training program must be integrated into simulator, ground school, and checking activities. Implementing a tracking system to diagnose trends is extremely helpful, as is constant quality control monitoring for both technical and human factors items. When the pilots respond to a training event by saying that "Hey, now that was good
training!" we will know that CRM is truly taking its proper place as part of the
overall training picture.
by: IntroductionWhen faced with a difficult decision, most people react by showing signs of increased levels of stress, irrational assessment, or some form of personality trait shift. All of these are indicators that
The effect has been termed the Immediacy Factor by this author. Considering that the time factor involved in decision making is not new, but looking at the time factor as having two separate and distinct values brings to light possible explanations for the common phrase that "... some people perform better under stress, and others perform worse..." The immediacy factor can be represented by a three axis model where
Both quantity and quality of time are proportional to performance. In simple terms, to have a maximum performance, the subject must maximize use of the time available, while making the least number of attempts at decision making. It would seem from an initial study that experience plays a major role in the maximization of the performance score, however, certain inexperienced subjects seem to have other traits that play a role in allowing them to maximize performance. This article first addresses some of the concepts of the human element that may play a role in the immediacy factor, then secondly, explains the immediacy factor model (by axis). Human Elements of Immediacy - Situational Awareness in Decision MakingSituational awareness refers to one's ability to accurately perceive what is going on .... Effective decision making is based in part on the correct interpretation of the "situation" by the decision maker. Individual perceptions, their relationship to the decision be made, and the effect of others (thought their perceptions) on the situation awareness of the decision maker all play a major role on an individuals decision making ability. According to Jensen, the underlying principle of the Cockpit Resource Management (CRM) training effort is better crew decision making. As indicated by experts' findings, one of the ma or causes of airline accidents today is the lack of teamwork in making decisions during unusual situations. Airline crews are required to make fast, safe decisions, usually in an information rich environment under stress and often the ragged edge of psychological overload. It is purported that the Immediacy Factor (and the related stress) limit the ability of one to interact in a team environment, thereby limiting the performance of both the individual and the team. IndividualismIn this article, the focus is on attitude and personality traits. It is recognized that not all individuals have mastered even the minimum skills required for a certain task, but the general concept of immediacy is not affected by the skill level, as much as the experience in the task at hand. While these may seem to be one in the same, refer to the section below on experience to read a comparative difference example. Skill, attitude, and personality traits work in conjunction with each other to establish the framework of a person's performance level. Skills are refined through training and practice, while attitudes are outlooks and opinions that are formed through personality traits. These traits are thought to be inherent at birth and developed throughout life. Successful performance on these three basic attributes to provide the effective guidance in all type of decision-making. There are three primary factors involved in the formation of a person's level of skill. The first factor is a natural sense or ability for certain tasks which is difficult to measure. The second primary factor relies on attention. The ability to concentrate on the task at hand and retain situational awareness is crucial to the decision making chain of events. The final factor in developing skill is the decision making process itself. A crucial component in decision making is the complete understanding of the occurring situation. All the facts must be gathered, examined, and systematically applied to the situation. There are three types of decisions; problem-solving decision, anticipatory decision, and decision by indecision. Judgment and decision making skills can be improved through training. Some effective training concepts would include; behavioral modification, risk recognition, and stress management techniques. According to Barton, "Attitudes are personal viewpoints, opinions, outlooks, prejudices; they are habits of thought, our attitudes develop our mental environment". Our reaction to a situation is prompted by the environment that created or attitude, and our success is achieved through self-motivation. The combination of a skill, attitude and personality traits can lay the foundation for [performance]. Personality traits are usually stable, deep-seated moral responses that are instilled at an early age. In 1964, Eysenck described aviators as being stable extroverts. The extravertive personality trait suggest that this type of person was under-aroused, always seeking stimulation. The diametric side of this being an introvert, an over-aroused individual often avoiding stimulation. Himmelweit stated that introverts and extroverts differ in overall performance as well their approach to completing the task. Introverts tend to work slowly and make fewer mistakes whereas extroverts work quickly with a high probability of error. StressStress is another important factor which is discussed. Stress decreases productivity, leading to "forgetfulness, lack of recognition of developing cues, lack of attentive focus, and in the worst case, an accident". Stress is the body's non-specific response to demands placed upon it. Whether the demands are pleasant or offensive, it is placed into two categories: chronic and acute. Chronic stress is the result of long term demands imposed on the body by life events. The "life events" or major changes in and individual's life can be positive or negative, yet still have the same effect on the body. Acute stress is the result of demands placed on the body by the specific or immediate task at hand. The demands could be anything from a collision warning at the Air Traffic Controller's console to losing a wallet. Chronic stress, if not dealt with properly, will have an affect on acute stress. If ignored, it will inhibit an individual's ability to deal with acute stress. These types of stress tend to have a synergistic affect. The accumulated effect of both types working together may have a more detrimental result than the sum of the individual stress types. This applies to cumulative effects of individual stressors as well. According to Thomas, there are ten basic categories of stress factors. These include conscious, unconscious, personality, psychological, physiological, environmental, [inter-relational], sociocultural, life change, and acute reactive. The conscious and unconscious factors deal mainly with an individual's ability to learn and concentrate, past experiences, and any underlying motivations. The personality factors deal with an individual's attitude towards stress; it also influences the way a person copes with stress. Psychological and physiological factors are related to mental state and physical condition which greatly affects the ability to deal with stress and its affects. Environmental factors involve the place of work or residence which deal with how an individual experiences related activities. Their perceptions and language used in expressing their personal experiences to others can be stressors in and of themselves. Sociocultural factors include economics, laws, regulations, and industry trends and exemplify how an individual can be influenced by the society in which they operate. Life change factors are any changes in a person's personal or professional life, whether positive or negative. Acute reactive factors can be defined as any situation that requires immediate corrective action. It has been accepted that everyone performance varies with a moderate level of stress. Performance can increase up to a certain point after which it will begin to deteriorate dramatically. This principle holds true for most individuals. Thomas divides stress into two categories based on performance. As the stress level increases, performance declines resulting in negative stress. The level at which stress degrades performance is different for each individual. These variables include personal life, fatigue, mental state, etc. Although stress can never be completely eliminated, it can be controlled through a conscious effort of awareness and action. Defining the Model - Perceived Time QuantityHow much time do I have until a decision must be made? As mentioned in the introduction, the perceptions of the individual is a key element in the decision making process. The personality of the individual, the environmental effects, and the introduction of group dynamics all shift the perceptions of the decision maker. The value of time in the immediacy model is a perceived value, and therefore, is affected by individual perceptions, environmental effects and the effect of others (though their perceptions) on the decision maker. First, is the effect of the individual perceptions and reality on situational awareness. The way we perceive the external environment to gain knowledge is based on at least four different sources: senses, authority, reasoning, and emotions. The senses are a fundamental source of information, but they are limited to observing immediate environment (such as the flight instruments which a pilot uses to monitor the aircraft and the environment). People of authority are a source of information with knowledge that is learned and by experience. Our own reasoning abilities is also a source of knowledge. Emotions effect our perception of reality by at least four different ways: they arouse us, help organize experience, direct and sustain actions, and communicate actions. The second area is the relationship between individual and crew situational awareness. This relationship is a complex situation where every person brings his or hers individual background, experience, and knowledge into a group interaction which affects other peoples perception of events and the whole resulting performance. There are four major areas affecting the cohesion and coordination of the crew.
The third area is identifying and assessing environmental and situational conditions affecting situational awareness. Effective situational awareness requires understanding of the human limitations and capabilities of perceiving the world as an individual as well as how this is integrated into a group environment. Individual behaviors that degrade situational awareness include task overload, boredom, complacency, uncertainty, distractions, frustration, and anger. Group behaviors that degrade situational awareness are "press on regardless," "it is always done this way," "not playing with a full deck," "group think," and "too much too soon, too little too late". All of the above mentioned areas of perception effect the way an individual reacts to cues and the proper perception of time in the immediacy factor is an important element is understanding the effect of time quantity in the model. Time QualityHow should I spend my time? During any decision making exercise, when the time available is perceived to be unlimited, there is a significant improvement in the ability of the subject to make the correct decision, mostly thought a trial and error approach. This trend levels off, and then falls drastically, as an effect of boredom. The concept of making a quality decision is therefore thought to be related (in part) to the time available, and the revision of the decision is facilitated thought the time remaining. Consider the following example: If a subject perceives five (5) minutes available for a before a decision must be made, and makes a less than optimal decision at three (3) minutes, then (according to the hypothesis of the Immediacy Factor) two (2) minutes remain for the subject to go back and revise the initial decision. If the revised decision is a better one, then the overall performance will be higher than the initial possibility, since the perceived time had not elapsed. This is an example of making a quality decision within the perceived time available. Quality of decision making depends highly on the decision making concepts of evaluation of facts and re-evaluation of the decision. Since most decisions are a link in a decision chain, the immediacy factor plays in heavily when decisions are linked together, and the perception of time is generally the limiting factor that keeps subjects from going back and reevaluating the quality of their decisions. For this reason alone, it is imperative that a quality decision is sought is for the first decision, and that enough time remains for the reevaluation process. PerformanceIt is known that stress is an inhibitor of performance, to a certain extent. This is especially true when considering excess stress, and the effect of stress on perception. According to Edwards in his book Pilot, 1990, there seems to be a tendency for stress to limit the perceptive ability of individuals, similar to that of "Tunnel Vision." Part of the need for the Immediacy Factor model is the fact that perception of time in decision making is not accounted for in current measures of performance where the subject being examined has self-induced stress which is derived from the choices of 1. How much time is available and 2. What task is in need of attention in order to maximize performance. The measurement of performance is always a difficult task, and this may prove to be the case with the immediacy factor model. The performance axis of the model is thought to be a mathematical derivative of the individual points of the time quantity (x) and time quality (y) axis, with a proportional relationship to the performance (z) axis. If this is the case, whatever the measure, the performance rating of the subject should increase with either an improved rating on the time quantity or time quality axis. Summary and ConclusionsThe elements of mentioned in the first section of the paper reveal how limited any
performance measure would be that did not consider time as both a value and quality, since
individuals would appear to react to time through a number of different stressors. The
Immediacy Factor is intended to provide an delineation between the relation of time as a
value chronologically, and the value of time as an efficiency or quality. The article
provides an outline of significant factors affecting the human when considering the
application of the Immediacy Factor model, as well as the general parameters for
application of the model. The classification / ranking scales (Axis') represent a linear
scale system that could serve both as a working example of the parameters in relationship
to each other and also as a method of classifying in a broader sense the level of
immediacy with respect to each of the axis of rating presented. For an overall Immediacy
Rating scale that ranges from Passive to Incapacitated, for example. Further work on the
application of the model should provide some methodologies to the CRM community when
attempting valuate the effect of stress on the human in a decision making environment.
by: What will Delta's 1994 CRM program look like? What led to your decision to focus on only one CRM marker? Can you expand on the issues of "routine" and "not
objective"? Does this focus expand beyond the classroom? How will you know if this experiment is working? Will you share information with "The CRM Advocate" for the next issue
on your joint program with flight attendants?
|
|
|