Facilitating LOS Debriefings:
A Training Manual

Lori K. McDonnell, Kimberly K. Jobe, R. Key Dismukes

NASA Ames Research Center

Part 4. The C-A-L Model in Action

The C-A-L Model provides a way to structure the debriefing. The first section, CRM, suggests strategies for helping the crew focus on CRM techniques that played a role in the LOS. The second section, Analysis and Evaluation, shows how to guide crews to identify and evaluate aspects of their performance that went well or could use improvement, including analysis of why the crew did what they did and why things turned out the way they did. The third section, Line Operations, provides a structure for helping the crew explore how they can apply what they learned from their analysis of the LOS to line operations.

A good way to organize debriefing each segment of the LOS is to show the appropriate video segment and then use the components of the C-A-L Model to guide the discussion.

C.- CRM - Applying the Company Model
The major purpose of LOS is to give crews a chance to practice using CRM concepts and techniques in realistic flight scenarios. Typically, crewmembers are much better prepared to talk about the purely technical aspects of a flight situation than the CRM aspects so they may need to be guided toward discussion of CRM issues pertinent to their performance in the LOS. Rather than discussing CRM as a set of abstractions, you can help crews the most by guiding them to consider how specific CRM techniques can be used to manage various flight situations.

Focusing on CRM
You can use several techniques to bring CRM into the crew's discussion in a concrete, relevant way.

+ Refer to the posted CRM concepts.

Each company teaches a specific framework for CRM concepts that reflects the company's philosophy. Posting this conceptual framework on wallboards and referring to it during the discussion can help the crew relate CRM concepts and techniques to specific operational situations.

   Effective:
IP: Which of these crew effectiveness markers do you think you used to resolve that de-icing report problem?
FO: We used 'Inquiry and assertiveness' and 'Vigilance' making sure everything was okay.
   Effective:
IP: What CRM technique do you wish you might have used a little better?
CA: 'Anticipate required actions and workload distribution.' That was a tough one for us because we were both very busy and there's not much he can do except fly the airplane if he's flying. We're supposed to each follow along on these checklists, and I think that's something we could have been better at.
IP: What techniques can you use to manage workload better in the future?
   Effective:
[The CA tells the FO to ask ATC for a holding pattern in order to have time to do a gear and flap extension during a video segment. The instructor stops the video, points to the posted CRM markers and asks questions.]
IP: What CRM principle is that an example of?
CA: Create time.
IP: And how did you create time?
+ Use CRM-specific questions.

Get the crew to explore specific CRM issues and techniques that presented themselves during the LOS.

   Effective:
  • "What CRM techniques might have helped in this particular situation?"
  • "What specific CRM techniques did you use that resulted m your successful outcome?"
  • "What kinds of challenges did the LOS present in terms of how to manage your time?"
  • "Looking at your workload management, what CRM techniques did you use in this LOS that you learned in ground school?"
+ Use guiding questions.

Guiding questions can be used to lead the crew to more specific and in-depth aspects of their LOS performance. In the following example the instructor encourages the crew to discuss specific aspects of teamwork that occurred during the LOS.

   Effective:
IP: Did you feel included?
FO: Yes.
IP: What did the CA do to make you feel included?

-or-
IP: Did you feel included?
FO: Not really.
IP: What could the CA have done to make you feel included?
In the next example, the instructor facilitates crew discussion of specific aspects of workload management that occurred during the LOS.
   Effective:
IP: Did you feel rushed or overworked or distracted perhaps?
FO: Yeah, I felt like I wasn't ready to turn in.
IP: But you turned in anyway?
CA: Yeah, I was ready. I didn't ask you [FO] if you were ready though.
IP: You weren't ready, FO? And could you have said something to the CA about that?
FO: I could have, but I usually don't say anything as a co-pilot unless I think I'm in dire danger.
IP: What would you have said if the FO had said "I don't feel ready"?
CA: I would have said "Let's take another turn in holding" then.

Reinforcing the Utilization of CRM through Crew Interaction
Encourage crewmembers to address each other directly. Instructors and crews too often fall into a pattern of discussion that centers entirely on the instructor. The instructor asks a question, a crewmember responds, the instructor comments, and the cycle repeats. The crew will benefit in several ways, however, if you can get them to discuss their performance in the LOS directly with each other. Interactive discussion between crewmembers during the debriefing allows them to practice CRM skills such as communication and problem solving. It also leads them toward the ultimate goal of being able to debrief themselves in line operations. You can counter the crewmembers' natural tendency to direct their comments to you by using the following techniques.
+ Ask crewmembers to discuss how they were affected by each other's actions.

It is important that crewmembers understand how their actions effect each other. By openly discussing these issues, crewmembers may become more aware of the impact of their actions and the importance of communicating what they are doing and why.

   Effective:
IP: When he was going through the checklist he said "It's in the other checklist" Did you know what he meant? Talk about that.
CA: I knew what you meant, but I just kept thinking "Are we on the right checklist?"
FE: I knew we were on the right checklist, it's just that the two are somewhat redundant. But I figured as long as we got all of the items done. But I should have said something to you.
CA: Okay. I guess I know that now, but I didn't at the time.
   Effective:
IP: You guys were very supportive in there. What did the Captain do to help create that environment?
FO: Well, he set the tone right off the bat during the briefing, telling us that if we had anything at all to say, we should speak up. So that made it a lot easier to say what was on my mind when I needed to.
+ Ask crewmembers to discuss what they were each thinking.

Encouraging crewmembers to openly articulate and discuss what they were g may help them understand each other's point of view and thereby enhance communication.

   Effective:
IP: CA, what did you do after you called for the engine fire checklist?
CA: I got on the radio to talk to ATC.
IP: And what did you think about that, FO?
FO: I couldn't believe you were talking to them. I thought we should have, at least had the first couple of items on the checklist attacked by then.
CA: And see, my initial gut feeling was "Get this thing headed for Boston now, whether you get the fire out or not, we need to be heading that way."
IP: Why do you think that CA?

A: Analysis and Evaluation of LOS Performance
Don't give your analysis or evaluation before the crew have completed theirs. For crewmembers to learn from their LOS experience, it is essential that they analyze and evaluate what happened. Crews should analyze both what went well during their LOS and what did not work as well. The analysis must go beyond simply naming the am strong and weak points of their performance. Crews can gain powerful insight by analyzing why things turned out the way they did, including factors that either enabled or hindered their success. Remember to refrain from giving your analysis until the crew have completed theirs.

Getting Crews to Evaluate their Performance
It is important for crews to learn to critically evaluate their own performance in the LOS so they can carry this skill over into line operations. As the crew discusses the LOS, you may find it helpful to use a wallboard to list "strengths" and "areas for improvement ' as they are identified by the crew. The following techniques may be useful in encouraging crews to evaluate their performance in depth.

+ Get the crew to talk about what went well.

Discussing what they did well helps the crew to recognize what strategies were effective in managing the LOS challenges and how these strategies might be used in line situations. This is also a good strategy to use when crews say everything went great during the LOS, so they can actively identify exactly what went well and why.

   Effective:
IP: FO, talk about one thing that went really well for you on that first leg.
FO: I think the fact that I didn't get overloaded at any that I was able to stay ahead of the airplane and because of that the rest of the leg went pretty smoothly.
 Ineffective:
IP: You did a good job on that. Did you know you did a good job on that?
FO: No.
IP: Well, you did.
   Effective:
  • "What aspects of your performance were particularly strong on the first leg of your LOS flight?"
  • "What did you like about your performance on this trip?"
+ Get the crew to talk about what could be improved, and how.

Discussing what did not go well helps the crew identify problems that occurred, examine why they occurred and determine how to resolve or avoid similar problems in the future.

   Effective:
IP: What happened on your preparation for the approach; would you have done anything differently?
FO: Well, I dropped the ball by not putting the approach in there. I don't know where I was when that was happening.
CA: Well, yeah, and I didn't pick up on that either. We both missed it.
   Effective:
IP: What did you think about your go around.
CA: I was coming down and I just hit vertical speed and then max power and then flaps 15. I was trying to be smooth, but I thought it was really poor, actually.
   Effective:
  • "Are you satisfied with how you handled the generator failure? Why or why not?"
  • "Discuss one thing that you would like to have handled better during your climb out?"
  • "Why did the miscue occur, and what situation led to both of you missing it?"
  • "What events led to SOPs not being adhered to?"
Encourage crew to discuss how to handle things that don't go well. + Troubleshooting: When the crew says everything went great.

If, when asked to evaluate their performance, the crewmembers say everything went great, facilitation may be required to encourage the crew to analyze and evaluate in more depth. If everything did go well in the LOS, encourage the crewmembers to discuss specific instances of good performance and analyze why they went as well as they did. Also, encourage them to discuss how they could have handled situations if they had not gone so well. If there were, in fact, situations that were not handled effectively, draw the crew's attention to a specific situation and ask if there is another way it could have been handled. Regardless of whether or not everything went smoothly in the LOS, it is important that the crewmembers understand both the factors that led to their successes and the factors that led to weaker aspects of their performance.

Eliciting Deep Analysis
Push the crew to go beyond just describing what happened. To learn deeply from the LOS experience and take the lessons learned back to the line, crewmembers need insight into why events in the LOS turned out the way they did. Crewmembers can gain this insight through indepth analysis of their LOS performance. You can help the crew analyze in depth by asking questions that require careful thought and detailed responses.
+ Ask questions that require description and analysis of LOS events.

Open questions that require descriptive or analytical responses lead crewmembers to explore issues more thoroughly, which opens the door for deeper learning.

   Effective:
  • "Describe what happened there and why it happened the way it did."
  • "What did you notice while watching that video segment of your LOS session?"
  • "Tell me what you did to handle that situation."
  • "If you had to report to the safety people about this incident, how would you describe the situation and how you handled it?"
 Ineffective:
Questions beginning with "Did you" or "Do you think' enable simple responses, and thus do not promote in-depth analysis.
  • "Did you follow the correct checklist for that problem?"
  • "Do you think you handled that situation effectively?"
+ Get the crew to analyze why they made the decisions they made.

Articulating why they did what they did helps crewmembers gain insight into their decision-making processes, as well as the factors that influenced or should have influenced their decision making.

   Effective:
IP: The RVR was down below twenty-four hundred, but you asked for two-twenty on the speed, and I was wondering why.
CA: Well, the jet stream was real low, and there was all kinds of weather, with a warm front coming through, so I'm thinking that if we're going to be hanging close we need to be clean maneuvering, so let's add a couple initially to make it smooth but still slow it down.
   Effective:
IP: Why did you turn off the pitot heat?
CA: Because I looked at the outside air temperature and it was plus fifteen C, so I figured we were good with that.
   Effective:
IP: Why did you decide to go back to Minneapolis instead of continuing on to Chicago? What was your thought process?
CA: Well, I was thinking that we were still a lot closer to Minneapolis than Chicago. Then again I knew the weather was not as good in Minneapolis as it was in Chicago, so if that sick passenger was stable in the back, I was all for going on to Chicago. But as he started to deteriorate rapidly, I thought the best course of action would be to turn around and go back because I thought the guy might die if I didn't get him back pretty quick.
   Effective:
  • "Why did you feel it was necessary to disregard ATC until you completed your checklist?"
  • "What made you decide to verify the information you received from ATC?"
When you analyze for the crew it leaves little for them to say. It also gives them the message that you are teaching them so they are not expected to analyze for themselves.
   Ineffective:
  • "The reason you got into trouble during the approach is that you didn't communicate to the CA that you were unsure of the clearance."
+ Get crewmembers to discuss what they were thinking.

Getting the crew to discuss what they- were g during the LOS can help them discover what information and events influenced their actions.

   Effective:
IP: What was going through your mind at that moment?
CA: Well, I was thinking about the sick lady in the back and that we had to get down and get her to a doctor, but you can't rush the approach because we might have to make a missed approach and then it would take even longer.
   Effective:
  • "What were you thinking when you received the traffic alert?"
+ Encourage the crew to discuss the factors that enabled or impeded their success.

Identifying underlying factors that enabled or impeded their success in the LOS can help the crewmembers recognize similar factors when they occur on the line.

   Effective:
IP: CA, was there something that helped make that windshear recovery go so well?
CA: Yes, FO was very assertively making speed callouts all the way that helped me stay on target.
   Effective:
IP: CA, can you think of any factors that contributed to your getting rushed on that approach?
CA: Well, in hindsight, I should never have accepted the turn onto base leg before we had finished the abnormal checklist.
FO: I wasn't sure if you realized we had not finished it. I wish I had queried you on that.

L: Line Operations - Applying Lessons from LOS
To help crewmembers transfer the lessons they learn in the LOS to the line, encourage them to discuss how LOS performance and associated CRM issues relate to effective line operations.

Get the Crew to Discuss Related Line Incidents
Getting the crewmembers to discuss actual line incidents and accidents related to CRM issues that arose during the LOS can help them to appreciate the role of CRM in line operations. For example, if a crew communicated well during the LOS which resulted m a positive outcome, the importance of their effective communication can be reinforced by having them discuss a line accident that resulted from poor communication.
   Effective:
IP: Looking back on the [XXX] accident we had where the guys taxied onto the wrong runway, what was the major problem there?
CA: Lack of communication. And not paying attention. When you're taxiing under conditions like that both guys better keep their heads where they should be. They should be thinking about nothing but taxiing.

Get the Crew to Discuss How to Apply Their Success to Line Operations
The next step in getting the crewmembers to transfer what they learn in the LOS to line operations is to get them to explore how they can apply techniques they utilized successfully in the LOS to overcome obstacles on the line. Getting crewmembers to talk about how they would handle difficult situations enables them to develop effective, pre-planned strategies for dealing with real situations when they occur on the line. For example, one crew's success in a particular LOS might have been aided by the fact that both crewmembers were good at explicit sharing of appropriate information. In the debriefing, you might find it useful to facilitate a discussion of how to deal with crewmembers on the line who are not as good at sharing information.
   Effective:
  • "What would you do if you ran into one of those guys on the line?"
  • "How would you handle the same situation if the CA/FO/FE was completely non- communicative?"
  • "In what other situations could you use that techniques'

Get Crew to Discuss What They Would do Differently
Discussing what they would do differently enables crewmembers to develop strategies they can use to make more effective decisions and avoid similar incidents in line operations. Ideally, crewmembers should discuss the strategies they can use to turn each item in their areas for improvement list into strengths. It is often useful to frame this discussion in terms of how the crewmembers can handle similar situations if they occur on the line.
   Effective:
IP: CA, tell me what you could have done to make the approach onto runway three- two work out better than it did.
CA: I could have just immediately gone to the approach page and put in runway three-two, brought it up and it would have probably had Grunds. Even if it hadn't, it would have had everything else we needed, and things would have gone a lot more smoothly.
 Ineffective:
IP: You could have avoided that by using your FO more effectively.
[Telling the crew what they should have done differently denies them the opportunity to figure it out for themselves.]
   Effective:
  • "What CRM techniques can you use to keep from getting overloaded in the future?"
  • "How will you manage that situation if it comes up tomorrow on the line?"

Get the Crew to Discuss How They Will Do Things Differently on the Line based on Their Experience in the LOS
This final step requires crews to specifically tie what they have learned in the debriefing to the line. Having crews discuss how they will perform differently based on what they have learned can help them make the connection necessary to transfer thoughts into actions.
   Effective:
  • "What would you do differently back on the line to avoid the problems that occurred during the second leg?"
  • "Let's say you're flying into LAX tomorrow, and ATC switches the runway on you at the last minute. What CRM techniques can you use to manage the situation?"

| PREVIOUS PART | TABLE OF CONTENTS | NEXT PART |

[ CRM-DEVEL Home ] [ What's New ] [ Resources ] [ CRM-DEVEL Mailing List ]
[ Neil Krey's Professional Aviation Instruction Forum ] [ Neil Krey's FLIGHT DECK ] [ Aviation Instruction Bookstore ] [ Aviation Instruction Career Center ]

Copyright © 1996-2005 by Neil C. Krey unless otherwise indicated.
Non-commercial reproduction rights granted if the following notice is included:
"Source: Neil Krey's CRM Developers Forum, http://www.crm-devel.org"