This epistle is a continuation of the ongoing metrics discussion. I think we
are close to getting a focus for a Tiger Team or two. From my own
observations and those of Vince Mancuso and Skip Mudge I draw the following
conclusions, remarks and questions.
Following John Wise’s lead to get to the underpinnings is the right place to
start. It’s pretty hard to measure something if you do not have a definition.
Vince describes in part the “management skill (CRM) performance “. If I had
my way I might use a different term, like performance skills, but that is
only a semantics problem. By that I mean a difference in language, more about
that later.
Both Vince and Skip seem to be working with at least a partly completed
program model which is founded on defined “management skill expectation” or
“Standard Management Procedures”. Both of these systems are/were developed
from some quantity of analysis. Perhaps those systems could be shared with
the Tiger Team (to be formed) even if in generic form. The Team could combine
those and other inputs from the community at large and come up with a truly
generic model of what a CRM program task list should be. That would put us a
long way down the road toward developing some kind of measurement system.
Another or additional approach would be to follow Vince’s focus group
approach ( a very viable and promising approach) by having the Team’s product
reviewed by focus groups from several different segments of the community.
This would be in concert with one of the early discussions of a Team’s
activities. The Team will keep working a problem until it is resolved or OBE
(overcome by events). Any entity in Group could participate in the Teams
activity for as long as they choose. Hopefully the final product of this team
would be a generic master task list for CRM applicable, at least in part, to
any aviation activity.
Another part of the aforementioned listserv inputs that caught my attention
was the discussion of process versus performance measurements. Maybe it is
my own lack of understanding of the terms, but I think performance
measurement is the key. From the CRM performance of the crew, how ever
measured, we determine the success of the CRM teaching/learning process.
That may or may not reflect the overall mission success.
For example, the crew interacted well in the pre-approach portion of the
flight (checklists, required calls, briefings, etc.). The pilot’s approach
was marginal. The LOE or checkride not good , but CRM OK. Add in severe
weather, rain, heavy turbulence, high cross winds and we mitigate our
evaluation for the pilot’s approach. The reverse can be true for CRM versus
the crew’s overall success. Seems like another good target for the/a Tiger
Team: Should the CRM evaluation/meteric be separate or integrated with the
overall evaluation process? If it is integrated, then how do we relate the
results back to the CRM program?
The administration of CRM evals or performance measurements should be
accomplished on the same LOEs or check flights as routine evals.
I really like Vince’s model of “Given Set of Circumstances”...”Appropriate
Response”. This appproach can be proactive in at least two aspects: 1) If the
Given Set is a routine part of the task list of the crew. (e.g., During the
briefing the pilot flying is expected to solicit inquiry and input from the
other crew members). 2) The crew or an individual recognizes an impending
situation (e.g., the application of good decision making skills to prior to a
problem arising).
Do we measure the crew or the individual? I say both. I really love loading
up the check airman. The impact of an individual crewman can be significant
on the success of the crew. However, part of our CRM training is/should be
how does the crew respond to the difficult crew member, even if, heaven
forbid, he/she is the Captain? (Now there is a true test for the crew and our
training program.). A principal difference between this approach and Vince’s
example of the college group eval is that we want to know if the crew AND the
individual were successful (CRMly speaking).
Dave Wilson
HTI