Re: Greg Deen's comments on metrics
CRMDEEN_at_aol.com
Sun, 29 Dec 1996 12:15:42 -0500
Metrics IS easy, if the training, management and evaluation systems are
in place. I was somewhat facetious in the term "easy"; I do think metrics
could be a "simple" part of the aviation system.
Ironically, we have had a metrics system for many, many years in the
form of flight evaluations. The evaluations are often combined and examined
for "trends", and the trends receive some higher level of interest,
training, or some other form of attention. For example, within the Air
Force, a trend on evaluations appears to indicate recent evaluations
discovered a lack of knowledge about a specific procedure, usually an
emergency procedure. The stan/eval office tells the commander, who tells the
training department, who finds a way to tell and teach the fliers of an
increased emphasis on that particular procedure. The fliers "bone-up" on the
procedure themselves, and/or the simulator refresher department inserts more
of those emergency procedures into the training syllabus. When the
evaluations quit detecting a lack of knowledge of that procedure, management
considers the training effective. (We'll discuss this effectiveness some
other time)
So what's going on here? A "metrics" system detected a potential
problem, management and training systems responded, and the "problem" went
away, or did it? Now let's apply that series to CRM.
A DC-8 runs out of fuel in a holding pattern. The investigation
revealed a lack of effective communication. Many years later, a DC-10
experiences a catastrophic hydraulic failure, and the crew is able to bring
the airplane to an "almost" landing, with an amazing amount of success. Two
very different outcomes within the same company. When the CVR transcript of
the DC-10 is applied to the decision-making process taught through that
company's CRM program, the answer is clear and fascinating. As the stress
and workload level of the DC-10's crew increased, so did the communication.
The communication was not just sporadic chit-chat; it was direct questions,
responses, suggestions, information, observations, advice, decisions and
reviews. As a quantifiable item, "ineffective" communication was the least
apparent item in the analysis. The method of decision-making trained revealed
itself on the CVR. What other system had changed? The entire culture of the
company, and the change didn't happen overnight; it took years.
Training began and survived the opposition, management supported the
concepts, and the evaluation system was tied to the training. All three
dynamics worked in concert; one won't survive without the other. How many
companies, including the military, are having difficulty establishing a CRM
program in just one area?
Now back to the metrics challenge: the metrics I started this reply
with was a traditional flight evalulation, which looks toward "technical"
skill. If the training system teaches CRM concepts as a technical skill, the
evaluation simplifies. As toward the MOST scenario I revealed, there are
lots of technical skills, including CRM teachings, available to "grade". If
the management system would back-up the CRM evaluation as it does the
technical, we'd make a lot of progress toward improving the teamwork of our
crews. That same crew flew a more demanding mission a few days later, and
their CRM was improved.
As of this time, I have received only two "evaluations" toward my story.
I'd like to see more. I do plan to compare my "debrief" with the ones I
receive off of this CRM channel. I will tell you that the two responses were
more specific and more technical than my own; probably due to a better
defined "system" in either training, management, or evaluation.
For those who haven't yet pitched in their opinion--come on in, the
water's fine.
Greg Deen
HTI-C-130 ATS