Much is being discussed about metrics; who, what, why and when, etc. While
this is all good discussion, it is even more important to keep focused upon
the goal of CRM training: to improve mission effectiveness. We are, or at
least I hope we are, striving to improve behavior of the aviation system.
Notice I am not just limiting this vision to the cockpit, nor to just the
aircraft. A holistic approach to the entire system is necessary. Improving
the communication between a pilot and co-pilot is a much-needed objective,
but it may not be enough to overcome intense management pressures upon a
pilot-in-command to complete a mission in the face of insurmountable risks.
Aviation is the result of the efforts of a lot of small teams, each with
their own internal and external dynamics. Understanding behavior within the
teams is paramount to evaluation and training is needed to reinforce the
effective and acceptable behaviors. Some seem to think metrics are the tool
to the understanding of the teamwork process, but simply understanding the
process is not the solution.
We may understand gravity, but we are not free of it. Understanding only
gives us the knowledge to either adapt or to utilize. For the aviation
systems to eventually behave effectively and flawlessly, we must develop an
understanding and knowledge of each team’s behavior dynamics. That
understanding comes from evaluation, which requires a tool, which we within
this group are calling “metrics.”
Metrics is simply the feedback to the training. Tests are metrics.
Performance evaluations are metrics. The cycle completes when we evaluate the
effectiveness of training. For those who are trying to establish a metrics
framework around the words “appropriate response for a given set of
circumstances”, I suggest look simply into your training document for the
objectives of the training. If you aspired to a training goal of
“recognize the loss of situational awareness”, did your training program
present the aviator some indications of this dynamic. If so, then an
evaluator’s job is easy. When the “given circumstances” occur, does the
aviator respond to them. If there was a response, then recognition occurred.
Whether the response was appropriate or not, is a different matter.
The development of metrics seems to be the “chicken or the egg” dilemma;
which comes first? The amount of research accumulated to date, and that
which is so readily available, could easily develop into a training program
of effective team behavior. Training a co-pilot to communicate to the pilot
is simple if management and instructors understand the process themselves.
Referring to a co-pilot as the “wetware beside you” is an indicator of
undeveloped training objectives. Training a co-pilot to spend a majority of
his time watching and listening to another pilot slowly programs that
individual to become a passenger, historically labeled the “co-pilot
syndrome.” When crews are trained to always share the workload, and when
company policy directs the pilot-in-command to delegate flying duties at
specific challenges, and when this behavior is reinforced within repetitive
training, metrics becomes easy. Given a circumstance, such as “destination
weather is below minimums, state intentions”, (famous ATC radio call), the
pilot-in-command delegated flying duties to the co-pilot. The evaluation of
this event is easy, if the training objective was clear and pursued.
Metrics is easy. All that is required is a training system, a management
system, and an evaluation system. Each of these teams need to work in
concert toward clearly defined behavioral goals. Aviation, particularly safe
and effective aviation, is the result of teamwork by teams. Each will be
judged by the philosophy of the metrics system used to understand the process
of CRM.
So which comes first, training or evaluation?
Greg Deen
HTI