The Metrics we have been applying to our CC-130 crews in a study designed to
investigate the level of safe flight performance is the assessment of crew's
mental models at the following levels
1) Simple awareness of a situation (the #4 prop governor)
2) Knowing the implications (prop run away)
3) Make and implement appropriate plans to cope with these implications
(that's what the PIC is paid for!)
This is very similar to a scheme put forward a few weeks ago by someone else
(really it is level1, 2, and 3 SA).
Secondary measures were
1) systems knowledge
2) task prioritization
3) workload monitoring
4) open loop communications (no acknowledgement, instruction not implemented
etc.)
We did this by categorizing the crew's communications and post flight debrief.
Using this process we found that the metrics distinguished between high
performing and lower perfoming crews as determined by independent check pilot
evaluations. This established the sensitivity of the metric in a known groups
manipulation. A first analysis of the data showed that it was the PIC's
behaviour that drove the overall performance of the crew. The remaining
analyses focussed on the PIC's communication patterns. AT this point it must
be said that our measurements were intended to distinguish between two
experimental states (i.e., we could assess the efficacy of various
manipulations such as work/rest schedule, level of experience, crew concept).
We do not have absolute levels that determine a minimum level for safe flight
performance. However we do intend to take this framework and develop a rating
system from it. I think one could develop standards from this. Further I
think it can be a basis for decision making training.
While developing the measurement process we became aware that a proportion of
the crews had, what I would judge to be, inadequate systems knowledge. We
also found that Captaincy (we are now in the process of trying to define just
what these skills are!) skills were lacking in some cases.
Now for your situation. I am not sure that you need a metric to score this
scenario (although the PIC clearly was unaware of the implications, made no
plan to cope, did not manage his/her workload by delegation, miss-prioritized,
lacked systems knowledge, ...). I would not get on an aeroplane commanded
(commanded? - well actually that was the problem wasn't it!) by that pilot (I
really hope he/she was joking when he/she said "I hate it when it's my
decision." It seems from the lack of cooperation from the rest of the crew,
that they weren't too happy either. The FE did try by prompting "are you
asking for a suggestion?" - the Navigator was less than subtle in his/her
contribution. This to me was a failing grade. If the Squadron doesn't look
carefully at this PIC to assess his/her suitablity for command, I would be
very suprised. At the very least some decision making training and a foray
back into the books on aircraft systems is indicated. I know that falling
experience levels is a problem in military aviation, but is this person really
up to the tactical role?
Keith Hendy
DCIEM, Canada
PS Greg, I would love to share experiences in what you have been doing with
the CC-130 community. We have been given the charter to look ways to improve
the training system in both systems knowledge and captaincy skills (a daunting
job). To try find out what the dimensions of Captaincy are we are putting a
knowledge elicitation package together that we can administer to our crews.
Perhaps this would be relevant to at least one of your PICs ;-)
Hello CRMers,
Yesterday I was conducting a MOST (military for LOFT) mission and the
conversations of metrics was on my mind. There was an interesting process in
the mission, and it occured to me that if I somehow relayed into this forum,
we could all give an opinion on the event, and that perhaps this would give
us all a common event to relate the metrics discussion to.
The crew consists of a PIC, a co-pilot, a Flight Engineer, and a
Navigator. In our crews, the engineer and navigator are not pilot-qualified,
and the engineer is an enlisted rank. The mission this crew was given was a
short-notice alert to provide airlift of 75 med-evac passengers from a
"near-combat", remote, small (4200ft) airfield. The political tensions
within the host country have motivated the American forces to quickly
withdraw civilians, wounded, and other non-combatants from the "near-combat"
environment.
The crew is on approach to the runway, and they have executed an
approach which is designed to avoid enemy threats. One mile from the runway,
a warning light illuminates, warning of a low fluid level in the propeller
governing system. (The flight manual clearly states the propeller must be
feathered prior to landing) The flight engineer notices the warning light,
and calmly advises the pilot that the light is on. On this first attempt,
the pilot is high and hot on the final approach, and it is for that condition
that he goes around, into a VFR pattern, right turns.
After the low approach, the pilot queries the engineer to repeat the
warning. The engineer tells him the #4 prop light is on, and the propeller
seems to be operating normally.The pilot thinks silently, while turning to
the downwind. The co-pilot and navigator are in the windows and verbally
advising the pilot about the runway location. The engineer asks the pilot if
he is going to shutdown the affected engine. The pilot responds "this is a
combat situation". No more is discussed while the pilot completes his
pattern. The second attempt to the runway is again high and hot; the pilot
goes around. During the low approach the runway controllers suggest the
pilot execute a tear-drop approach and try the other direction over the
flatter terrain. The engineer now reports the propeller is having difficulty
maintaining proper speed, and tells the co-pilot to de-select the electronic
speed governing system. The propeller speed stabilizes. The navigator and
co-pilot advise the pilot they will direct him to fliy a larger pattern, and
give him a longer final. As the pilot turns onto final approach, he asks the
engineer if they have sufficient "three-engine capability" for the take-off.
The engineer calmly says "well, not really, sort of close, yea--maybe. We
really shouldn't do it, it's your decision." The pilot then mumbles "I hate
it when it's my decision." The engineer, after a pause, says "are you asking
for a suggestion?" The pilot then says "yes, what do you suggest?" The
engineer says" the propeller is getting worse, we don't need to get stuck
here, I suggest they find another airplane to do this." The aircraft is
one-half mile on final, and the pilot has a very good final approach this
time. The pilot mumbles "gee, I'm not sure what to do." The navigator,
noticing the aircraft is less that one-quarter mile from threshold, 75 feet
above the ground, says "we'll know your decision shortly." A moment later,
the pilot adds power and announces "crew, we're on the go."
The crew aborted the assigned mission and diverted to repair facility.
So, CRMers of the metrics discussion; how would you apply your metrics
theory to this training event?
Greg Deen,
HTI--C130 ATS