re:Re: Markers

Lt Col Tony T. Kern, X-3230 (KernTT.DFH.USAFA_at_usafa.af.mil)
Tue, 29 Dec 98 13:55:05 MST

Mike and CRM friends
Kudos on the right questions to be asking. I know that assessment is going
to be key to in competitive funding issues of the future. I have been giving
considerable thought to the process and have utilized a different approach,
one I believe to be more naturalistic, but fright with pitfalls as well. In
the 1950s, John Flanagan initially utilized the "critical incident" method to
find "decisive moments" that led to success or failure in a given event. (He
was studying the washout rate in pilot training at the time. It involves
surveying practitioners with a short "situation-behavior-consequence" form.
>From these data, a qualitative coding process is used to surface trends in
behaviors and perceptions. On the surface, it seems exceedingly simple, but
in practice it is very labor intensive. None the less, I have now used the
process over twenty times and analyzed over 5000 critical incidents, and find
it too valuable to abandon. It also is key to gaining practitioner support
for training interventions that arise from the uncovered trends. A second
drawback is that qualitative methods of inquiry come under scathing attack
from some sectors of the academe, who tend to burn incense at the altar of
pointless numerical precision. While it might not make the journals, it makes
for reasonable CRM assessment and damn good inputs for training materials to
follow-on from the assessment. Happy holidays to all!

Tony Kern

----------------------[Reply - Original Message]----------------------

Sent by:"Mike Nendick" <avmdn_at_cc.newcastle.edu.au>

At 09:44 28/12/98 +0800, Julian wrote:
>
>Dear All, I am interested to know who is using markers with their LOFT/
>CRM sim sessions. ...................
>We see the importance of marking the sim
>sessions. Is it valuable or is it critical? Does it go against the
>basics of CRM? Is it productive or counter productive? Is it a
>re-emergence of the 'check mentality'? Are the bean counters out of
>control? Is there a reluctance of pilots/managers to admit they are
>human? Do you know how well you will perform on the third missed
>approach?(sress!!!!) ........................
>So, should we mark or not???????

Julian
Hi there. I believe that this should be an interesting and important thread
as CRM goes into "5th generation" and beyond.
There is no doubt that there are serious pros and cons to the assessment
issue.
However I think that without comprehensive evaluation of process and
outcome, including group and individual performance, there can be little
improvement nor understanding of what is working, what is not, and why. Also
snap shot assessment can be supplemented by studies over time to pick
trends and so on. At some stage individuals will be made accountable for
poor human performance on the flightdeck, in the cabin, or on the hangar
floor beyond the technical focus of current assessment systems.
The critical factors will be defining appropriate markers, assessing
objectively to them, and using the results primarily as a training and
development tool, rather than a heavy handed threat and negative
reinforcement regime.
It would be interesting to compile the markers in current use in airlines
and their application in the assessment process.
There is a lot of work ahead in this area of human factors evaluation for
aviation personnel.
"We" need to educate before "they" regulate....., don't we?

regards, Mike

Mike Nendick
Department of Aviation & Technology
The University of Newcastle
Callaghan
NSW 2308
AUSTRALIA

ph (++61-2) 4921-7396
fax (++61-2) 4921-8742
homepage: www.newcastle.edu.au/department/av/index.

=====================================================================