<< I sincerely believe that the CRM community will not be able to
empirically test or validate their contribution in any substantive way
(beyond perception surveys) until the underpinnings are addressed. Many of
the challenges with the exiting performance measurement systems lie in the
flawed design of the assessment tools/metrics. Underlying this whole
challenge is what John refers to as the theoretical underpinnings of
empirical support. >>
Pardon my intrusion. I'm not in aviation, but am extensively involved in
human error modeling for abnormal occurrences in nuclear plants. I find much
of the dialog in CRM issues to parallel our concerns, too. For example,
after Three Mile Island, our industry drastically changed how it approached
human factors (new types of procedures, simulator-based training, improved
interface designs, etc.), but even after the programs have been in place for
~10 years, it's very hard to "prove" what has been the overall effect.
Improvement or not? Again a lack of theoretical underpinnings, only
conjecture.
Our research now is focusing on the combination of human factors issues in
particular ("contingent") conditions--i.e., training may be excellent, but
you can get into conditions (maybe because of earlier errors) where following
your training (or maybe with a single faulty display) will lead you to "do"
the wrong thing. We have some cases in the power industry, and am aware of
events like Kegworth. Our work is based on much of Reason's principles of
"latent failures"--they are part of what we search for. (Reason is an
occasional consultant to our program.) My reason for this intrusion is to
observe that the frustrations expressed are not unique to the aviation field,
but I think are partly the problem of working at the current boundary of
knowledge about human error mechanisms and the conditions that set them up in
technological settings.
I'm looking forward to the Av. Psych conference next year (I'm based in
Columbus), where I hope I can hear much more about your "side's" perspective
on this field.
John Wreathall
The WreathWood Group.