Robin asked for non-aviation experiences. At the same time Vince Mancuso
confirmed the importance of performance data collection as part of
programme effectiveness audit. I think the two are related.
Work with railway operators in the UK convinces me that the difference
between viewing training as (a) one-shot intervention and (b) an ongoing,
monitored process is absolutely crucial for any kind of effective training
to happen, regardless of the specific training context. This confirms a
comment of Clay Foushee's - AQP makes sense as a paedogical system
independant of the specific training application.
The rail problem I was looking at was Signals Passed at Danger (SPaD), the
equivalent of altitude bust. This is a chronic problem and none of the
usual initiatives (SPAD database, Driver At Risk programme, retraining)
showed much success in reducing the frequency or severity of incidents and
accidents (sound familiar?).
In a fairly typical case, a driver slid through a red signal after
apparently ignoring a reminder board. He was put on the At Risk programme,
the offence was classified as a Disregard and he was given supplementary
training on rulebook signals procedures.
Looking more closely, we found that the same signal had been involved in
SPAD incidents with other drivers 7 times in the previous 5 years, about
once every 9 months. The problem was recognised, so the reminder board had
been installed recently. Right on time, after another 9 months, our driver
missed his signal. Further points: although classified as a Disregard (a
serious infraction), the handwritten account from the Traction Inspector
clearly described the incident as Distraction, a less severe offence, due
to a late-boarding passenger with baby at the immediately preceeding
station. But the wrong box had been ticked, and the system was geared to
respond only to machine-encodable report data, not to freeform accounts.
In fact, there was no way to accurately reflect cause on the SPAD reporting
form: the only tickable causes were driver error or environmental
conditions.
Was the driver's situation awareness at fault? He forgot the last signal
passed was at Yellow, because it was situated before the station. But the
next signal should normally have cleared to Yellow before he left the
station, somewhat late due to the PAX. This time the signal did not clear,
because it had failed to red (maintenance defect).
In this case, since the causes of incidents were passively excluded from
the reporting system real causes were never identified or addressed with
suitable (human factors) training and reinforcement. Since no training
existed to counter human factors hazards, any retraining actually given was
in fact either pointless or punitive.
Most importantly, since there was no quality management system to identify
problems, propose structural and training solutions and monitor real
training effectiveness, the situation never got better - no matter how many
technology fixes were added to the pie.
Recently we had a major UK rail accident involving a collision between a
freight train and passenger express at Southall. The cause? It looks, so
far, like SPAD. And the first reaction has been to charge the driver with
manslaughter.
Summarizing, Dr. Mancuso is stating best practice. There is no point in
investing in training if that process is not designed to be monitored
routinely. The AQP programme provides a good structure for data collection
and monitoring, but any well-implemented ISD system will do that. "First
look" is a particularly interesting approach but I suspect that less formal
methods could achieve the same end. The key factor is to ensure that
training is an open feedback loop with as many feedback sources as you can
manage, especially incident reports from your own operations and pilot
assessments based on some approach with real inter-rater consistency. This
does take time, but the alternatives are even more expensive.
This sort of training structure is non-specific: it applies to process
control, including petrochem. as well as operating rooms, medical emergency
handling, rail and marine - all topics which have come up in this forum at
times in the past.
Apologies for some digression, but there are large areas of industry which
continue to make elementary safety-critical mistakes simply because they
haven't benefitted from aviation experience. Robin's work deserves our
support.
Regards,
Rick Heybroek
LOFTwork ltd.
>>>>>>>>>>>>>>>
Vince said:
>>>Performance measurement systems could take months to design and take
even
longer to show performance trends. Performance data collection designed to
show trends should begin from day 1 of the program. Performance
measurement should be a part of initial CRM program architecture.
Task analysis and performance measurement are the two parts of the
instructional systems design (ISD) process that are glossed over with
disturbing frequency in the CRM community. Airlines that elect to build an
AQP, for example, will not get their programs approved unless the program
has clear and unambiguous CRM task list as well as performance measurement
systems. <<<
<<<<<<<<<<<<<<