In a message dated 97-07-27 20:01:35 EDT, you write:
<< If I have students take a written examination on the terminology and
definitions presented in a course of instruction, am I only testing recall?
Isn't the real test to see if actual usage of the CRM techniques described
by the terminology will occur after training?
Is it enough to test for memory recall? Does that "validate" the
training?
>>
In our view, you are quite right. While we have been strong proponents of
testing for the past fifteen years or so. By the same token, other than
regular "homework" assignments (it's a 12-month course) we do not have any
paper and pencil training or evaluation aids (and while we do provide
feedback on all the "homework" assignements, we do not grade them.)
Our goal is improved decision making performance on all flights. We feel that
this is the only valid evaluation. If you are looking to evaluate either the
crews or the training itself, the evaluation should ideally be delayed -- a
year or more after the initial training (after all, everyone should be doing
on-going recurrency training). Six months is an anbsolute minimum if you just
can't wait for some sort of results. We also prefer to evaluate line crews.
This is a more realistic environment than the simulator. Of course, you can't
do this and get reliable results with all CRM programs, as the material is
either too subjective or the skills are so dependent upon externally
triggered events that you may not be able to adequately observe the CRM
skills on all flights. However, a well-designed process-driven CRM system can
be quite effectively evaluated on any flight, even the easy, routine day VFR
flight with no problems (traffic, terrain, passenger, scheduling,
equipement....)
Skip Mudge
CMR, Inc