>
>Tracy et.al.
> I commend you on your scientific approach and efforts with respect
>to CRM and its impact on the U.S. Naval Service. Having come from the U.
S.
>Navy myself I agree that it has been a great resource for flight crews. I
>do, however disagree with two points. Having been an ACT Instructor
myself,
>the program in the mid 1990's did in fact embrace the goal of eliminating
>human error. I understand this goal, however, I was always skeptical as to
>the ability to do just that(error elimination). The program did not take
the
>approach of error management which has recently become a more widely
>accepted framework.
>The other point is that I disagree that mission success is the only data
>point to be used to assess a CRM program. I can give you a few examples in
>the Space Shuttle operational arena were all mission objectives were
>completed above expectation and there were major CRM flaws present. Many
>more examples can given within the training environment. The mere fact
that
>one received CRM/ACT training does not guarantee mission success.
>Conversely, mission success does not always imply that the most effective
>CRM practices were used. What I do believe is that the more one is exposed
>to ACT training the more likely one applies its concepts into the cockpit.
>The success and effectiveness of a CRM/ACT program largely depends on how
>pervasive the concepts are embedded within an organization's culture. I
>believe the observable skills are the best measure of a programs success
and
>not necessarily mission success. We in the Space Flight Training world
>challenge shuttle crews to critique crew performance with respect to the
>crew processes and not just on the technical outcome. If during a
>simulation a crew performs an fuel cell shutdown on the wrong fuel cell and
>successfully recovers and safely lands the vehicle, the mission may have
>been successful yet major CRM issues may be present. What process lead to
>the mistake or simply put why was the incorrect fuel cell shutdown?
>Communication, Situational Awareness, Crew coordination may be the problem.
>
>Just food for thought
>
>David Rogers
>Guidance & Control/Propulsion Instructor
>Space Flight Training Division
>Mission Operations Directorate
>Johnson Space Center
>Houston, TX
>281-244-7354
>> ----------
>> From: TRACY S. DAY[SMTP:tmday_at_leading.net]
>> Sent: Thursday, July 30, 1998 5:32 PM
>> To: crm-devel_at_db.erau.edu
>> Subject: Re: Assessing HF programs
>>
>>
>> Lt Col Kern,
>> ACT in the military is meant to be a tool to increase mission
>> effectiveness.
>> This is mostly done by eliminating human error (human factors). In a
>> recent
>> study I completed I used weapon drop accuracy as a measure of mission
>> effectiveness for a multi-crew aircraft the P-3. I used paired t-test to
>> compare the drops of ACT trained crews to non-ACT crews and found a
>> significant difference with a 95% confidence level that yielded a higher
>> mean for the ACT trained crews. Assessing human factors must use some
>> measureable trait and mission sucess is only one. A more detailed and
>> costly approach would measure the amount of error reduction created by a
>> training program. One tool that can help accomplish this is CADS a
>> computer
>> aided debrief system (developed by ERAU labs and Airborne Data
>> technologies)
>> that allows digital audio and video of the simulator to be recorded along
>> with abiltiy to use a hand held computer to mark error times and the type
>> of
>> error for feedback during debrief. A program that would allow an outside
>> contractor to measure effectiveness through a system like CADS would be a
>> more objective and detailed review. My graduate research paper will be
>> complete by the middle of Aug and if you would like a copy to see how I
>> evaluated the P-3 ACT program I can probably email it to you.
>> V/R
>> LT Tracy Day
>> VP-30 ACT program manager
>> -----Original Message-----
>> From: Lt Col Tony T. Kern, X-3230 <KernTT.DFH.USAFA_at_usafa.af.mil>
>> To: crm-devel_at_db.erau.edu <crm-devel_at_db.erau.edu>
>> Date: Thursday, July 30, 1998 10:41 AM
>> Subject: Assessing HF programs
>>
>>
>> >
>> >Collegues
>> > I have undertaken (been tasked) to develop a model for evaluating
>> >"human factors training courses" (whatever that means). My early
>> >research indicates that most organizations -- if they dare to assess at
>> >all -- currently use safety stats and student satisfaction to gauge the
>> >effectiveness (whatever that means) of their programs. For reasons that
>> >are obvious to this group, these criteria are inadequate. My task is
to
>> >develop a tool that an "outside assessor" (whatever that means) can use
>> >to evaluate a contractor delivered training program. I have four
>> >questions for the group.
>> >1. What measures (other than student satisfaction and safety stats) are
>> >used in your programs to assess program effectiveness?
>> >2. What, other than safety, are the goals and objectives of an HF
>> >training program?
>> >3. What other tools might one consider for assessing HF program
>> >effectiveness?
>> >4. What other research has been done in this area?
>> >
>> > Thanks
>> >
>> > Tony
>> >2. How do you feel about an outside assessor evaluating your program
>> >with a standardized tool?
>> >
>>
>