Re: Markers

Mike Nendick (avmdn_at_cc.newcastle.edu.au)
Mon, 04 Jan 1999 10:27:50 +1100

At 12:53 3/01/99 +0000, Piers wrote:
>If you haven't read it already, you might be interested in UK CAA paper
>98005 which addresses this [assessment] issue.
>
>JAR OPS already requires assessment of CRM as part of the overall
>assessment of operator proficiency checks (base checks), line checks and
>command upgrades (but not LOFT). Given that we are required to assess, it
>seems to me that the issues of reliability and validity must be addressed,
>though the validity issue is particularly problematic; as Rick Heybroek
>says: it is not really clear how closely specific process variables are related
>to outcomes.
>
>Regarding reliability, I was talking the other day to a fleet Senior
>Training Captain who expressed the opinion that, since he would personally
make the final
>judgment regarding a pilot's career ("chopping", demotion etc.) the issue
>of inter rater reliability between his other training captains was not
important.

Piers
Thanks for your response. I have read the 98005 paper, and have followed the
JAR OPS development which is one of the reasons my interest in this subject
is being rekindled. Certainly if JAR/FAA regs harmonise on this issue, and
then other regulators such as Australia and NZ CASA & CAA respectively
follow suit, there will be some interesting changes in current airline
processes to occur - at least in this part of the world....
Rick's point is valid as without comprehensive evaluation how do we
determine any type of validity? What exactly should we be assessing?
Like current selection methods, we can well determine those who will pass
the initial training courses, but the real outcome is years down the track
and that we cannot yet precisely determine, nor are we sure of the relevant
selection criteria for long term success, which in the airlines is captaincy
I guess as one example.
On your latter point regarding reliability I have too heard similar
sentiments locally, thus my point of the importance of educating within or
without any regulatory framework.
regards, Mike

Mike Nendick
Lecturer/Researcher
Department of Aviation & Technology
The University of Newcastle
Callaghan
NSW 2308
AUSTRALIA

ph (++61-2) 4921-7396
fax (++61-2) 4921-8742
email: avmdn_at_cc.newcastle.edu.au
homepage: www.newcastle.edu.au/department/av/index.