CRM Training Fails Because of What Trainees Already
Know;
Not Because of What They Don't Know
Paul Baxter
Transfer of learning in training
In many training programs the learner may appear to be able to adopt the desired behaviors during the training session, e.g., "crew members ask questions regarding crew actions" (one of Helmreich's Crew Performance Markers) but when they get into the air much of what they apparently learned during training sometimes seems to disappear. This is especially noticeable when things get busy or in emergency situations. Under high work load and stressful conditions people invariable revert to their "own way" and forget their training. This is known as the, "transfer of training problem".
The transfer problem plagues most training efforts. For example, recent research from the University of Texas tells us that it can take up to 1,000 hours for an experienced pilot to become fully competent and comfortable with the flight management system. The transfer problem applies to all training and learning situations; not just flight training - it is universal and hence extremely important.
The traditional answer to the transfer problem is to make them practice - "just keep practicing and it will come, eventually". While this is true and most will catch on soon and others will take longer, this time honoured remedy is very slow and expensive. The effects associated with the transfer problem, namely frustration, extra training time and cost, dropout rate, increased likelihood of error, incidents or accidents, all take their toll.
The transfer problem is the real bogey in CRM training (and in any other training) and it has to be addressed.
Old learning interferes with new learning
I maintain that one of the main causes of the transfer problem and the associated slow rate of learning progress is interference associated with the learner's prior knowledge. Much of the earlier psychological research by a fellow called Benton Underwood taught us the powerful effects of prior
knowledge on attempts to learn new knowledge and skills. The brain mechanism responsible for the proactive (forward acting) interference exerted by prior knowledge over new learning was called proactive inhibition or PI for short. PI is best understood as the interference (forgetting) exerted by prior (old) knowledge over new knowledge. PI explains why old habits die hard and why trainees often appear to forget what they have just learned and revert to their old ways of behaving and performing. PI lost its popularity as a research topic many years ago. Psychology is, unfortunately, obsessed with recency in research and tends to ignore much valuable early work. However, the PI research still remains unchallenged and it was not until recently that its full implications for all kinds of training and learning training have become apparent. More about this later.
Australian National Training Authority Research
I did some research in 1996/1997 in the occupational skills area with Australian government funding to trial a new approach to changing work habits/behaviors and skills. This was a comparative methods study incorporating a control group and an independent statistician did the data analysis. In this controlled field experiment we compared conventional methods of training and error correction with a new method called Old Way/New Way, developed in Australia by Harry Lyndon, an educational psychologist. I should point out that we achieved a degree of experimental control and precision not usually obtained in comparative methods experiments - while most would agree that this kind of research design is highly desirable, you don't see too much of it because it is difficult to set up. Consequently, we felt we could place some degree of confidence in our research findings.
We had 3 matched groups of trainees totaling 34 adults in 8 different manual skill areas and 12 trainers. Trainees had typical performance errors and bad work habits that trainers encountered in students year after year. These performance faults had proven resistant to all correction attempts yet were important for developing competence. In other words the kinds of work habits and errors they made posed significant problems for them.
The research design was as follows: one group was corrected using "conventional" skill correction which typically can be described as, "No, don't do it that way (usually accompanied by explanation); watch me - do it this way; now you copy it; now go away and practice it." Just about all teaching and training follows this general format. The second group received skill/habit correction using Old Way/New Way which is a metacognitive approach to habit and skill correction. It helps the person first develop an awareness of their own (incorrect or sub-optimal) way of behaving or acting, then enables the person to discriminate their way from the "correct" or better way of acting; and then gives them practice in that new way. The third group of trainees received no correction, although they were of course corrected when the experiment was completed, in time to take their exams. The control group was necessary to confirm that any observed improvements in learning and error rate reduction were actually due to the treatments (method of correction) and not due to some other confounding variable such as "getting smarter", "getting tired", and so on.
The results indicated that Old Way/New Way was vastly superior to conventional skill/habit correction. Habit correction was improved from an average of 4% correct before intervention to 94% correct afterwards when using Old Way/New Way. Conventionally corrected trainees improved from 4% to 25% and the control group showed no improvement. Furthermore, the dramatic improvement obtained with Old Way/New Way was immediate after one session (there was no "adjustment period" to the new skill, technique or procedure as is usual with other methods), was maintained over three post-test periods, and was irrespective of skill type being considered.
Advantages of a new method of training
This experiment, while dramatic, would of course not be conclusive by itself but when considered with other published research evidence and years of practical application (both paid and unpaid) in industry, education and sport the picture that emerges is very clear. Old Way/New Way is very effective in changing actual human behavior and misconceptions and improving conceptual understanding in many different performance settings. We typically achieve 80% or higher transfer of skill after one or two sessions and we expect 100% behavior change after 2 to 5 brief sessions spaced at fornightly intervals. The learner is also able to apply self-correction after one session and since he or she becomes 90% effective in detecting whenever they revert to their "old way", the rate of improvement is understandably rapid. Old Way/New Way can deliver these learning gains because it by-passes the PI mechanism. So now, old habits no longer die hard.
Some of the recent applications of this methodolology include changing work habits among aluminum casting pit crew members, flying light helicopters (re-training pilots to use the correct technique for recovery from rotor stall), and in competitive sport (quickly correcting technique difficulties that cause injury or make for uncompetitive performance in golf, cricket, football, rugby, swimming, track and field, etc). These case studies are all described and explained on my web site: www.personalbest.com.au The case studies also include an example of how assertiveness training was accelerated and how this changed behavior quickly transferred to the workplace. While solid research results can be impressive, to me it is even more important to be able to show that the method actually works with real people in the real world where they have to perform, often under pressure.
Old Way/New Way is the result of many years of effective collaboration between researchers and practitioners in many applied fields. Consequently, it is not only an effective learning method but it also has a sound theoretical underpinning. A detailed explanation of the significance of interference from prior knowledge, the meaning or errors and how instructors deal with them, and the PI effect and how this is overcome therefore now follows.
We believe this theoretical framework is a novel interpretation and synthesis
of established learning principles and is soundly based on established
research in learning, errors, habits, memory and transfer. This methodology
has strong implications for the acceleration of learning in all areas
including conceptual as well as skill learning.
The theoretical underpinning for this innovative methodology is covered
under these headings:
- The importance of habits
- What can we learn from mistakes?
- How do teachers respond to errors?
- Why do old habits (knowledge, skills, attitudes, beliefs) die hard?
- Why is prior knowledge so important?
- What are the implications for training practice?
- A new model for training and learning
- What are the benefits, limitations and special requirements of Old Way/New Way?
- List of published research papers
The importance of habits
We learn things by paying attention, observing, copying and practising. We know we have "learned" something when we can remember it, i.e., recall some information or repeat an action when necessary. Learning, an internal process not directly observable, is therefore inferred from an observed change in behaviour or performance.
Having been practiced over and over, learned actions and reactions have become instinctive and happen without us having to concentrate on what we are doing. We say our brain is on "automatic pilot". We are operating under "force of habit". Most of what we do from day to day consists of those automatic sequences of actions and reactions we call habits. When what we do automatically is correct and appropriate for the situation, we say we have developed, "good habits". Good habits or skills enable us to function effectively. However, when our actions and reactions are inappropriate, maybe even unsafe, we say we have developed "bad habits". By the time we realise what we have done it is too late to stop. We find that we are, "the prisoners of habit".
What can we learn from mistakes?
When someone makes an error it is usually taken as a sign that the person has _not_ learned something. Many errors result from careless mistakes, inattention or inexperience and these kinds of errors usually disappear as the person gains mastery. Lets call these Type 1 errors.
Unlike the random errors made by beginners, errors made by more experienced performers show a pattern - they are consistent, repetitive and clearly not due to carelessness and inexperience. These "expert errors" are also persistent in that they resist eradication attempts. They are examples of what are called "habit errors", "learned errors" or "interference errors". Let's call these Type 2 errors.
Instead of indicating the absence of learning, habit errors are a sign that the person _has_ learned something - he has learned how to do it "wrong". Error analysis can help identify which kind of error we are dealing with by looking for a pattern or consistency. Type 1 and Type 2 errors require quite different teaching approaches.
How do trainers respond to errors?
Type 1 errors indicate that the desired learning did not occur. Perhaps the person was inattentive, careless, distracted or unmotivated or the teaching method was inappropriate or the learning was unsuccessful for some other reason. The remedy is to re-teach using appropriate compensations and adjustments. Type 2 errors, however, indicate that learning did take place but instead of learning the "right" way, the person unfortunately learned a different and "wrong" way.
Conventional training methods point out errors whenever they occur and then to emphasize the correct answer. Because errors are believed to indicate the absence of learning, i.e., failed learning and by association possibly failed teaching, trainers prefer not to dwell on errors any more than necessary. This predisposition is also motivated by the mistaken belief that paying too much attention to an habitual error will make it even more difficult to eradicate. What the trainer is basically saying to the trainee in all this is, "That's wrong; don't do it that way; I don't want to see you doing it wrong; do it this way; copy me; now go and practice the right way". This approach can be effective over an extended period of intensive remediation but learning gains are typically slow. Transfer of learning to performance settings outside the remedial setting is usually poor because whenever the trainer is not directly attending to the trainee the cues to correct performance are removed and so the trainee typically reverts to his old wrong way.
Type 2 errors indicate that we are dealing with an _unlearning_ situation rather than a straigh forward re-teaching situation. Incorrect prior knowledge stands in the way of learning progress and it cannot simply be ignored or glossed over; it must first be corrected, i.e., unlearned.
Why do old habits (knowledge, skills, behaviors, attitudes, beliefs) die hard?
Prior knowledge and skills can either enhance or interfere with subsequent learning. If pre-existing learning is correct and is consistent with new learning, then the old enhances the learning of the new. However, if prior knowledge is faulty and therefore differs from the new, correct information being taught, then the brain immediately detects this conflict and involuntarily activates a brain mechanism called proactive inhibition.
Proactive inhibition (PI for short) is a knowledge protection mechanism. PI was intensively studied by Benton Underwood and its powerful interference effects are demonstrated in classical psychological experiments using the mirror drawing apparatus, the learning of word lists and the Stroop Colour Charts. PI can be described as, "old learning interfering with new learning".
Underwood's ground breaking work spanned a lifetime and still remains unchallenged but proactive inhibition eventually became somewhat unfashionable (psychology suffers from an unfortunate obsession with recency in research). It was not until 1989 that Lyndon developed a synthesis of the PI literature and presented his novel interpretation of how PI was the underlying cause of most non-achievement and apparent failure to learn. According to Lyndon, PI's function is to automatically protect everything we have learned and to resist and slow down any attempt to change these things. PI preserves prior knowledge against the onslaught of change demands. Without PI we would face having to relearn everything from one day to the next.
Unfortunately, because PI does not "know" whether our current knowledge base is "right" or "wrong", it equally protects incorrect as well as correct knowledge and skills. This is why old habits die hard. PI works by accelerating the forgetting of new knowledge and skills whenever these conflict with prior knowledge and skills. Telling a person he is wrong, however well-intentioned and received, only serves to activate PI which then inhibits retention of the new learning the teacher is trying to impart. Within a matter of minutes or hours the learner appears to forget what he has just been taught and reverts to his old incorrect knowledge and skills. This is one of the main reasons why people so often return to their old ways after soon after being taught something new, despite being well motivated to change. Everyone has PI but some have more of it than others, hence some individuals adapt to change remarkably easily while others find it hard going.
Why is prior knowledge so important?
"The single most important factor influencing learning is what the learner already knows. Ascertain this, and teach her/him accordingly." (David Ausubel). Most trainees are not a "blank slate" for the trainer to write to. Even beginners already have at least some knowledge or experience of the topic or skill being taught. More experienced practitioners come to the learning situation loaded with prior knowledge of the topic or skill being taught.
Some of this prior knowledge is correct but often, for one reason or another, some of it is incomplete or incorrect or both. People often, "get it wrong". When these wrong ideas and technique problems are not picked up early enough and corrected, they become ingrained and like all bad habits are then much harder to change. Consequently, many trainees come to the classroom equipped with a raft of ingrained misconceptions, learned errors, technique faults and bad habits upon which to base subsequent learning. In other cases, a person's prior knowledge can soon become outdated or unsuitable because they need to change their knowledge and skills due to upgrading requirements or because they are transitioning. In such situations the errors that trainees make and the length of time it takes to achieve competence and feel comfortable with the "new way" poses real problems for both trainee and trainer.
What are the implications for teaching practice? A new model for training and learning
The Lyndon Model offers educators and learners an alternative to straight forward re-training. It is called Old Way/New Way (copyright E.H. Lyndon 1973). [To avoid possible confusion over terminology, the Old Way/New Way methodology is also part of a learning package designed by Lyndon called the Conceptual Mediation Program (CMP), Copyright Department of Education, Training and Employment, South Australia].
Instead of always re-teaching, the trainer now has an additional methodology to add to his or her repertoire of skills. Which method to use depends on the results of diagnostic testing and error analysis. If testing reveals underlying Type 1 errors, then re-training is in order. On the other hand, if Type 2 errors are present then re-teaching will not work because telling the learner he or she is "wrong" will arouse proactive inhibition and this will protect his or her prior (incorrect) knowledge and skills from change by causing accelerated forgetting of the new (correct) knowledge, thus producing an apparent re-emergence of the prior incorrect learning (reversion to the old way). Type 2 errors require correction using Old Way/New Way because this method bypasses the PI effect and allows rapid change to occur.
We maintain that because few learners are a clean slate the likelihood of finding learned errors or "old ways" is actually quite high. Learned errors tend to be underestimated due to the predominance of a particular testing procedure. Conventional assessment and testing is typically aimed at testing recognition rather than recall. In other words, in a recognition test some cues to correct performance are usually present and more or less obvious. Recall testing on the other hand is much more demanding and even more so is testing for application of knowledge. Since recognition testing is the norm and cues as to the correct performance are present in this type of test, the extent of incorrect prior knowledge and skill is probably underestimated by these tests. Recognition testing does not test for transfer of learning, in other words. The lack of transfer only shows up later in real world performance situations.
We maintain that, apart from obvious cases where student motivation and ability and teaching prowess are clearly deficient, most non-achievement in learning is in fact due to the interference effects of prior learning (PI) and is not due to "learning disabilities" as such.
Furthermore, PI is a problem not just with classic non-achieving or "remedial" learners, although it is likely that these individuals have fallen into this predicament because they have more PI and hence more "wrong old ways". Misconceptions and technique problems are widespread in every aspect of learning and human performance. Because the level of PI does not seem to be correlated with intellectual ability in general, learners of average or higher ability also suffer to a greater or lesser extent from misconceptions and skill difficulties.
Misconceptions and technique difficulties are not unknown even among elite and highly experienced performers as revealed in studies of elite athletes and experienced airline pilots, for example. University of Texas flight training research, e.g., showed that even experienced pilots had "wrong" ideas about the way the FMS functioned and consequently tended to either over- or under-estimate its capacities and acted accordingly. Prior knowledge, whenever it is incorrect or incomplete, means that it takes longer for learners to become competent and knowledgeable in their field.
To some extent then, much of what we try to learn can potentially suffer interference from the PI effect due to the ever-likely conflict between our prior learning and the new learning. This interference and the associated accelerated forgetting of the new knowledge becomes apparent in greater time (and expense) required to achieve competence. Since interference from prior knowledge can affect almost all learning, we claim that almost all learning can be _accelerated_ once these interference effects are acknowledged and an effective compensatory learning methodology is applied.
How do you apply Old Way/New Way?
The Old Way/New Way process has a number of steps. How these are applied and what I actually say and do with the person who is trying to change depends, of course, on the specifics of the "problem" but broadly speaking these are the steps involved.
Step one is error diagnosis. I have to determine if we are dealing with a learned error or a simple transient "mistake" because the remedy is different in each case. Mistakes can be dealt with using conventional training or re-training or perhaps they are to be expected as part of the normal learning process. Learned errors, however, require Old Way/New Way because trying to correct them using conventional means (e.g., "that's wrong, don't do it that way; watch me - do it this way; now you copy me; now go away and practice") does not work - it arouses proactive inhibition and causes interference with new learning (that very thing you're trying to teach the person) so that the new learning suffers accelerated forgetting (within minutes or hours) and the old inappropriate learning returns. This is the basis of the transfer of training problem so well known to trainers and trainees alike.
Error diagnosis relies on expert input, e.g., from a flying instructor. I'm not a pilot or a sports coach so when I work with pilots or elite athletes I'm part of a team who works out the problem. The team consists of the pilot who is trying to change; the instructor and me as the change facilitator. Together we work out what's going wrong. My contribution is to help the person change over to the "new" way, once this has been identified. At this point you can probably see how this methodology would ideally be part of the instructor's own set of on-the-job training tools on which he or she could draw during normal day-to-day instruction.
Step two involves teaching the person the difference between his or her "old" and "new" way, i.e, the way they normally or often do it, and the way they should be doing it. This difference is then reinforced until it is clear in their heads, before proceeding with the final step which is the practice of the "new" way.
Old Way/New Way often appears to be counter-intuitive
Conventional training and teaching methods never spend much time on the "error" apart from pointing it out first time and reminding the person each time they do it wrong. Trainers are not interested in seeing errors because they are a sign that the desired learning has not occurred - the training that was provided did not take - training failed. Old Way/New Way practitioners look at errors quite differently - if the error is consistent then it indicates that learning did take place. Unfortunately, for one reason or another, that learning was incorrect - he learned to do it "wrong". But you have to start with what the person already knows, which is their error. You can't ignore it and simply teach the right way over the top of it and hope that the wrong way will go away, because it doesn't go away. It returns to haunt you when you least expect it.
Conventional training simply points out the error and then goes straight into showing the person why its wrong, what they should be doing instead and then gets the to practice the right way straight off. Conventional training never spends much time looking at what kinds of error patterns the person reveals, and even if it did do that and found that the person's error was consistent and therefore "learned", it has no other remedy to offer than the time-honoured one of telling the person to practice, practice, practice.
Here's another issue. Your training may appear to be successful because the trainee is doing all the right things during training and also in the simulator and most times when up in the air. But when the going gets tough and stressful then and only then will you find out whether what you taught them has really taken. It is in those moments that people revert to their early experience, their familiar, well-practiced routines, even though these may be sub-optimal, completely innapropriate or downright dangerous in the situation.
Old Way/New Way by-passes the interference from prior learning and accelerates the change process because it addresses these fundamental learning processes, while conventional training does not really deal with them at all.
Another part of what I do is try to make the person an informed participant in the self change process. Remember that the process enables individuals to change themselves - no one can change us - change must come from within. I find it works better when the person or crew receives some preliminary instruction in our novel version of the learning process; why old habits and skills die hard, why self change is so slow and frustrating and how this knowledge will enable them to take direct personal control of the self change process. While it is also possible to ,"do it" without giving them this knowledge, the results are not as good and the counter-intuitive aspects of the methodology tends to put people off if they haven't been primed beforehand. After all, what would you think if I asked you to "do it wrong for me", again and again? Without the proper sharing of knowledge about the methodology the client would soon start to treat you with suspicion, and understandably so.
I hope this explains broadly yet clearly enough what I do as an Old Way/New Way facilitator of change. The range of applications for this methodology are still being charted and each day we hear of new and exciting ways people have used it to change their lives.
What are the limitations and special requirements of Old Way/New Way?
Old Way/New Way is not for everyone and it is certainly no quick fix. We like to think of it as an "intelligent fix". For the method to work you really do need someone who is able to identify exactly what is being done wrong, what should be done instead and what the differences are between these two ways. Not everyone is that analytical.
Secondly, you need people who are willing to change or at least see a need to change. The methodology is able to sort out those people who are *really* unwilling from those who appear to put up a resistant front because they believe, deep down, that it is impossible to change themselves (based on past experiences of failure perhaps). If you're truly unwilling and refuse to cooperate then nothing, not even Old Way/New Way will help you change. However, even where there appears to be an "attitude problem" or a "culture problem" believed to be responsible for lack of change, we have shown that Old Way/New Way, when properly handled, can achieve self change among employees (refer to the ALCOA/KAAL case history). Thirdly, for the methodology to work you need someone who is prepared to practice their new way. Without practice, as we all know, what you learn is soon forgotten and this is no exception.
Finally, self change is an intensely personal process and can be demanding. The person must be prepared to put in the effort and cooperate with the facilitator all the way through the process. The facilitator can share with them the knowledge they need to achieve self change, but the effort is something they alone can produce. However, we can usually promise them that if they are prepared to put in the effort then they will get the rewards; whereas with conventional training and learning they might put in the same effort or more and be disappointed with the result.
© Dr Paul Baxter, Personal Best Systems®, 1998. All rights reserved. Brisbane, Australia.
Published Research (copies available on request)
Baxter EP et al. 1996. Skill Correction and Accelerated Learning in the Workplace. Paper presented at the Annual Conference of the Australian National Training Authority Research Advisory Council, 3-4 November, Melbourne.
Baxter EP & Dole SL 1990. Working with the brain, not against it. British Journal of Special Education 17:19-22.
Dawson, Chris and Lyndon, Harry. 1997. Conceptual Mediation: A New Perspective On Conceptual Exchange. Research In Scence Education. 27:2:157-173.
Dole SL 1991. New ways for old. Teaching Mathematics 17:3.
Lyndon EH 1989. I did it my way! Australasian Journal of Special Education 13:32-37.
Rowell JR et al. 1990. Changing misconceptions. International Journal of Science Education 12:2:167-175.
Department of Education, South Australia. 1983. Education Gazette. Vol. 11, No. 11, p.289.
E-mail: pbaxter@personalbest.com.au.
Copyright © 1996-2005 by Neil C. Krey unless otherwise indicated.
Non-commercial reproduction rights granted if the following notice is included:
"Source: Neil Krey's CRM Developers Forum, http://www.crm-devel.org"