Changing education: It works over there, so let’s try it here

PISA studies have been the single biggest source of mis-direction in education policy change in my opinion (I haven’t counted it. When I get the time, I might). When Finland was on top of the league tables,  everyone wanted a Finnish-style education system. The various education tourists from countries all over the world visiting Finland were looking at what was happening at that particular point in time as the model for achieving the very impressive results that were being attained in 2000 and using their observations as the basis for forming education policy in their own countries. The problem is that results at any particular point in time are the cumulative effect of the many years of education preceding that snapshot.

Tim Oates has dissected the longitudinal factors involved in the success of the Finnish system in this paper on Finnish Fairy Stories. Far from the much touted successful snapshot of PISA 2000, he reveals a successful revolution of the Finnish system transitioning to comprehensive education starting in the 1970’s which enabled the Finns to achieve the results they did. However, he also suggests that what was happening in the late 1990’s in Finnish education, leading up to PISA 2000 lead to a gradual decline in standards. This is borne out by subsequent PISA studies where Finland has dropped in rankings.

Gabriel Sahlgren has published a fascinating, detailed study of the real success story of Finnish education and lessons other education systems could learn from its development: Real Finnish Lessons. In this paper, he talks about the ‘iron cage of history’: The fact that results are cumulatively built over time and that rather than look at what is happening now to explain successes or issues in an education system, we need to look at what has happened in the past to enable it to become the way it is.

This focus on contextual factors, and historical influences is at odds with political desire. Politicians are looking for fast, quick fix solutions to intractable educational issues. After all, it is the immediate successes that generate votes. However, focussing on low hanging fruit without dealing with the ageing older crop or the ripening new one, leaves problems on the tree for future generations to deal with: exactly the situation Finland is facing at the moment.

Now everyone wants to be like Shanghai. Well in an interesting experiment, the BBC catalogued five Chinese teachers taking over a British school. Here is what they found. Similarly, the conclusions the programme comes to is that the education system in China works because of the supporting context: societal demand for positive results, parental support (and pressure) of learners to achieve high standards, a strong work ethic, uncompromising control structures, strong sense of duty, respect for teachers and strict discipline common in the wider society. Many of these factors are alien in modern-day Britain.

There are lessons to be learned from other contexts.The problem with educational tourism is, that, like that beautiful multi-coloured wall-hanging you bought in Vietnam that really doesn’t fit with your Scandinavian living room at home, what appears to work in that context, may not when ported to a completely different one.

Monitoring large scale teacher development projects

Why monitor?

Monitoring large-scale ELT projects is essential to enable project managers to accurately assess changes happening as a result of training; to represent training programme outcomes accurately to stakeholders and to ensure that the change project is on track.

Monitoring throughout the project lifetime ensures that what is happening is what is supposed to happen. It focuses on the processes of development to ensure project outcomes are achieved. By forming a constant framework for information flow through the project, monitoring not only generates data for evaluation, but enables action to be taken to ensure the project progresses as planned.

Evaluation processes are often prioritised over monitoring. This can be damaging in that it focuses too much on results and not enough on the processes that achieve them. Over-focus on evaluation can also happen when finding evidence to illustrate project results is considered only as an afterthought. This leaves it too late to build in monitoring processes, and loses the opportunity to use monitoring data to take action during the lifetime of the development process. (Markee 1997)

An ideal monitoring framework

Looking at the whole project cycle from initiation, through implementation to institutionalisation (Fullan 1989), I created an ideal framework for project monitoring and evaluation with Kirkpatrick levels (see Table 1 and link above for what I call ‘The Big Scary Diagram’) on one axis, and project stages (before, during, immediately after, 2-3 Months after and 1-2 years after) on the other.

Table 1: Kirkpatrick’s four-level model of evaluating teacher response to training (after Kirkpatrick 1998. See also Fullan 1989)

Level	Description	Key question to answer
1	Reaction	Did trainees like it?
2	Learning	Did they learn anything?
3	Behaviour	Could they use what they learned?
4	Results	Did they change their workplace behaviour?

The Kirkpatrick levels build on one another from the shallowest response (reaction) to the deepest (results).

Within this grid are questions, based on the four levels that need to be asked at each of the project stages for each of the project components. I constructed a flowchart of all the processes that could be monitored for all the stages of the project cycle including suggested tools to use at each stage. Here is an excerpt from the process flow diagram. This section (Figure 1) looks at 2-3 months after the training course has concluded:

Figure 1: Ensuring Implementation: Monitoring 2-3 Months After Training (ToTs = Trainers of Teachers)

This framework gives the project manager a range of options to choose from at every stage of the project cycle. You cannot implement the whole framework. It would be logistically complex, cost way too much and create unmanageable amounts of data. You can choose from the options what is most appropriate for your project, what the key questions are that you need to answer and how you can best evidence them.

Planning Considerations

Frequently, monitoring and evaluation are not budgeted for in project plans: an oversight which can prevent any real, meaningful impact being measured. Poorly thought through monitoring plans can be overly bureaucratic, generate too much data to be efficiently processed, or generate the wrong kind of data to make any useful decisions about the project. Unsystematic data collection, or cherry-picking of the best data that only shows where the project worked, can lead to biased representations of the project and inaccurate reporting to stakeholders of teacher development needs. Plans featuring little or no monitoring prevent learning from taking place throughout the project cycle which decreases the likelihood of current and future project success.

Very few projects will be able to monitor all of the steps in the process outlined in the framework and managers need to be selective. The main factors to consider are:

• Budget constraints
• Political issues
• Time constraints
• Educational Culture
• Personnel availability
• Other context factors: Availability of information. Ability of the individuals within the system to provide the information. Willingness of the individuals within the system to cooperate.

Conclusion

Without monitoring programmes, we have little evidence that our training programmes are having the desired impact. End of course questionnaires are not evidence of change in the classroom. Only longer term monitoring focussed on deeper levels of change is likely to provide the evidence we need to prove to our stakeholders that our training programmes work, given preparation, time, support and manageable processes.

Including ministries of education key personnel in the analysis, design, monitoring and evaluation of the training program based on this framework can help us to increase buy-in. This can open opportunities for looking at improving performance management systems and identifying opportunities for integrating training and learned behaviours into the workplace more systematically. Please see the British Council BLISS project for an implementation of this framework.

References

Fullan, M. 1989. Implementing educational change: What we know. World Bank. (Retrieved 16 June 2010 from: http://www-wds.worldbank.org/external/default/WDSContentServer/WDSP/IB/1989/07/01/000009265_3960929042553/Rendered/PDF/multi_page.pdf
Kirkpatrick, D. 1998. Evaluating training programmes: The four levels. San Francisco: Berrett-Koehler Publishers Inc.
Markee, N. 1997. Managing Curricular Innovation. Cambridge: Cambridge University Press.

Other references used:

Fullan, M. 2007. The New Meaning of Educational Change. Teachers College Press/ Abingdon: Routledge
Alderson, J & Beretta, A. 1992. Evaluating Second Language Education. Cambridge: Cambridge University Press
Silver, H. 2004. Evaluation research in education. Faculty of Education, University of Plymouth. Available online at: http://www.edu.plymouth.ac.uk/resined/evaluation/index.htm
Tribble, C. 2000. Designing evaluation into educational change processes. ELT Journal 54(4): 319-327; doi:10.1093/elt/54.4.319
Contact: alanmac@nile-elt.com

Measuring the Impact of Training: Kirkpatrick levels

Learning Measurement Levels 1-4 (Kirkpatrick)

Knowing there is a definitive need to measure the impacts of a large corporate cost like learning, it is fitting to have an industry acceptable model for doing so. This model is actually one that has been in existence since the 1950’s but continues to be accepted today using technology and creativity to maximize its benefits for the modern corporation.

In 1959, Donald L. Kirkpatrick, author, PhD, consultant, past president of the ASTD and KnowledgeAdvisors Advisory Board Member published a series of four articles called “Techniques for Evaluating Training Programs.” The articles described the four levels of evaluation that he had formulated based on his work for his PhD dissertation at the University of Wisconsin, Madison. Later, Kirkpatrick wrote a book (Donald L. Kirkpatrick, Evaluating Training Programs: The Four Levels, 2nd Edition, Berrett-Koehler Publishers, Inc, San Francisco, 1998) and it is now in its second edition. This book was a source for the information on the following pages related to Levels One through Four.

Kirkpatrick’s goal was to clarify what evaluation meant. The model clearly defined evaluation as meaning “measuring changes in behaviour that occur as a result of training programs.” The model itself is composed of four Levels of training evaluation. A fifth level, ROI has been added since then. The fifth level was the brainchild of Dr. Jack J. Phillips, Ph.D., author, consultant and KnowledgeAdvisors advisory board member and strategic partner. The illustration below and subsequent commentary summarize Kirkpatrick’s Four Levels and Phillips’ Fifth Level.

Level One – Reaction

Per Kirkpatrick, “evaluating reaction is the same thing as measuring customer satisfaction. If training is going to be effective, it is important that students react favourably to it.”

The guidelines for Level One are as follows:

• Determine what you want to find out
• Design a form that will quantify the reactions
• Encourage written comments and suggestions
• Strive for 100% immediate response
• Get honest responses
• Develop acceptable standards
• Measure reactions against standards, and take appropriate action
• Communicate reactions as appropriate

The benefits to conducting Level One Evaluations are:

• A proxy for customer satisfaction
• Immediate and real-time feedback to an investment
• A mechanism to measure and manage learning providers, instructors, courses, locations, and learning methodologies
• A way to control costs and strategically spend your budget dollars
• If done properly, a way to gauge a perceived return on learning investment

Level Two – Learning

Level Two is a ‘test’ to determine if the learning transfer occurred. Per Kirkpatrick, “It is important to measure learning because no change in behaviour can be expected unless one or more of these learning objectives have been accomplished. Measuring learning means determining one or more of the following.”

• What knowledge was learned?
• What skills were developed or improved?
• What attitudes were changed?

The Guidelines for Level Two are as follows:

• Use a control group, if practical
• Evaluate knowledge, skills, and or attitudes both before and after the program
• Use a ‘test’ to measure knowledge and attitudes
• Strive for 100% response
• Use the results to take corrective actions

The benefits to conducting Level Two Evaluations are:

• Learner must demonstrate the learning transfer
• Provides training managers with more conclusive evidence of training effectiveness

Level Three – Behaviour

Level Three evaluates the job impact of training. “What happens when trainees leave the classroom and return to their jobs? How much transfer of knowledge, skill, and attitudes occurs?” Kirkpatrick questions, “In other words, what change in job behaviour occurred because people attended a training program?”

The Guidelines for Level Three are as follows:

• Use a control group, if practical
• Allow time for behaviour change to take place
• Evaluate both before and after the program if practical
• Survey or interview trainees, supervisors, subordinates and others who observe their behaviour
• Strive for 100% response
• Repeat the evaluation at appropriate times

The benefits to conducting Level Three evaluations are as follows:

• An indication of the ‘time to job impact’
• An indication of the types of job impacts occurring (cost, quality, time, productivity)

Level Four – Results

Per Kirkpatrick, Level Four is “the most important step and perhaps the most difficult of all.” Level Four attempts to look at the business results that accrued because of the training.

The Guidelines for Level Four are as follows:

• Use a control group if practical
• Allow time for results to be achieved
• Measure both before and after the program, if practical
• Repeat the measurement at appropriate time
• Consider costs versus benefits
• Be satisfied with evidence if proof not possible

The advantages to a Level Four evaluation are as follows:

• Determine bottom line impact of training
• Tie business objectives and goals to training

Learning Measurement Level 5 (Phillips)

Level Five is not a Kirkpatrick step. Kirkpatrick alluded to ROI when he created level Four linking training results to business results. However, over time the need to measure the dollar value impact of training became so important to corporations that a fifth level was added by Dr. Phillips. Dr. Phillips outlines his approach to Level Five in his book Return on Investment in Training and Performance Improvement Programs, Butterworth Heinemann Publishers, Inc, Woburn, MA 1997. Dr. Phillips has written extensively on the subject, publishing or editing dozens of books on the topic of ROI.

The Guidelines for Level Five are as follows:

• Use a control group, if practical
• Allow time for results to be achieved
• Determine the direct costs of the training
• Measure a productivity or performance before the training
• Measure productivity or performance after the training
• Measure the productivity or performance increase
• Translate the increase into a dollar value benefit
• Subtract the dollar value benefit from the cost of training
• Calculate the ROI

ROI calculations are being done by a few world-class training organizations. They help these organizations:

• Quantify the performance improvements
• Quantify the dollar value benefits
• Compute investment returns
• Make informed decisions based on quantified benefits, returns, and percent return comparisons between learning programs

Dr. Phillips has created an ROI Methodology that he conducts certifications and workshops on and has helped training organizations use the right tools to measure the ROI on organizational learning.

Click here to view an illustrated summary of his methodology.

The methodology is a comprehensive approach to training measurement. It begins with planning the project (referred to by Dr. Phillips as an Impact Study). It moves into the tools and techniques to collect data, analyze the data and finally report the data. The end result is not only a Level 5 ROI but also measurements on the Kirkpatrick 4 Levels as well. This yields a balanced scorecard approach to the measurement exercise.

The above has been edited from:

© 2004 Global Learning Alliance and Knowledge Advisors

http://www.trainingreference.co.uk/training_roi/roi2.htm

Technology In Education

Well, Here is a completely copied presentation. My Kazakh audience today were asking me, how do I deal with plagiarism. I told them, no ideas in this world are new and they have to be referenced. I hope I have properly done that here. Technology in education. If you see something you don’t like, let me know.

More secrets and lies

The last post was about the evidence-base for teaching. Stephen Pinker applies the same principle to grammar rules and elegantly debunks rules that we were all most likely told about in school but flout on a regular basis. Stephen Fry has been known, in his more pompous Latinesque moments, to berate his quiz show contestants on their ‘incorrect’ grammar, when in fact, there is nothing wrong with what they are saying:

• We need to lovingly embrace split infinitives.
• Also, you can start sentences with conjunctions.
• Who/ Whom? Who cares?
• like? Such as? As? It’s a formality difference.
• Dangle yer modifiers as ye may…
• Prepositions at the end of sentences are in!
• It is I. It is me? Get a life!
• That and which are interchangeable.
• Modifying absolute adjectives is completely fine.
• Although both are available, fewer people use fewer than less, and ’10 items or less’ is not a mistake.

 

Myths and Realities

Four major ideas in education have been debunked without my noticing. Since they are so widespread in their infiltration of presentations, papers and webpages, I thought I better do my bit to spread the word. All are intuitively compelling. None have any research base to back them up. As a matter of balance, I think i’ll work on an entry on the importance of intuition in teaching 😉

1. The Learning Cone is fake.

This research never took place. The numbers are all made up. The diagram is based on Dale’s Cone of Experience:

Dale’s  ‘Audio visual methods in teaching’ (1957) states that it is dangerous to see the bands as inflexible divisions. The cone was not to be taken absolutely literally. It was designed as a visual aid to help explain the interrelationships of the various types of audio-visual materials, as well as their individual ‘positions’ in the learning process.

He said “The cone device is a visual metaphor of learning experiences, in which the various types of audio-visual materials are arranged in the order of increasing abstractness as one proceeds from direct experiences.”

It was a theoretical construct that derived from research experience, but Dale never attached numbers to his cone. Any representation of the cone with numbers attached is a fiction.

I have been using the fake cone for many years in training sessions and presentations. A quick scout on the internet shows that many PhDs and university departments have been similarly hoodwinked…so I don’t feel so bad about it. Now I can still use it, but not in the same way as I have been. This is a great lesson in sourcing information, and making sure to find the origins of theoretical or so-called research based models. Always check the provenance!

There is a good article about it here.

2. Multiple Intelligence Theory is just that: there is no credible scientific evidence to back it up.

3. There is no evidence that learning styles assessment has any real purpose.

A Critical review of 13 models of learning styles concludes that the field is confused and pseudo-scientific. An additional research review with recommendations for teacher training practice here.

4. Neurolinguistic programming has no research base:
Thirty-Five Years of Research on Neuro-Linguistic Programming. NLP Research Data Base. State of the Art or Pseudoscientific Decoration?
The huge popularity of Neuro-Linguistic Programming (NLP) therapies and training has not been accompanied by knowledge of the empirical underpinnings of the concept. The article presents the concept of NLP in the light of empirical research in the Neuro-Linguistic Programming Research Data Base. From among 315 articles the author selected 63 studies published in journals from the Master Journal List of ISI. Out of 33 studies, 18.2% show results supporting the tenets of NLP, 54.5% – results non-supportive of the NLP tenets and 27.3% brings uncertain results. The qualitative analysis indicates the greater weight of the non-supportive studies and their greater methodological worth against the ones supporting the tenets. Results contradict the claim of an empirical basis of NLP.

Witkowski, Tomasz (2010). Polish Psychological Bulletin, 41, 2.

Evidence-Based Teaching

Ben Goldacre has been advising the MoE in the UK since 2013 on what teaching based on actual research results might look like. He said “This is not about telling teachers what to do. It is in fact quite the opposite. This is about empowering teachers to make independent, informed decisions about what works, by generating good quality evidence, and using it thoughtfully…Every child is different, of course, and every patient is different too; but we are all similar enough that research can help find out which interventions will work best overall, and which strategies should be tried first, second or third, to help everyone achieve the best outcome.”

A lot of research has been done on this already. Hattie synthesised 800 meta analysis incorporating 50,000 studies of educaitonal practice to come up with an effect size for every educational factor studied. It is critiqued in Invisible Learnings? A Commentary on John Hattie’s book: Visible Learning: A synthesis of over 800 meta-analyses relating to achievement by Snook, O’Neill, Clark, O’Neill, and Openshaw, New Zealand Journal of Educational Studies; 2009, 44, 1, p 93. If anyone has a copy of this, I would love to read it. It is summarised here.

Main critiques appear to be the way the results are being used to bolster political agenda they have no connection to, and the lack of factors studied that impact on learning but are outwith the scope of the study: socio-economic factors, nutrition for example.

Hattie found that a methodology called Direct Teaching has the greatest positive effect on learning but possibly because of the cumulative effect of a number of its components. The problem with effect size is isolating the different components enough to be able to measure their effect. On reflection, this is also the problem with the fake stats attached to the Cone above. Lecture: About what? How long? By whom? Delivered how? Hattie goes some way to specifying this:

“We know that students in lectures learn most in the first 8 minutes, only recall
three things at most after one hour, and that if the content does not shake
their prior beliefs they file away the fascinating facts in the deepest recesses
of their brain, if at all.” (1999)

But because of the complexity of the processes of teaching and learning, it is very difficult to tease out all the relevant factors. And if you do, will it tell you anything more than you already knew? Will it make better lessons more common? Will more students learn more as a result?

Well let’s just say, we will have more information to work with and a better chance of enabling more learners to meet their learning outcomes as a result. And when Mr. Goldacre’s pilots with the English education system publish their results, we will see what the future holds.

See also Geoff Petty’s (2009) Evidence-based teaching: A practical approach (Nelson Thornes)

The Game

Intro:

I recently discovered this and thought it must have been the result of a bad day at the British Council. However, it turns out that it was originally written in July 2002, between universities in Japan. It surprises me that this came out of my head, but I do like it, so I am sharing it with you. Paul Evans, after reading it, shared this fascinating link on game theory.

Games are fun. Games are good. Games have rules, and a board, a limited field of conflict, a start and an end.

This game started so long ago that the original players have been lost in time. This game’s field of conflict has expanded so much that it involves everything of a person’s life.

This game had a board, but the board has changed, and consequently so have the rules. It is a curious game because previously all the roles and rules were defined, but now that the board has changed, the rules and roles have also changed.

Last week the game was chess. This week it is, depending on your point of view, Snakes & Ladders, Monopoly, Scrabble, Patience, Blind Jigsaw, Trivial Pursuit, Spin the Bottle, Truth or Dare or Calvinball.

Snakes & Ladders

One step forward leads you down a snake or up a ladder. Who knows what will happen? The dice controls your life. Around the corner lie snakes that could eat you alive, but there are also snakes that can help you. Ladders may be useful, but they may also have broken rungs that need repairing. You roles the dice and you takes your chances.

Monopoly

How much can you get your hands on? As many players as can fit around the board. One for themselves, and all for whoever has the most money. Ostensibly the winner is the one with the most property and cash at the end of the game. The sad thing is, if the game ends, everybody loses, because there is no profit allowed: it is all Monopoly money, not real currency. Or is it?. ‘Chance’, and ‘Opportunity House’ cards are taken. The consequences cannot be seen. You may well ‘win’ a beauty contest, but that purely shows short-term profit and has nothing to do with long-term gain, or respect. What happens when you get old? And you are going to get old fast. We all do.

You can also go to jail for taking a wrong step. ‘Get-out-of-Jail-Free’ cards can help you out, and the next roll of the dice sends you right back in. Do you really want to have lots of play-money, lot’s of responsibility, and no respect because you screwed everybody else?

Scrabble

What is an acceptable word? Is it acceptable to others? Do they have the same understanding of it as you do? Oops…the rules changed. Suddenly you have to supply a word in a specific context that others relate to…and, Oh no, rule change number two involves you using real words in genuine sentences. ‘Genuine’ is defined as ‘real’, ‘honest,’ ‘true.’ Can you do this? Can you be true to other people at the same time as being true to yourself?

Patience

Involves solving a huge organizational problem by moving pieces of the problem from one place to another until the various pieces disappear, and cease to be a problem. You know how the suits should be ordered, but making the cards fit can be a lot more difficult than you think. Sometimes, the problem solves itself because of the way the cards lie. Sometimes, you have no way of winning. Sometimes, you get stuck and you need to restart the game until you find the combination of conditions you need to complete it. Sometimes you need to give up and get on with your life.

Blind Jigsaw

Not dissimilar to patience except that you do not know what the picture should look like until you are well into the game. You try your best to put all the bits together. Sometimes it works, sometimes you realize there is a piece in the wrong way round and it needs to be changed. Sometimes you find a piece that does not belong in this puzzle, it’s in a completely different picture and needs to be removed. And there is always a piece missing.

Trivial Pursuit

An endless, torturous, pathetic exercise in answering meaningless questions about meaningless things that cannot possibly matter to anybody that has a real brain. True vegetable fodder. People who win this game need more to do with their time. If you ever want to exercise the parts of your brain that have atrophied from inactivity, this is the one for you.

Spin the bottle

The ultimate semi-control game. We all know that we can try to spin it to hit the one we want to kiss, but we are never really sure if it is going to hit the target. If it does not, what do we do? Do we go, “NO WAY!”, or do we take the punishment? Who gets the jobs now? Kiss HIM? I would have great difficulty hugging him! But I suppose if you look around, I don’t have much choice. Oh well, let’s suck it and see!

Truth or Dare?

Do you dare or do you come clean? Do you fudge or are you up front? Will they believe you or will they continue with their previous conceptions? Your choice, but when it comes down to it, they WILL find out when you are bluffing, and if you are, they WILL crucify you: not a happy ending for a game.

 

Endgame?

No.

New game?

Yes.

Calvinball is the easiest metaphor: whoever has the ball, makes the rules. You need to make sure that you are making the rules and that everyone else is following them. In order to do this, you need to hold the ball.

You may stumble, but you will keep running. There is no field other then what you define. There is a board, but it is conceived of broadly and there are snakes, which you need to avoid, ladders that you need to take, extra throws, and a hell of a lot of chance cards. There are also obstacles, jails, dangers and tribulations. So what do you do?

Sometimes the ball is in your hands and sometimes it is not. You can stop playing at any time and this is a healthy move. However, it is always nice to pass the ball onto someone who will win the game. Oh, wait a minute, no-one CAN win the game because it just keeps going on and on and on, and it will do, without you, or your poxy proxy.

 

This, is the game of li(-es)-fe.

Small Steps in English Language Teacher Development in Surin: Step One

Based on a report submitted to Surin Education Service Area 3 and published in this form originally by the Volunteer Educational Network (VEN)

Introduction

This paper first describes the attempts of one non-profit organisation (NPO) to help teachers in rural north-east Thailand to cope with the radical national curriculum guidelines being implemented in 2003 nationwide by the Thai Education Ministry. It then describes the results of observations of six teachers who attended those workshops leading to a discussion of implications for future teacher development efforts. Finally a low-cost, local-teacher centered development framework is outlined.

 

Background

The Thai education Ministry is radically changing the way in which English education is structured and taught in Thailand (Mackenzie, 2003; ONEC, 2002). The main changes outlined in the National Education Act (NEA) of 1999 include introduction of:

• early education; addition of four more years of English education (starting from first grade rather than fifth).
• naturalistic approach to language learning; listening and specking focus for fist two years, before introducing reading and writing.
• dictation of methodologies to be used by teachers; TPR in first two grades, Theme- and Task-based approaches in fourth to sixth grades, content-based in secondary.
• emphasis on the four C’s: communication, culture, connections and community.
• movement towards wholesale adoption of communicative and autonomy fostering approaches to language teaching.
• desire to give schools more freedom in constructing heir own curricula.

 Although great strides have been made in the provision of general education to a larger proportion of the population with increased numbers of teachers being hired and student/teacher ratios significantly below Organisation for Economic Cooperation and Development (OECD) and World Education Indicators (WEI) averages (AWEI, 2000) as Smith (2003) notes, modification of the national curriculum is too slow to respond to these changes. While Smith advocates that the modern educational framework should stress the importance of continuing self-learning and developing the problem solving capacity of children, I suggest that first it needs to focus on the development of teachers, since it is only through teacher development that the education system will be able to meet the lofty goals it aspires to. Unfortunately, although the need for teacher development programs has been identified (albeit a little late, see TERP, 2002; Fry, 2002; ONEC 2002), as yet relatively few resources have been assigned to this field. When they have the emphasis has been on secondary and tertiary education (TERP, 2002) rather than the primary sector where the changes are most radical. Indeed because of the rapid increase in the number of teachers required to meet the needs of the expanded primary education curriculum, many primary teachers have received no training as teachers (TERP, 2002, p. 6) or are trained in subjects other than the subject they are teaching.

Further, despite the necessary conditions for sustainable development clearly outlined by the ASEAN summit 1998 for environmental projects (ASEAN, 2002b, p. 2) including, “…a series of medium term action plans which will set strategies and specific activities with measurable targets and outputs including means of implementation and mid-term review mechanisms,” very little has been planned in terms of teacher development. Ironically, the ASEAN framework has been applied to student development and results in terms of student assessment outcomes (ONEC, 2002). This may be partially rectified by the ASEAN Committee on Education (ASCOE) (founded September 2002), although its stated mission is to promote ASEAN awareness in primary and secondary schools (ASEAN 2002a) which may lead to increased rather than decreased burdens on teachers who are already over-stretched.

The Thai education reforms have deep and long-lasting implications for teacher:

• Education-Ministry-controlled textbooks have been removed from the curriculum in order to force teachers to become their own materials developers.
• The previous, concrete, grammar-based syllabus has been replaced by competency targets which most teacher do not understand. The aim is to force teachers within individual schools to become their own curriculum developers.
• The education ministry desires to make schools more autonomous and distribute responsibility for educational decisions throughout the education system.

Clearly without attention to teacher development, the goals of the Thai education reforms are unlikely to be achieved.

 Participants in CANHELP-Thailand who were participating in school construction projects in rural Thailand noticed the situation in Thai English language education. In conversation with Thai teachers who expressed a need for basic training in communicative methods and ongoing opportunities for professional development, they decided to set up an organisation that dealt specifically with these issues. Now called the Volunteer Educational Network (VEN), they ran their first series of workshops in Sakeow  province in 2001. The second series ran from July 28^th to August 1^st when VEN conducted five days of workshops entitled “Recipes for English Teachers” in the Thong Tharin Hotel in Surin city, Surin Province, Northeast Thailand for 150 teachers from Educational Service Areas (ESA) 1, 2 and 3 organised by the chief administrator for English Language Education in the Area, Mr. C. Chachai. Ten volunteer instructors (of whom I was one) lead the workshops. Teachers were split into five groups based on the grade level they taught. Each group was lead by a two-volunteer-team and were subdivided into two smaller groups of around 15 people lead by one volunteer each. Different volunteer teams operated independently, sometimes leading their individual groups separately and sometimes combining the groups.

The stated aims of the workshop were to:

• introduce teachers to communicative language learning (CLL) activities
• give teachers the opportunity to use these ideas with real students in real classroom situations
• introduce teachers to classroom observation and give them the opportunity to observe CLL activities used in real classroom situations
• o  encourage teachers who already use CLL activities to share them with others and develop their ideas in a public forum.

The basic timetable is shown in Table 1.

 

Table 1:          Outline of the VEN Workshop for Surin, July 2003

Date	Topics
Monday 28th	Orientation & Introduction to CLL principles and sample activities by volunteers. Micro-teaching of teacher-generated activities.
Tuesday 29th	Continued Micro-teaching of teacher-generated CLL activities. More CLL activities from volunteers. Lesson planning in preparation for demonstration lessons and observation preparation.
Wednesday 30th	Demonstration lessons and observations I Observation feedback sessions. Re-planning the demonstration lesson.
Thursday 31st	Demonstration lessons and observations II Observation feedback sessions. Orientation to the Mini-conference.
Friday 1st	Mini-conference: First sessions lead by volunteers Second sessions lead by selected teachers and guests from Sakeow. Closing Ceremony and feedback questionnaires.

Two weeks later, from Monday August 18^th, to Friday August 22^nd, I visited six schools in ESA 1 and 2 to observe classes, interview one teacher from each school who had attended the VEN workshop and discover what changes, if any, teachers had made to their teaching as a result. This report represents the conclusions and ideas for future development suggested by the data gathered during those visits. While this report highlights problems that I think are prevalent among Surin English teachers, this does not mean that all teachers have these problems, but merely that since one or two teachers in this small sub-sample displayed the following behaviours, there should be cause for concern that this is symptomatic of more general trends within the Surin teacher population. As such, highlighted problems should be viewed as the basis for future development and should enable VEN to plan future workshops in different areas with an understanding of the consequences of the current focus.

 

Principal Observations

All teachers interviewed noted that their teaching had changed in a positive direction as a result of the workshops. This was also supported by my observation of classes they taught. However, many noted that there were a number of underlying problems impeding their and other teachers’ development and therefore the development of English Language teaching and learning in the area. These are considered below.

Teaching Changes

Of the six teachers I observed, I believe two to stand out as being most aware of student-centered CLL principals and practice.

The first has enabled her students to:

• suggest alternative ways of learning and using the target language
• construct activities to use with their classmate for language learning (one group made a card game to practise sentence construction)
• suggest their own topics for projects (local attractions for tourists)
• use their own content for language tasks (for example each student explains how they made a flower from recycled materials)

The second asks students to:

• tell her what vocabulary they want to know in English depending on the topic (helping students to personalise their vocabulary choices and increasing the likelihood of remembering the words learned)
• making their own versions of songs taught and writing their own songs on their own topics of choice
• volunteer without fear of repercussions or negative feedback

Looking across the other teachers as a whole, the following are signs of change toward more communicative, student-centred methods:

• using more pair and group-work
• using more games, songs, and interactive activities in class.
• using more English in class when giving instructions and commenting on student efforts.
• planning lessons more.

 

Teacher Development

Teacher development is a long slow process. While the above developments can be seen as positive outcomes of the workshops and a development in communicative language teaching, it is important to see this as the first step on the way towards future development which needs to be guided. Teachers, having started to change how they teach, need to be helped to develop, just as they are helping their students to develop. The following describes specific action points that need to be dealt with from now on so that teachers do not become ‘activity-centred’ rather than student-centred. It is also important to emphasise to teachers that making connections within and across lessons will help students to learn more language and that since the Education Ministry has devolved curriculum development power to teachers, they must see the big picture of the whole curriculum they are teaching as well as thinking about the minute-to-minute activities that they use in class.

Activity-Centered vs. Student-Centered

There is evidence that teachers are generally more activity-centred than student-centred at present. For example, although they have created lesson plans based on linguistic objectives, some teachers use activities in an arbitrary order, rather than moving in a logical order from more to less-guided and structured activities, from presentation to production activities or from teacher-fronted to student-generated activities. This may be due to a weakness in lesson planning technique. During the workshop, many teachers noted that this was the first time they had had to make a lesson plan. Practice in lesson plan construction and delivery then needs to remain a priority.

Teachers also tend to focus more on having the students do the activity rather than making the activity work in the way intended, having the students produce language in order to complete the activity, or choosing to use appropriate activities to elicit particular language.

For example, in order to have students practise the structure, “Are these Xs yours?/ Is this X yours?” one teacher had students write a word and their name on a card. She then mixed up the cards and re-distributed them randomly and asked students to go around the class asking the target questions while substituting the things on the card. Students were asked to cover the name, but by this time many had already read it. Students ran around the room, some asking the classmate who wrote the word one question in English (if any), others covering the name, showing the card to a classmate and looking at them in a questioning manner while making a non-verbal questioning sound. When asked whether she thought the activity was a success, the teacher answered affirmatively.

While the teachers’ observational abilities are obviously in question, the main reason for the failure in the activity was that the teacher did not think through exactly what the students needed to ask/say, and therefore what information needed to be on the card and what did not. She was focussing only on the completion of the activity in order to move onto the next point in the lesson.

Attention also needs to be paid to appropriateness of activity to the language being taught. In one lesson observed, students were performing skits. Two groups of students presented highly entertaining imaginative skits with well developed roles based on restaurant language but the third presentation on directions was straightforward and uninteresting. One reason for this is that while giving directions is a function, being in a restaurant is a situation. Situations give students a lot more scope for imaginative language use than functions which are linguistically more limited. Indeed situational language uses a variety of functions (see Figure 1) while functions might use a variety of sentence-level grammatical constructions and these in turn can have different vocabulary substituted into them.

 

 

Figure 1:          The different levels of language use

 

This suggests that role-plays are best for situational language while functions may be best taught using games or information gap activities. Sentence level grammar might be best taught in repetitive tasks like learning a song or information gap activity while vocabulary can use flashcards, pictures, or individual research.

Connections

Other teachers appear not to have understood lesson planning yet. Their lessons are a mixture of songs, games and choral repetition of vocabulary, using structures with no apparent connection to each other. One teacher observed used the stand-up, sit-down song as a warm up, had students repeat greetings and responses (“Hello”, “How are you”, “I’m fine thank you”) and ended with fruit names as a vocabulary building exercise. While the teacher stated her underlying logic as the first activity was a warm up reinforcing instructions language, the second was a review of useful language that students need to know and the third was the actual object of the lesson, there should be at least some kind of connection within the lesson and the language used. Connections can and should be made between:

• familiar language and newly introduced language
• language used in previous lessons and the current language objective and then current objectives and future language lessons
• language taught and students’ personal experience
• content used for teaching and language used to describe it.

Attention to Language Development Through the Curriculum

Connections also need to be made between the individual lessons being taught and the larger English language curriculum both for that grade and for other grades. Currently, very similar language is being taught at all levels of the education system. This language is generally of a very basic type: likes/dislikes, haves, colors, fruit, animals. Teachers, when asked, claim that students do not know the language so they need to continuously review the same language. Theoretically, language ability should develop through the language curriculum by revisiting basics but also moving forward as in Figure 2.

 

Figure 2:       The theoretical spiral of language development across the curriculum (Adapted from Bruner 1960.)

Currently however, there is very little development of language ability among students. This may be due to a number of factors such as:

• teachers not pushing students enough to try to develop their ability
• teachers playing safe and teaching things that the students are familiar with and have been taught before.
• teachers not motivating students to learn
• teachers not constructing curricula coherently enough to engender language development
• teachers testing too much and teaching too little
• teachers using too much Thai in class
• teachers not being able to use English to a standard high enough to be able to teach it.

 

Translation and Repetition

When observing classes I also noted that many teachers were:

• Automatically translating instructions given in English or vocabulary introduced in English into Thai. For example, one teacher said, “stand up” in English and then repeated the instruction in Thai. When teaching things in the classroom another teacher pointed to the thing, said the word in English, had students repeat it, then said the word in Thai and had students repeat that.
• Automatically having students repeat everything after the teacher. One teacher was teaching numbers, colors and nouns: “three orange crabs”, after repeating three times, she moved onto the next card “one black crab” and repeated that three times. No opportunity was given for students to generate the language without the teacher prompt.

Reasons given for these behaviors were that this was necessary because:

1. it helped students understand
2. students need to use the language
3. it was the way teachers were told to teach language in teacher-training college

While repetition is necessary for remembering language, enabling students to repeat the target language in different, real communication situations should be the teachers aim, not simple choral repetition. In response to the three points above:

1. Repetition is not an aid to understanding. It may be an aid to memory in lower-level learners. However, repetition with constant translation is unlikely to lead to memorisation as there is no real need to learn the foreign language when the first language is constantly used. At some point (as early as possible) the repetition needs to stop and independent production by the students should take over from the teacher-lead chorus. Teachers also need to test whether students really understand what they are repeating by having them use the language in a different order from which it was repeated. With lower level learners, choral repetition with pictures and words should move away from pictures towards words only which should them be randomly presented in order to familiarise students with the written form in order to transition to reading.
2. Repetition is not using language. Using language is students choosing appropriate linguistic forms (sentences and vocabulary) in order to complete a communication task. Repetition is not communication.
3. Teachers have undoubtedly be thoroughly trained in choral repetition since it is remarkably consistent across all teachers observed and is symptomatic of “traditional” audio-lingual training rather than “modern” communicative methods.

Over-focus on vocabulary

Another major consistency across observed lessons was the focus on vocabulary. Teachers are overly concerned with students learning a large number of words rather than a large number of sentence patterns or other linguistic features. In only one class that I observed was there any teaching of situational language or functional language. Class time is generally used in choral repetition of vocabulary or sub-sentence-level phrases, work which may be better suited to private study rather than class-work. In focussing on the vocabulary, teachers are neglecting other parts of language that are as, if not more important (see Figure 1).

The Implemented Curriculum

Some teachers believe they are implementing student-centered CLL methods in class while they are actually using different forms of teacher-centered methods. This is either a problem of perception, or a misunderstanding of what student-centered CLL involves. This is possibly the most problematic issue observed because:

• Teachers develop a false sense of confidence
• Teachers who are confident that they are using CLL may be advising others incorrectly
• Once new methods are used, they become ingrained and difficult to change
• The only way this can be observed is in the classroom while the teacher is teaching

Figure 3 shows the different levels of curriculum and how they are assessed. Traditionally, it is the job of the supervisors in each area to collect data that schools generate in the form of reports and test results. So far, this role has not included the collection of lesson and curriculum plans, or the regular observation of teachers in classrooms teaching. Also, data collection is most often for evaluative purposes than for developmental purposes. In other words the information flow is one-way from the schools to the supervisors unless the school asks for help.

Figure 3:          Different levels of curriculum implementation and how they are assessed

 

Suggestionsfor Future Development

English language teaching in Surin Province is at a critical stage in its development. Current policies of sporadic workshops for a limited number of teachers are unlikely to meet the challenges laid down by the Education Ministry in its reformed curriculum. In order to meet the challenges head on, it is necessary to form a multi-faceted approach (see Figure 4) which involves running regular workshops for re-training teachers in CLL methods, developing a regular communication system for teachers to share ideas and help each other to develop, developing a feedback system so that teachers can come to know that what they are doing in class is actually what they planned and helps their practice of CLL methods to develop.

Figure 4:          A multi-faceted approach to English language teacher development in Surin Province

Underlying Problems

In participating in teacher development activities in Surin, it is important not to ignore the larger contextual factors (of which there are many) that impede progress. These include but are not limited to:

• Large classes
• Generally one English teacher per school
• A heavy teaching workload
• A heavy administrative workload
• Other school duties (making/distributing school meals, extra-curricular activities)
• National hiring policies that do not take into account the need for a specific type of teacher in a specific school
• Low level of English of the English teachers, many of whom are not English teachers by training
• Largely audio-lingual teacher training
• Many teachers have been teaching the same way for a long time and therefore find it difficult to change
• Sudden pressures on individual teachers to become curriculum writers, materials developers and testers in a very short time with minimal training.

Most of these factors are systemic and therefore require larger organisational changes on the part of the Education Ministry. Few can be dealt with on a local level. However, we should expect that if teachers are taking part in teacher development activities in the English language, their ability will increase.

Teacher Training

Current teacher training methods need to be investigated in order to ensure that CLL is introduced at as early a stage as possible in the trainee teachers experience. I had the opportunity to observe a trainee teacher teaching an English class. The teacher ‘supervising’ her was not present for most of the lesson and feedback on the lesson was not given as a matter of course. There was very little evidence that the teacher had been trained in CLL methods. She used a CLL textbook but managed to remove all the communicative activities outlined in the text and had students copy dialogs for forty-five minutes out of a sixty-minute lesson. The final five minutes of the lesson involved teacher-led practise of the dialogs.

In talking to the supervising teacher, I suggested that she give the teacher feedback and suggestions for making the lesson more communicative. She resisted this suggestion and asked me to do it. I insisted that this should be her responsibility since she was going to be supervising this teacher throughout her training.

It is imperative that teachers coming into the system are well trained in CLL if the quality of English education in the region is to improve. To this end, I suggest close collaboration with the ESA supervisors and the local teacher training colleges in order to create a set of guidelines for both supervising teachers and trainees that emphasise the importance of observation and feedback. Teacher training colleges may also be able to introduce a series of observations where trainee teachers observe more experienced teachers using CLL methods in class.

Ongoing Observation & Feedback

In order to fully understand what is happening in classes, teachers and supervisors need to embark on a program of observation and feedback that focuses on how the lessons taught in Surin schools could become more communicative. It is important that these observations be development- rather than evaluation-focussed. In fact, I would suggest that they should be non-judgmental. Check-box lists are less useful than statements of what was observed and questions about what was happening in class and the results of classroom activity. Focus on what students are doing and language they are producing (or not!) is imperative. These observations should be regular, and should become an integral part of teacher development in the region.

Teacher Development Networks (TDN’s)

Since most schools only have one English teacher, there is little chance for teachers to communicate with each other about teaching. However, there are many schools that are within three to five kilometres of each other suggesting that it would be relatively easy for teachers from different schools to come together in casual informal settings over a meal or drinks after school, during holidays or at weekends to meet and discuss teaching issues. Although this superficially seems like an increased burden on teacher leisure time, these TDN’s could in fact decrease the workload of teachers by sharing the development burden. Suggested functions for these groups are:

• Sharing lesson plans and teaching ideas.
• Collaboration on curriculum plans for a number of schools in the area.
• Giving feedback on the classroom implementation of those plans.
• Creating peer-observation schedules.
• Discussing and formulating evaluation strategies.
• Discussing wider-school issues

Essentially then these networks would be peer-support groups for English teachers in the area which would decrease the overall workload and could have a reporting function to the ESA supervisors.

 

Surin Teacher-Lead Workshops

This week, several teachers who took part in the VEN workshop will hold another seminar for teachers who could not attend. This is a very positive step since it acknowledges that there are a large number of very talented teachers within the area who understand CLL methods and are interested in continuing to help English language education in the area develop. It is very important that these workshops continue in tandem with other development methods. Only having sporadic workshops is not going to make a significant impact. Having a regular workshop program run by the teachers themselves along with TDN’s and an observation program will likely lead to larger gains.

Provincial/ESA Newsletter

Finally, the idea of a newsletter for the province or newsletters for each ESA was suggested in the VEN workshop. The presentation of this idea attracted seventeen people who actively took part in discussion of the issues involved in making this a reality. It is imperative that this enthusiasm be tapped and an editorial team be formed in order to help make at least one newsletter for the province a reality. It is also important that this be published regularly and that adequate funding be available for one copy to be distributed to all teachers in the province.

 

Conclusion

It is important that teachers feel that they are embedded in a learning community that is interested in their problems, contributes to their development, has the best interests of students at heart and cares about them as teachers. It is also important that Surin Province English education supervisors be as supportive as possible in implementing any or all of the above suggestions. This may require a change in the role of the supervisors and a culture change within the ESA’s. Exactly what these changes are and how they can be managed is beyond the scope of this report, but I hope that continued communication within Surin Province, and between Surin, VEN and myself will enable these systemic changes, which in turn will enable the development of the above framework for sustainable teacher development and help the field of English language teaching in the area to become more communicative, more supportive and ultimately more successful. As Chapman and Adams (2002) note, even one inspired, creative teacher can make a difference. In Surin, the aim is to inspire a body of teachers and enable them to cope with the lofty goals of Thai education policy in a way that balances their personal development, the ideals of autonomy theory, the needs of the system and the availability of local resources.

References

ONEC (Office of the National Education Commission, Office of the Prime Minister in Cooperation with the Ministry of Education and the Ministry of University Affairs Kingdom of Thailand). (2002). Education in Thailand 2002/2003. Available online at

 http://www.onec.go.th/publication/education03/Sar_education.htm

Mackenzie, A. S. (2003). Curriculum development in Thailand. In A. S. Mackenzie & Tim Newfields, Curriculum innovation, testing and evaluation (pp. 59-67). Tokyo: The College and University Educator (CUE) Special Interest Group (SIG) of the Japan Association for Language Teaching (JALT). 

ASEAN (Association of Southeast Asian Nations), (2002b). ASEAN report to the World Summit on Sustainable Development. Jakarta: ASEAN Secretariat.Available online at: http://www.aseansec.org/pdf/WSSD.pdf

ASEAN, (2002a). ASEAN annual report 2002-2003. Jakarta: ASEAN Secretariat Available online at: http://www.aseansec.org/ar03.htm

AWEI (Analysis of the World Education Indicators), (2001). Teachers for tommorrow’s school. Paris: UNESCO-UIS/OECD

Fry, G. (2002). Synthesis report: From crisis to opportunity, the challenges of educational reform in Thailand. Bankok: ONEC & ADB (Asian Development Bank). Available online at: http://www.worldedreform.com/pub/fulltext1.pdf

TERP (the Thailand Educational Reform Project), (2002). Teacher development for quality learning. Brisbane: Office of Commercial Services, Queensland University of Technology. Available online at: http://www.onec.go.th/publication/4512901/fulltext.pdf

Chapman, D. & Adams, D. (2002) The quality of education: Dimensions and Strategies, education in developing Asia Vol. 5. Hong Kong: University of Hong Kong Comparative Education Centre for the Asian Development Bank.

Smith, K. (2003). Education Reform Framework for Sustainable and Continuing Self-Support.Available online at: http://www.worldedreform.com/intercon2/ab3_1.htm

Fredrickson, T. (2002). Taking the BIG VIEW. Bangkok Post, December 17^th,  Available online at: http://www.bangkokpost.com/education/site2002/cvdc1702.htm

Bruner, J. (1960). The process of education. Cambridge, M

Publications Online

Here is a list of online publications I have written or had a hand in. There are a lot more, but they are only available on paper. Felt the need for a summary list for my own reference, so here it is:

British Council (2013). Can English medium education work in Pakistan? Lessons from Punjab. British Council: Lahore.

Mackenzie, A. S. and Padwad, A. (2012, March). Working across Cultures – Issues in Managing a Teacher’s Association. DevelopingTeachers.com

Mackenzie, A. (2008). English Next in East Asia. In Primary Innovations: A collection of papers. British Council: Bangkok.

Mackenzie, A. (2008). CLILing me softly in Thailand: Collaboration creativity and conflict. Onestopenglish.com.

Shipton, I., Mackenzie, A. & Shipton, J. (2006). The child as a learner. British Council: Teachingenglish.

Mackenzie, A. (2005). Current developments in EFL curriculum reform in Thailand. Paper delivered at Language and Development Conference, Addis Ababa.

Mackenzie, A. (2003). Keisen University’s Center for English Education and Research. In A. Mackenzie and T. Newfields, Curriculum innovation, evaluation and testing. Tokyo: JALT.

Mackenzie, A. (2003). EFL curriculum reform in Thailand. In A. Mackenzie and T. Newfields, Curriculum innovation, evaluation and testing. Tokyo: JALT.

Mackenzie, A. (2003). HELP!. The Language Teacher, 27, 2, 17.

Mackenzie, A. (1999). Taking a walk on the WILD-er side of teacher development, The Language Teacher, 23, 11, 31-33, 39.

Mackenzie, A. (1999). Interactive student-generated vocabulary quiz, The Language Teacher. 23, 7, 25-26.

Mackenzie, A. (1999). A product development simulation for business classes, The Language Teacher, 23, 3, 113-14.

Mackenzie, A. (1997). Using CNN in the EFL classroom, The Internet TESL Journal, February.

Wild-e Lives!

Thanks to the wonders of internet archiving, WILD-e is still available to view. WILD-e was a teacher development website that my friend and mentor Nanci Graves and I compiled in the late 90’s and then thought was lost forever when the hosting account and web address accounts ran out. However, here she is! Looking a bit clunky (believe me it was cutting-edge at the time!) in all its hand-coded glory.

Nanci sadly died last year so this post is a tribute to here. When I have the time, I would like to resurrect WILD-e in some form. If you want to help me: get in touch.