Read Microsoft Word - A Modest but Meaningful Contribution of Educational Effectiveness and Improvement-1.doc text version

A Modest but Meaningful Contribution of Educational Effectiveness and Improvement

Bert P.M. Creemers

University of Groningen Faculty of Behavioural and Social Sciences

1. High expectations combining Educational Effectiveness Research and Improvement

Educational effectiveness research (EER) can be seen as a conglomerate of research in different areas: research on teacher behaviour, curriculum, grouping procedures, school organisation, and educational policy. The main research question of EER is which factors in teaching, curriculum, learning environment et cetera, in education at different levels, such as the classroom, the school and the above school levels, can directly or indirectly explain the differences in outcomes of students, taking into account background characteristics such as ability, SES, and prior attainment. EER does not attempt to invent new ideas or programmes but to concentrate on understanding the lessons to be drawn from existing practices. In this way, EER attempts to establish and test theories which explain why and how some schools and teachers are more effective than others. This research question has been pursued for many years, sometimes with an optimistic view about the contribution of education to student outcomes and sometimes with a common feeling that education cannot contribute much to student outcomes, at least when adequate control for student background characteristics like ability, motivation and social-cultural background is applied. There are many different stories to tell about the start and the development of educational effectiveness research and improvement efforts and they might differ from one country to another. One way to tell the story about EER is to start with the publication by Brookover et al. (1979) in the USA and Rutter et al. (1979) in the United Kingdom. These two effectiveness studies, undertaken independently, were concerned with examining evidence and making an argument about the potential power of schooling to make a difference to students' life chances. Both studies showed that schools differ from each other in what they achieve with comparable groups of students. This was an optimistic point of view because many studies published in that period showed that teachers, schools, and maybe even education in general, did not make much


difference at all, after taking into consideration student background characteristics, such as ability and family background. In the 1970's, studies by Coleman et al. (1966) and Jencks et al. (1972) were taken as sufficient proof for the proposition that `Schools do not make a difference'. This pessimistic feeling was also fed by the failure of large-scale educational compensatory programmes in the USA and other countries. After the initial effectiveness studies of Brookover et al. (1979) in the US and Rutter et al. (1979) in the UK a magnitude of research studies has been published about factors that make a difference between more and less effective education. The results of the studies of the first phase of EER revealed that teachers and schools differ among themselves in performance. At almost the same point in time this research was published in both the United States and the United Kingdom and the results got much attention in both the scholarly and the popular press. Ron Edmonds (1979), a school-board superintendent, particularly addressed educational practitioners and Brookover et al. the educational community. The five factor model, which he developed, influences education policy and practice. School reforms and improvement efforts were rated on the five factor model in order to `install' the factors in schools. In the US, EER and improvement efforts based on EER influenced both educational policy and practice between 1980 and 1990. Gradually the interest in EER decreased, the reasons for which will be discussed in the next section. In other countries, like Australia and Hongkong, interest for EER was raised by the publications in the US and the UK about the differences between effective and less effective schools and classrooms and supported by the improvement efforts and reforms undertaken in the respective countries (Townsend, Clarke & Ainscow, 1999). In mainland Europe countries are rather different with respect to educational effectiveness and improvement. In The Netherlands effectiveness and improvement research started already in the eighties and caught up quite easily with international (US and UK) research in this area. In some other countries, e.g. Cyprus it started rather late, and individual researchers picked up ideas from educational effectiveness and improvement. Even at the start there were some differences in interest between countries. In some countries the emphasis was originally on research into educational effectiveness and the methodology which is used in studies, like in The Netherlands. In other countries the emphasis or the starting point is educational reform and the emphasis is on the research and evaluation of educational innovations. Sometimes this turns into research on the effectiveness issues. This is not necessarily the case in all countries. In The Netherlands the emphasis is on research and it is used more or less in reforms. Belgium - Flanders is another example where originally it started as a research exercise and gradually it was used for reform efforts as well. In other countries however the prime interest is on the reforms in education and


especially curriculum reform and decentralisation of education. In relation to the improvement interests there is also more emphasis on issues like educational leadership, management organisation and curriculum issues. Examples are Belarus and Hungary and to a lesser extent Norway, Sweden and Germany. In some countries interest in educational effectiveness and improvement comes up and gradually declines. This is reflected in country reports as published in connection to the annual conferences of the International Congress of School Effectiveness and Improvement (Reynolds, Creemers & Peters, 1989; Creemers, Peters & Reynolds, 1989; Creemers & Osinga, 1995). Reviews about the development of educational effectiveness research and improvement provide a similar picture (Townsend, Clarke & Ainscow, 1999, and Teddlie and Reynolds, 2000). Countries only appear once and others are quite stable over time. When we also take into consideration publications in international journals like the Journal for School Effectiveness and School Improvement, we see quite a constant interest in educational effectiveness research in Belgium-Flanders, The Netherlands and Cyprus.

2. The reality: the combination of diverse programs The International Congress for School Effectiveness and School Improvement (ICSEI) was established to promote educational effectiveness research and school improvement efforts and especially to combine educational effectiveness and school improvement in order to contribute to educational policy and practice. It became clear, however, that educational effectiveness needed further development in theory and research. In the last 25 years, EER has improved considerably by listening to the criticisms on research design, sampling and statistical techniques. Methodological advances have enabled more efficient estimates of teacher and school differences in student achievement to be obtained (Goldstein, 2003). Anyone who now attempts to measure the effects of educational systems, schools and teachers has already encountered two methodological imperatives: `Collect longitudinal data'; and `Pay attention to the multilevel organisational structure' in which education occurs. As far as the theoretical aspects of the field are concerned, progress was made by a more precise definition of these concepts used and the relations between the concepts (e.g., Mortimore et al., 1988; Scheerens, 1992; Levine & Lezotte, 1990). However, there is a shortage of well-developed models from which researchers can build theory. The problem is aggravated by infrequent use of whatever models exist (Scheerens & Bosker, 1997). As a consequence, most of the studies on educational effectiveness are atheoretical and are concerned with the establishment of statistical relationships between variables rather than with the generation and testing of theories which could explain those relationships and contribute to the establishment of strategies for improving


educational effectiveness (Creemers, 2002). There are several reasons to argue that there is a need to develop and test models of effectiveness that could help us explain differences in student learning results by specifying the relationships between the components in the models and student outcomes. There is, however, also a downside to the methodological and theoretical advances. The knowledge base of educational effectiveness was less than expected in policy and practice, and in some parts less supported by empirical evidence than required. Moreover, a successful implementation of the evidence based knowledge was more complicated. Reforms and improvement efforts deal with participants who have their own views on effectiveness and quality instead of effective factors and characteristics. This resulted in a distinction, and sometimes tension, between school effectiveness and school improvement. This is described in several studies (see e.g. Creemers & Reezigt, 1997; Reynolds, Teddlie, Hopkins, Stringfield, 2000). All these descriptions result in a plea to (re)-establish a relationship between the two in order to make use of the mutual benefits of educational effectiveness on the one hand and educational improvement on the other. Empirically validated knowledge should be used in educational practice for educational improvement and the results of the evaluation of improvement efforts create the basis for theories about educational effectiveness. However, in general educational improvement concentrates on changes in schools through specific improvement projects. These projects emphasise the role and co-operation of different participants such as school-leaders, teachers, parents, students together and the support by internal and external advisors. Further educational improvement takes place through specific projects like for example Improving Educational Leadership (Huber, 2004) and specific strategies like school improvement through performance feedback (Visscher & Coe, 2002). These projects are only loosely related to EER but include some aspects of the knowledge base in the improvement efforts. This is also, and to an even larger extent, the case in educational reforms initiated at the system level as can be seen in programs like `No Child Left Behind' in the US and similar ones in other countries ( UK and Hong Kong) . In a gradual, incremental way elements of EER and theory are included, especially the, criticised, emphasis on student achievement gains and the added value of education. However, EER is no longer the leading principle in educational reforms and improvement efforts. This is fortunately so because research and theory in educational effectiveness cannot provide a complete knowledge base yet, and even when this may be the case in the future, policy and practice include other aspects as well and have their own responsibility (such as the participants and the change processes).


Educational effectiveness research is more successful in the evaluation of school reforms and improvement programmes. The design of the studies reflected the conceptual frameworks of educational effectiveness e.g. in the evaluation of the educational priority programme in The Netherlands and the design of the Dutch cohort studies in primary and secondary education and for the evaluation frameworks used by inspectorates in different countries like e.g. in BelgiumFlanders and The Netherlands. Mostly smaller school improvement projects make a link between the educational effectiveness knowledge base and the implementation of knowledge in the strategy for school improvement and finally evaluation in terms of student outcomes. In for example The Netherlands, Houtveen carried out different projects in which the school effectiveness knowledge base was combined with ideas about adaptive instruction and was implemented in schools. The results are successful as can be seen in the evaluation of the mathematics programme (Houtveen, Van de Grift & Creemers, 2004). Other examples are the school improvement programmes developed by Slavin (1996) and Hill & Crévola (1999). In an international programme in which several European countries have taken part an attempt is made to combine the effective educational knowledge base with school improvement programmes and to look for successful combinations of the two in European countries. The project resulted in a framework that can be used to design and implement an effective school improvement project (see for more details the SESI special 2005). The general conclusion can be drawn that at present the influence of educational effectiveness on practice and policy is modest and sometimes criticised (Thrupp, 2001 and the debate around it). The results of improvement projects in which effectiveness and improvement knowledge is combined points at the possibility to increase the contribution of effectiveness and improvement on educational practice. This issue points also to the fact that international comparative research and collaboration in the area of school effectiveness and school improvement is needed (Reynolds, 2000; Creemers, in press). 3. The future: the richness of the uncompleted · · .Educational policy (reform programmes)/ educational practice (improvement efforts) and EER have their own aims and programme, but may benefit from cooperation. This cooperation might be `fragmented' and restricted to elements, aspects or components.


For EER and Theory it holds that the inclusion of the results from research on differential effects ­ next to generic factors ­ might increase the relevance for practice and policy (moreover).


Teaching and learning are dynamic processes that are constantly adapting to changing needs and opportunities. Therefore, studies investigating the process of change in schools may have important implications for modelling educational effectiveness. Furthermore, such studies may help us look at the functioning of each factor using a dynamic rather than an instrumental perspective. For example, researchers could examine each factor using not only strictly quantitative ways of measuring each factor such as how frequently tasks associated with a factor take place but also qualitative ways that can help us find out when and under what conditions the functioning of a factor improves learning. Also, these studies will contribute significantly in the development of a dynamic model of EER (Creemers & Kyriakides, 2006) which could help us identify factors that are not only associated with student achievement at a certain period but are also able to explain changes in the effectiveness status of teachers, schools and educational systems. The model does not only concentrate on the current situation of schools and teachers but also illustrates the actions that have to be taken in order to improve their effectiveness. It is also expected that these actions may vary according to the current situation and the context of the school and its teachers (Datnow, Borman, Stringfield, Overman & Castellano, 2003; Stringfield, 1995). As a consequence, specific strategies for improving teaching could emerge from this model.



Brookover, W.B., Beady, C., Flood, P., & Schweitzer, J. (1979). School systems and student achievement: schools make a difference. New York: Praeger. Creemers, B.P.M. (1994). The Effective Classroom. London: Cassell. Creemers, B. P. M. (2002) From school effectiveness and school improvement to Effective School Improvement: background, theoretical analysis, and outline of the empirical study, Educational Research and Evaluation, 8 (4): 343-362. Creemers, B.P.M. & Kyriakides, L. (2006). A critical analysis of the current approaches to modelling educational effectiveness: the importance of establishing a dynamic model. School Effectiveness and School Improvement (in press). Creemers, B.P.M., & Osinga, N. (Eds.). (1995). ICSEI country reports. Leeuwarden: Gemeenschappelijk Centrum voor Onderwijsbegeleiding in Friesland. Creemers, B.P.M., Peters, T. & Reynolds, D. (Eds.) (1989). School Effectiveness and School Improvement. Amsterdam/Lisse: Swets & Zeitlinger. Creemers, B.P.M., & Reezigt, G.J. (1997). School effectiveness and school improvement: Sustaining links. School Effectiveness and School Improvement, 8(4) 396-429. Coleman, J.S., Campbell, E.Q., Hobson, C.F., McPartland, J., Mood, A.M., Weinfeld, F.D., & York, R.L. (1966). Equality of Educational Opportunity. Washington, DC: US Government Printing Office. Datnow, A., Borman, G.D., Stringfield, S., Overman, L.T. & Castellano, M. (2003). Comprehensive school reform in culturally and linguistically diverse contexts: Implementation and outcomes from a four-year study. Educational Evaluation and Policy Analysis 25, 143-170. Edmonds, R.R. (1979). Effective schools for the urban poor. Educational Leadership, 37(1), 15-27. Goldstein, H. (2003). (3rd Edition). Multilevel Statistical Models. London: Edward Arnold. Hill, P. & Crévola, C.A. (1999). Key features of a whole-school design approach to literacy teaching in schools, Australian Journal of Learning Disabilities, 4(3), 5-11 Houtveen, A.A.M., Grift, W.J.C.M. van de & Creemers, B.P.M. (2004), Effective School Improvement in Mathematics, in B.P.M. Creemers & D. Reynolds, School Effectiveness and School Improvement, Lisse: Taylor & Francis. Huber, S.G. (2004). Preparing School Leaders for the 21st Century. An International Comparison of Development Programs in 15 Countries. London/New York: Routledge-Falmer. Jencks, C., Smith, M., Acland, H., Bane, M.J., Cohen, D., Gintis, H., Heyns, B., & Michelson, S. (1972). Inequality: a Reassessment of the Effects of Family and Schooling in America. New York: Basic Books. Levine, D.U., & Lezotte, L.W. (1990). Unusually effective schools: a review and analysis of research and practice. Mortimore, P., Sammons, P., Stoll, L., Lewis, D., & Ecob, R. (1988). School Matters: The Junior Years. Wells: Open Books. Reynolds, D. (2000). School effectiveness: the international dimension. In Teddlie, C, Reynolds, D (red), The International Handbook of School Effectiveness Research, London: Falmer, pp. 232-256.


Reynolds, D., Creemers, B.P.M. & Peters, T. (1989). School Effectiveness and School Improvement. Proceedings of the First International Congress, London 1988. Cardiff: University of Wales College/Groningen: RION. Reynolds, D., Teddlie, C., Hopkins, D. & Stringfield, S. (2000). Linking school effectiveness and school improvement. In Teddlie, C, Reynolds, D (red), The International Handbook of School Effectiveness Research, London: Falmer, pp. 206-231. Rutter, M., Maughan, B., Mortimore, P., & Ouston, J. (1979). Fifteen Thousand Hours. London: Open Books. Scheerens, J. (1992). Effective Schooling: Research, Theory and Practice. London: Cassell. Scheerens, J., & Bosker, R.J. (1997). The foundations of educational effectiveness. Oxford: Pergamon. Slavin, R.E., (1996). Education for All. Lisse,: Swets & Zeitlinger. Stringfield, S. (1995). Attempting to enhance students' learning through innovative programs: the case for schools evolving into high reliability organisations. School Effectiveness and School Improvement, 6(1), 67-96. Teddlie, C. & Reynolds, D., (2000). The International Handbook of School Effectiveness Research. London: Falmer Press. Thrupp, M. (2001). Sociological and political concerns about school effectiveness research: time for a new research agenda. School Effectiveness and School Improvement 12(1), 7-40. Townsend, T., Clarke, P., & Ainscow, M. (Eds.) (1999). Third Millennium Schools. A World of Difference in School Effectiveness and School Improvement. Lisse:Swets and Zeitlinger Visscher, A.J. & Coe, R. (2002). School Improvement through Performance Feedback. Lisse: Swets & Zeitlinger.



Microsoft Word - A Modest but Meaningful Contribution of Educational Effectiveness and Improvement-1.doc

8 pages

Report File (DMCA)

Our content is added by our users. We aim to remove reported files within 1 working day. Please use this link to notify us:

Report this file as copyright or inappropriate


You might also be interested in

Microsoft Word - EURASIA_v7n3_Eilks.doc