Read What Shapes Language Teaching Trends text version

ANUPI

2007

GETTING TO KNOW STANDARDIZED TESTS OF ENGLISH LANGUAGE: CELE UNAM LANGUAGE TESTS, ELASH, TOEFL IBT

FRANCES BOYD

American Language Program, Columbia University

BARBARA BYER

Departamento de Linguistica Aplicada, Centro de Enseñanza de Lenguas Extranjeras, UNAM.

YLDA FARRE-RIGAU

University of Puerto Rico (retired); College Board Puerto Rico and Latin America

For some educators, the very phrase "standardized test" conjures up dispiriting images of students cutting class to cram for the test or teachers dumbing down curricula to teach to the test. In such scenarios, the desire for top scores on high-stakes standardized tests may seem to trump serious efforts to teach and learn. In English Language Teaching (ELT), standardized tests have been improved in recent years, yet they necessarily remain limited by our current notions of language proficiency, by the nature of testing itself, and, especially, by how language professionals choose to use the results. On balance, though, it is clear that some of these tests can perform useful functions and may even have a positive influence on English language education. In this paper, based on a plenary panel, we profile three standardized tests that are very different in form and purpose: the CELE UNAM Language Exams [Foreign Language Center, National Autonomous University of Mexico], ELASH [English Language Assessment System for Hispanics] from the College Board, and TOEFL iBT [Test of English as a Foreign Language, Internet-based test] from Educational Testing Service. By definition, standardized tests specify where, when, how, and for how long test takers may respond to the tests; also, such tests must meet strict standards of reliability and validity in construction. Here, the purpose is to describe each of these three English-language exams (or sets of exams), to compare and contrast them, and to raise questions about the alignment between these standardized exams and the local educational goals of administrators, teachers, and students. In the interests of full disclosure, we should state our involvement with the tests we write about. First and foremost, we are teachers with more than 80 years of combined ELT classroom experience. In addition, Barbara Byer ran the Evaluation and Certification Department at CELE UNAM for 14 years; Ylda Farre-Rigau is a consultant to the College Board Puerto Rico and Latin America; and Frances Boyd has collaborated with Educational Testing Service to produce TOEFL iBT preparation books.

CELE UNAM LANGUAGE EXAMS

CELE has been designing and applying certification tests in foreign language since its establishment in 1966. These exams are created by UNAM for those enrolled in a wide variety of its own degree-granting programs. Serving approximately 16, 000 UNAM students annually, the proficiency and reading comprehension exams certify that students

-1-

ANUPI

2007

have successfully met graduation requirements in English (and/or in several foreign languages). The passing score is 60% on each version. Written, administered, and scored in-house, these exams are accepted by UNAM and its branch and affiliated campuses throughout the Republic. The scores are also accepted for certification purposes by several other Mexican universities.

The homepage for CELE UNAM Language Exams is http://ianua.cele.unam.mx.

Perceptions of the Test Imagine: You have finally finished your major and are making the last corrections on your thesis. There are a lot of people waiting for you to graduate: parents, boss, girl/boyfriend, the thesis advisor who is retiring. However, your school records indicate that you haven't complied with your major's language requirement. Unless you pass a CELE reading comprehension or proficiency exam, there will be no graduation. Why does this happen? Many students perceive their language requirement as just one more administrative stumbling block on the long road to graduation. Others, with less confidence and preparation in a foreign language, fear the exam. Unfortunately, there are also teachers who fear the exams, mostly because the tests are an external evaluation of their courses. On the other hand, there are teachers who ignore the fact that their students will have to take one of these exams during their university years. Consequently, they don't prepare their students adequately in academic reading skills. Teachers also question the validity of basing something as important as graduation on the results of one test. This is a consideration that users of all high-stakes standardized exams have to deal with. We all know that no test can reflect everything a student knows or is able to do in a language. Still, these certification exams are well-respected and widely seen as fair and authentic. It is interesting to note, for example, that examinees often comment positively about the quality and interest of the texts that appear on the exams.

-2-

ANUPI

2007

Purpose The UNAM language exams are designed to meet the needs of UNAM degree candidates in a wide variety of fields. Therefore, they must satisfy very specific criteria in terms of topics, level of difficulty, cost, ease and flexibility of application. Each major or graduate program sets its own entrance and exit requirements, including those for language. The idea behind these language requirements is to ensure that students have the basic language skills in order to function in their academic and professional lives. Most graduates are expected to read a wide variety of technical texts in their fields with an acceptable degree of comprehension. A smaller number of departments also require that graduates have commensurate levels in listening, speaking, and writing. Content and Approach To meet the needs of the various University departments, CELE creates two types of language exams: reading comprehension and proficiency. Last year, 14,500 students sat for reading comprehension exams, while 1,400 took proficiency exams. Each test is carefully tailored to the population it serves. Most of the reading comprehension exams, far and away the more common, are written for specific academic areas (art, biology, law, medicine, and so on), though some general interest exams are produced. Generally, they contain two authentic texts in the foreign language, up to a total of approximately 2000 words, plus multiple-choice or gap-filling questions. Texts are selected from specialized academic publications and textbooks, not materials published for the general public. Testing points can be main ideas or supporting arguments, author's intent or conclusions, the purpose of an example, a relevant fact, recognition of the document's original purpose, inferences using information from the text, and word meaning determined by context. Dictionaries are permitted, and directions are in Spanish. The number of items varies, but the test is no longer than 90 minutes. In contrast, the proficiency exams typically measure all four language skills. For listening comprehension, candidates listen to an authentic video recording then respond to short-answer comprehension questions and multiple-choice grammar and lexis questions. To demonstrate speaking ability, candidates are interviewed. For writing, they are given a

-3-

ANUPI

2007

task that relates to their academic interests, for example, a description of a research project that they have recently completed or are planning to undertake. The reading comprehension portion is essentially the same as in the reading-comprehension only exams; however, dictionary use is not permitted. There are other differences, too. The proficiency tests are longer -- 2 ½ to 3 hours-- and all instructions are in the target language. Moreover, these exams are designed to be at the upper-intermediate level, with each skill given equal weight in the final score. Impact on Curriculum and Motivation To pass the reading comprehension exams, students need a general knowledge of English with, above all, experience reading authentic texts in their fields: biology, business, engineering, and so on. Students can develop an adequate level of reading by taking four-skills courses or, more indirectly, by taking content courses in their major, many of which use textbooks and other materials in English. Fluent reading can also be honed in optional separate-skills courses such as those offered at CELE. To pass the proficiency exams, students must demonstrate upper-intermediate levels of reading, as well as of listening, speaking, and writing. Students who fail the CELE UNAM language certification exams often do so because they are overconfident and fail to read carefully, run out of time, or are not familiar enough with the test format. Of course, there are also those who simply do not have adequate knowledge of the language. For reading comprehension courses, instructional test preparation, and exam practice, there are many resources available. CELE itself publishes course books and materials. It also offers online practice exams. In addition, other UNAM schools publish materials, as do commercial publishers [1].

ELASH

First administered in 1999, the English Language Assessment System for Hispanics, or ELASH, is intended primarily as a placement test. It was created by the College Board Puerto Rico and Latin America to help institutions place Spanish-speaking students enrolled or planning to enroll in English classes in their own countries. Currently serving institutions in

-4-

ANUPI

2007

eight countries, ELASH I and ELASH II spread students over a wide range, from novice to advanced levels. Written and scored by the College Board but administered by each institution, these 100-120-minute exams are gaining in popularity in Latin America, the United States, and Puerto Rico.

The homepage for ELASH is: http://oprla.collegeboard.com/ptorico/latinam/program/elash.html.

Perceptions of the Test ELASH is seen as a useful institutional instrument with particular relevance for Spanish speakers intending to study English. All of the test items are in English, but the directions on the test and the Student Guide that describes the test are both in Spanish. The content of the exam is culturally sensitive. Moreover, the items take into account linguistic interference between Spanish and English. Test administrators can choose between ELASH I (novice to low-intermediate) and ELASH II (high-intermediate to advanced), depending on the English Language level of their population. Both versions are paper-and-pencil tests that can be given on dates chosen by each institution, making them convenient and affordable. Purpose ELASH is designed to meet the needs of institutions that teach English to Spanish speakers as well as employers who want to measure English skills. Candidates for this exam may be in schools, in levels from sixth grade on to university, or they may be in the workplace. As a placement tool, ELASH I and II sort groups of students from very beginning through advanced levels of English. ELASH score reports serve the institutions in several ways. To start, College Board provides an alphabetized list of individuals, each with English-language proficiency scores on a scale from 40-200. To help school and other officials interpret the scores, Proficiency Stages and Proficiency Descriptors are included. Beyond this, College Board provides a statistical analysis of each group of test takers: distribution of total test scores, distribution by content areas represented on the exam, mean scores on the Exposure to English Questionnaire, and mean scores by age and grade level, if applicable. In the near future, institutions will also be able to align ELASH scores with the descriptors of the Common European Framework for Languages. Content and Approach ELASH measures listening and reading comprehension using content from a range of subjects. In a multiple-choice format, both ELASH I and II have three sections: listening

-5-

ANUPI

2007

and comprehension; language usage and "indirect" writing; vocabulary and reading comprehension. On each test, there are 135 items to be completed in 100 minutes. In the listening section, students hear pieces of increasing length, from rejoinders to short conversations to short talks and extended discourse. Items test their understanding of main ideas and details. In the language usage and indirect writing section, test takers encounter increasingly complex grammatical and sentence structures. They also move from improving sentences to improving paragraphs. Finally, in the vocabulary and reading section, test takers figure out meaning from context in excerpts of increasing length and complexity. In the construction of ELASH, great care is given to cultural sensitivity. Content in the reading sections covers issues in Latin America that are relevant to the test takers' reality. Moreover, the item writers and exam committee members are teachers who actually work in the different countries where the exams are used. They are, therefore, familiar with cultural differences and nuances of language. Impact on Curriculum and Motivation The publishers do not recommend any specific test preparation. When ELASH is used for placement, the best preparation is simply to become familiar with the mechanics of a multiple choice test: item types, instructions, time limits, and answer sheet format. In those countries and institutions where ELASH has other uses, teaching to the test is never recommended. Again, the best preparation is familiarity with item types and general academic content. Each institution decides when to administer ELASH, how often, and for what institutional purpose. For more information, test takers can read the ELASH Student Guide, which is in Spanish. Because ELASH focuses on listening, language usage, and reading comprehension, institutions may want to refine their placements or admissions criteria with additional speaking and writing assessments. To link placement to the local curriculum, ELASH users can make good use of the score summaries regarding content area knowledge and exposure to English. The Proficiency Descriptors include additional information for classroom teachers. These "can

-6-

ANUPI

2007

do" statements describe what the students should be able to do within each specific language skill.

TOEFL IBT

First administered in 1964, the Test of English as a Foreign Language, or TOEFL, was created by Educational Testing Service to be used in the admissions process for international students applying to English-speaking colleges and universities. The TOEFL has been radically revamped and, beginning in 2006, the new communicative version has gradually become available in most countries, certainly throughout Mexico. Serving over 800, 000 international students annually, TOEFL iBT places students on a scale up to 120, where 80 is frequently set as the minimal score for admission. Written, administered, and scored under the auspices of Educational Testing Service, TOEFL iBT is a four-hour exam that students take online at approved testing centers.

The homepage for TOEFL iBT is www.ets.org/toefl or www.toefl.org.

Perceptions of the Test In its earlier versions, TOEFL was a test students could beat by cramming themselves full of rare grammatical points and arcane vocabulary. We occasionally had students at Columbia's American Language Program with very high scores, say 600 or more, who could barely speak. In contrast, in the new Internet-based version, TOEFL tests not what you know but what you can do in English. Most teachers and test takers consider it a fairer and more authentic test. Purpose TOEFL iBT is intended to measure ability to communicate in academic settings where English is the language of instruction. Inside the classroom, this includes the skills of understanding and taking notes on lectures; answering and posing questions; reading, discussing and writing on academic topics. Outside the classroom, communication includes requesting information from a librarian, negotiating housing issues, requesting appropriate medical care, and so on.

-7-

ANUPI

2007

Score reports give a composite score as well as performance descriptors by skill. The test discriminates best at the upper-intermediate to low-advanced levels of proficiency; less usefully at the very lowest and highest ends. Content and Approach The Internet-based TOEFL is a dynamic test of communicative competence and academic skills with integrated-skill tasks, note taking, a speaking section, and items requiring critical thinking in every section. In simulated lecture situations, test takers must show that they can learn as they go: for example, take notes on a 5-minute lecture on how baby birds learn their species' song, and then use these notes to respond to comprehension, inference, and vocabulary items. Challenging integrated tasks ask students to read a short passage on the theory of team-building, for instance, listen to a businessperson give an example that disputes the theory, then summarize the gist of the arguments in writing. This task is challenging precisely because it effectively integrates reading and listening comprehension, critical thinking, and summary writing. In a short form, this simulates what university students often have to do. Speaking tasks on the new TOEFL are particularly difficult as students have very short preparation (15-30 seconds) and response times (60 seconds or less) and face the awkwardness of talking to a computer. Moreover, some questions require students to answer big and potentially emotional items such as commenting on a person who influenced your life. Impact on Curriculum and Motivation Revising the TOEFL was a calculated decision both to create a more up-to-date test and to stimulate positive washback in the field of ELT. By applying the latest research in the field, the test designers want to exercise their influence on English-language teachers and students to do the same. When the test improves, teaching to it may actually have some positive results. For many institutions and programs involved in teaching English, keeping pace with the new test may require far-reaching change. The curriculum that helps students learn academic English as well as score well on the TOEFL is one that is organized thematically, with integrated skills and critical thinking at all proficiency levels. Already, ELT curricula in

-8-

ANUPI

2007

some places are evolving to offer students rich, content-based lessons on unfamiliar topics (or aspects of topics) and new points of view. For many native-speaking and non-nativespeaking teachers alike, there are also challenges. Learning how to evaluate speaking and writing with rubrics and to give developmental feedback is one challenge. Reimagining ourselves as co-learners on "a journey of discovery" with students is another [2]. Displaying curiosity and the confidence to learn from mistakes in order to inspire these qualities of mind in students may be still another. Perhaps the wise words of Sir Winston Churchill can encourage us in this period of adjustment to more communicative and integrated approaches to English-language teaching and learning. "Success," he famously remarked, "is the ability to go from one failure to another with no loss of enthusiasm." For TOEFL preparation efforts also, the Internet-based test implies dramatic change. Instead of learning the tricks of the trade, it is now a rational choice to learn "the trade" itself: English. When academic English students learn language through thematic units incorporating integrated skills and critical thinking, they are gradually training their brains to succeed at university as well as on the TOEFL. Such academic skills include identifying main ideas and supporting details, making inferences, comparing and contrasting, paraphrasing, identifying rhetorical structure in reading and listening texts, and so on. For some students and teachers, both the language skills and content have a strong cultural component; thus, the learning is multi-layered and complex. The test designers acknowledge a place for "instructional test preparation," that is, thematic units that mimic the test items on TOEFL iBT. With such materials, students move step-by-step to improve their language and academic skills, while gradually developing awareness of these skills and honing test-taking strategies [3]. To help teachers, Educational Testing Service (ETS) offers training kits and online practice in scoring; for students and teachers, there are explanations and practice tests at the ETS web site as well as practice books available for purchase from many ELT publishers [4]. In addition, experienced teachers still see a place for short, focused test-prep classes for students who have already attained at least a high-intermediate level. One 30-year veteran remarks: "Test prep is like a sport." She goes on to expand the sports analogy: "You need a mindset, a strategy to help you perform against your big opponent

-9-

ANUPI

2007

on that one day. Preparation should be intensive -- 4 to 6 hours a day-- just like the exam. It helps students with their mental state." [5] Although cramming cannot develop communicative competence in English, in the run-up to the exam, a simulation of test performance conditions can help test takers show what they know on the test day. Such a course might consist of about 30 hours of work on practice exams (with English keyboard and microphone) spread out in 5 sessions, about a month before the test date.

CONCLUSION

Standardized tests are ultimately a convenience: a quicker, cheaper alternative to more painstaking and personal measures of language proficiency. Still, as we have seen with the CELE UNAM Language Tests, ELASH and TOEFL iBT, standardized tests need not have a negative effect. In fact, they may complement teacher-led assessments and give students independent confirmation of their skills. Further, such tests have the potential advantages of eliminating some bias and of drawing us into a more public dialog about assessment. This conversation may include stakeholders as varied as academic departments, workplaces, non-profit agencies, and commercial publishers. The complexity of language assessment and the need for it on such a large scale help explain why it has spawned such an enormous industry. Among the myriad instruments that may also be of interest are US-published ACCUPLACER and The Michigan Test; and UK-published IELTS and Cambridge First Certificate. While the choice of standardized tests seems wide, the educational purpose is specific and often local: admission, placement, and/or certification. Just how a standardized test aligns with one's purpose and, further, how such a test influences the teaching and learning of English are issues that merit our professional attention.

- 10 -

ANUPI

2007

NOTES

1. For classroom-based reading comprehension courses , see these CELE materials: Alvarez, G. y Galicia, F., Manuel de Comprensión de Textos Médicos en Inglés, CELE, UNAM, 1996. Martineck, L. y Tobío, C., Exámenes de Comprensión de Lectura en Inglés: Medicina, CELE, UNAM, 1995. Tobío, C., Martineck, L. y Moguel, C., Exámenes de Comprensión de Lectura en Inglés: Contaduría y Administración, CELE, UNAM, 2003. Tobío, Carmen, Exámenes de Comprensión de Lectura en Inglés, CELE, UNAM, 4ª ed. 2004. DVD: Valdés, Jesús, In love with Shakespeare o cómo usar el diccionario de inglés. For an online reading-comprehension course, go to http://ianua.cele.unam.mx/leeringles/ For online practice exams in reading comprehension, go to http://conorare.cele.unam.mx/complectauto. Then, click on "Examenes." 2. Numrich, Carol. 2001. Theme-Based Instruction: Fieldwork in a Small Connecticut Town. In Understanding the Courses We Teach: Local Perspectives on English Language Teaching, edited by John Murphy and Patricia Byrd. Ann Arbor, MI: University of Michigan Press. 3. Boyd, F. and C. Numrich, eds., NorthStar: Building Skills for the TOEFL iBT. In cooperation with Educational Testing Service. White Plains, NY: Pearson Education, 2006. Intermediate, high-intermediate, and advanced. Teaching manuals with detailed instruction in scoring speaking and writing with rubrics. 4. For teachers, Educational Testing Service (ETS) offers: Propell Workshop for TOEFL iBT. Teacher Workshop. www.ets.org/toefl/workshops.html. For students and teachers, ETS also offers: TOEFL Practice Online, an official web site. TOEFL Academic Speaking Test, practice tests for in-class or online use. 5. Interview with Janet Shanks, October 19, 2007, at the American Language Program, Columbia University.

- 11 -

Information

What Shapes Language Teaching Trends

11 pages

Find more like this

Report File (DMCA)

Our content is added by our users. We aim to remove reported files within 1 working day. Please use this link to notify us:

Report this file as copyright or inappropriate

1340279


You might also be interested in

BETA
SEVENTH GRADE
Umbra Institute
Microsoft Word - Compilation bib _2003-2005_ current.doc
SEATTLE UNIVERSITY