Assignment 7

A history of standardized testing in the United States reveals that during most of the decades in the middle of the twentieth century, standardized tests enjoyed a popularity and growth that was almost unchallenged. Standardized instruments brought with them convenience, efficiency, and an air of empirical science. In schools, for example, millions of children could be led into a room, seated, armed with a lead pencil and a score sheet, and almost instantly assessed on their achievement in subject-matter areas in their curricula. Standardized test advocates' utopiandream of quickly and cheaply assessing students across the country soon became a political issue, and would-be office holders to this day promise to "reform" education with tests, tests, and more tests. Toward the end of the twentieth century, such claims began to be challenged on all frunts (see Medina & Neill, 1990; Kohn, 2000), and at the vanguard of those challenges were the teachers of those millions of children.

ELD STANDARDS

The process of designing and conducting appropriate periodic reviews of ELD standards involves dozens of curriculum and assessment specialists, teachers, and researchers (Fields, 2000; Kuhlman, 2001). In creating such "benchmarks for accountability" (O'Malley & Valdez Pierce, 1996), there is a tremendous responsibility to carry out a comprehensive study of a number of domains:

a) literally thousands of categories of language ranging from phonology at one end of a continuum to discourse, pragmatics, functional, and sociolinguistic elements at the other end;

b) specification of what ELD students' needs are, at thirteen different grade levels, for succeeding in their academic and social development;

c) a consideration of what is a realistic number and scope of standards to be included within a given curriculum;

d)a separate set of standards (qualifications, expertise, training) for teachers to teach ELD students successfully in their classrooms; and

e) a thorough analysis of the means available to assess student attainment of those standards.

Standards-setting is a global challenge. In many non-English-speaking countries, English is now a required subject starting as early as the first grade in some countries and by the seventh grade in virtually every country worldwide. In Japan and Korea, for example, a "communicative" curriculum in English is required from third grade onward. Such mandates from ministries of education require the specification of standards on which to base curricular objectives, the teachability of  which has been met with only limited success in some areas (Chinen, 2000 Yoshida, 2001; Sakamoto, 2002). California, with one of the largest populations of second language learners in the United States, was one of the first states to generate standards. Other states follow similar sets of standards.

CASAS AND SCANS

At the higher levels of education (colleges, community colleges, adult schools language schools, and workplace settings), standards-based assessment systems have also had an enormous impact. The Comprehensive Adult Student Assessment System (CASAS), for example, is a program designed to provide broadly based assessments of ESL curricula across the United States. The system includes more than 80 standardized assessment instruments used to place learners in programs diagnose learners' needs, monitor progress, and certify mastery of functions basic skills. CASAS assessment instruments are used to measure functions reading, writing, listening, and speaking skills, and higher order thinking skills CASAS scaled scores report learners' language ability levels in employment and adult life skills contexts.

The competence cover language functions in terms of

a)      resources (allocating time, materials, staff, etc.),

b)      interpersonal skills, teamwork, customer service, etc.,

c)      information processing, evaluating data, organizing files, etc.,

d)     systems (e.g., understanding social and organizational systems), and

technology use and application.

TEACHER STANDARDS

In addition to the movement to create standards for learning, an equally strong move ment has emerged to design standards for teaching. Cloud (2001, p. 3) noted that a student's "performance (on an assessment) depends on the quality of the instructional program provided, which depends on the quality of professional development."

Kuhlman (2001) emphasized the importance of teacher standards in three domains:

a)      linguistics and language development

b)      culture and the interrelationship between language and culture

c)      planning and managing instruction

Professional teaching standards have also been the focus of several committees in the international association of Teachers of English to Speakers of Other Languages (TESOL).

THE CONSEQUENCES OF STANDARDS-BASED

AND STANDARDIZED TESTING

A couple of decades ago I had the pleasure and challenge of serving on the TOEFT Research Committee. Among other things, it was a good opportunity to hear soul of the "inside" stories about the TOEFL. One of those stories, as told by Russ Webster (personal communication), illustrates the high-stakes nature of this global marketed standardized test. A ring of enterprising "business" persons organized a group of pretend the takers to take the TOEFL in an early time zone on a given day. (In those days the the were administered everywhere on the same day across a number of time zones. TOEFL administrations ended in some East Asian countries as much as 8 to 14 hour before they began in the United States.) The task of each test-taking "spy" was not to pass the TOEFL, but to memorize subset of items, including the stimulus and all of the multiple-choice options, a immediately upon leaving the exam to telephone those items to the central or nizers. As the memorized subsections were called in a complete form of the TOI was quickly reconstructed. The organizers had employed expert consultants to great the correct response for each item, thereby re-creating the test items and the  correct answers! For an outrageous price of many thousands of dollars, prearran buyers of the results were given copies of the test items and correct responses a few hours to spare before entering a test administration in the Western Hemisph. The story of how this underhanded group of entrepreneurs were caught brought to justice is a long tale of blockbuster spy-novel proportions involving the and eventually, international investigators.

Test promoters commonly use such findings to support their claims for the efficacy of their tests. But several nagging, persistent issues emerge from the arguments about the consequences of standardized testing. Consider the following interrelated questions:

a)      Should the educational and business world be satisfied with high but not perfect probabilities of accurately assessing test-takers on standardized instruments? In other words, what about the small minority who are not fairly assessed?

b)      Regardless of construct validation studies and correlation statistics, should further types of performance be elicited in order to get a more comprehensive picture of the test-taker?

c)      Does the proliferation of standardized tests throughout a young person's life give rise to test-driven curricula, diverting the attention of students from creative or personal interests and in-depth pursuits?

d)     Is the standardized test industry in effect promoting a cultural, social, and political agenda that maintains existing power structures by assuring opportunity to an elite (wealthy) class of people?

TEST BIAS

It is no secret that standardized tests involve a number of types of test bias. That bias comes in many forms, language, culture, race, gender, and learning styles (Medina & Neill, 1990). The National Center for Fair and Open Testing, in its bimonthly newsletter Fair Test, every year offers dozens of instances of claims of test bias from teachers, parents, students, and legal consultants (see their website: www.fairtest.org). For example, reading selections in standardized tests may use a passage from a literary piece that reflects a middle-class, white, Anglo-Saxon norm. Lectures used for listening stimuli can easily promote a biased sociopolitical view. Consider the following prompt for an essay in "general writing ability" on the IELTS: You rent a house through an agency. The heating system has stopped working. You phoned the agency a week ago, but it has still not been mended. Write a letter to the agency. Explain the situation and tell them what you want them to do about it. While this task favorably illustrates the principle of authenticity, a number of cultural and economic presuppositions are evident in such a prompt, calling into question its potential cultural bias.

ETHICAL ISSUES: CRITICAL LANGUAGE TESTING

One of the by-products of a rapidly growing testing industry is the danger of an abuse of power in a special report on "fallout from the testing explosion," Medina and Neill (1990, p. 36) noted: Unfortunately, too many policymakers and educators have ignored the complexities of testing issues and the obvious limitations they should place on standardized test use. Instead, they have been seduced by the promise of simplicity and objectivity. The price which has been paid by our schools and our children for their infatuation with tests is high. Shohamy (1997, p. 2) further defines the issue: "Tests represent a social technology deeply embedded in education, government, and business; as such they provide the mechanism for enforcing power and control. Tests are most powerful as they are often the single indicators for determining the future of individuals." Test designers, and the corporate sociopolitical infrastructure that they represent, have an obligation to maintain certain standards as specified by their client educational institutions. These standards bring with them certain ethical issues surrounding the gate-keeping nature of standardized tests.

These issues are not new. More than a century ago, British educator E Y. Edgeworth (1888) challenged the potential inaccuracy of contemporary qualifying examinations for university entrance. In recent years, the debate has heated up. In 1997, an entire issue of the journal Language Testing was devoted to questions about ethics in language testing One of the problems highlighted by the push for critical language testing is the widespread conviction, already alluded to above, that carefully constructed standard ized tests designed by reputable test manufacturers are infallible in their predictive validity. One standardized test is deemed to be sufficient; follow-up measures are con sidered to be too costly A further problem with our test-oriented culture lies in the agendas of those who design and those who utilize the tests, Tests are used in some countries to deny citzen ship (Shohamy, 1997, p. 10).

Tests may by nature be culture biased and therefore may disenfranchise members of a nonmainstream value system. Test givers are always in a position of power over test-takers and therefore can impose social and political ideologies on test-takers through standards of acceptable and unacceptable items. Tess promote the notion that answers to real-world problems have unambiguous right and wrong answers with no shades of gray. A corollary to the latter is that tests presume to reflect an appropriate core of common knowledge, such as the competence reflected in the standards discussed earlier in this chapter. Logic would therefore tate that the test-taker must buy in to such a system of beliefs in order to make the c! Language tests, some may argue, are less susceptible than general knowledge tes to such sociopolitical overtones.

The research process that undergirds the TOEFL ga to great lengths to screen out Western cultural bias, monocultural belief systems, other potential agendas. Nevertheless, even the process of the selection of contes alone for the TOEFL involves certain standards that may not be universal, and the fact that the TOEFL is used as an absolute standard of English proficiency by most versities does not exonerate this particular standardized test. As a language teacher, you might be able to exercise some influence in ways tests are used and interpreted in your own milieu. If you are offered variety of choices in standardized tests, you could choose a test that offers least degree of cultural bias. Better yet, you might encourage the use of multi measures of performance (varying item types, oral and written production, other alternatives to traditional assessment) even though this might cost money. Further, you and your co-teachers might help establish an institution system of evaluation that places less emphasis on standardized tests and emphasis on an ongoing process of formative evaluation. In so doing, you mis be offering educational opportunity to a few more people who would other be eliminated from contention

EXERCISES

[Note: (1) Individual work: (G) Group or pair work; (C) Whole-class discussion.)

1. (C) Evaluate the standards-based assessment movement. What are its advantages

and disadvantages? How might one compensate for potential disadvantages?

2. (1) Consult the California English Language Development Test websites listed.

From what you can glean from that information, how would you evaluate the

CELDT in terms of content validity, face validity, and authenticity?

3. (1) Consult the TESOL website on teacher standards (page 109). What would

you say are a few minimal standards that language teachers should measure

up to? In your own institutional context, or one that you are familiar with,

how would you assess a teacher's attainment of those standards?

(G/C) Look at the four questions posed on page 111 regarding the conse-

quences of standardized testing. In groups, one question for each group, or as

a whole class, respond to those questions,

5. (1) Log on to the website for the National Center for Fair and Open Testing

(see page 111). Report back to the class on the topics and issues sponsored

by that organization.

6. (C) Explain the claim that "test-takers are political subjects in a political con-

text" (page 113) and Shohamy's assertion that large-scale standardized testing

is the "agent of cultural, social, political, educational, and ideological agendas."

Draw up a list of Dos and DON'Ts through which teachers might overcome the

potential political agendas in the use of standardized tests.

Source:

Brown, H. Douglas. 2003. Language Assessment Principles and Classroom Practices. San Francisco, California