A
history of standardized testing in the United States reveals that during most of
the decades in the middle of the twentieth century, standardized tests enjoyed
a popularity and growth that was almost unchallenged. Standardized instruments brought
with them convenience, efficiency, and an air of empirical science. In schools,
for example, millions of children could be led into a room, seated, armed with
a lead pencil and a score sheet, and almost instantly assessed on their achievement
in subject-matter areas in their curricula. Standardized test advocates'
utopiandream of quickly and cheaply assessing students across the country soon
became a political issue, and would-be office holders to this day promise to
"reform" education with tests, tests, and more tests. Toward the end
of the twentieth century, such claims began to be challenged on all frunts (see
Medina & Neill, 1990; Kohn, 2000), and at the vanguard of those challenges
were the teachers of those millions of children.
ELD STANDARDS
The process of
designing and conducting appropriate periodic reviews of ELD standards involves
dozens of curriculum and assessment specialists, teachers, and researchers
(Fields, 2000; Kuhlman, 2001). In creating such "benchmarks for accountability"
(O'Malley & Valdez Pierce, 1996), there is a tremendous responsibility to
carry out a comprehensive study of a number of domains:
a) literally
thousands of categories of language ranging from phonology at one end of a
continuum to discourse, pragmatics, functional, and sociolinguistic elements at
the other end;
b) specification
of what ELD students' needs are, at thirteen different grade levels, for
succeeding in their academic and social development;
c) a
consideration of what is a realistic number and scope of standards to be included
within a given curriculum;
d)a
separate set of standards (qualifications, expertise, training) for teachers to
teach ELD students successfully in their classrooms; and
e) a
thorough analysis of the means available to assess student attainment of those
standards.
Standards-setting
is a global challenge. In many non-English-speaking countries, English is now a
required subject starting as early as the first grade in some countries and by
the seventh grade in virtually every country worldwide. In Japan and Korea, for
example, a "communicative" curriculum in English is required from third
grade onward. Such mandates from ministries of education require the specification
of standards on which to base curricular objectives, the teachability of which has been met with only limited success
in some areas (Chinen, 2000 Yoshida, 2001; Sakamoto, 2002). California, with
one of the largest populations of second language learners in the United
States, was one of the first states to generate standards. Other states follow
similar sets of standards.
CASAS AND SCANS
At
the higher levels of education (colleges, community colleges, adult schools language
schools, and workplace settings), standards-based assessment systems have also
had an enormous impact. The Comprehensive Adult Student Assessment System
(CASAS), for example, is a program designed to provide broadly based assessments
of ESL curricula across the United States. The system includes more than 80
standardized assessment instruments used to place learners in programs diagnose
learners' needs, monitor progress, and certify mastery of functions basic
skills. CASAS assessment instruments are used to measure functions reading,
writing, listening, and speaking skills, and higher order thinking skills CASAS
scaled scores report learners' language ability levels in employment and adult
life skills contexts.
The competence cover
language functions in terms of
a) resources
(allocating time, materials, staff, etc.),
b) interpersonal
skills, teamwork, customer service, etc.,
c) information
processing, evaluating data, organizing files, etc.,
d) systems
(e.g., understanding social and organizational systems), and
technology use and application.
TEACHER STANDARDS
In addition to the
movement to create standards for learning, an equally strong move ment has
emerged to design standards for teaching. Cloud (2001, p. 3) noted that a student's
"performance (on an assessment) depends on the quality of the
instructional program provided, which depends on the quality of professional
development."
Kuhlman (2001)
emphasized the importance of teacher standards in three domains:
a) linguistics
and language development
b) culture
and the interrelationship between language and culture
c) planning
and managing instruction
Professional
teaching standards have also been the focus of several committees in the
international association of Teachers of English to Speakers of Other Languages
(TESOL).
THE
CONSEQUENCES OF STANDARDS-BASED
AND
STANDARDIZED TESTING
A
couple of decades ago I had the pleasure and challenge of serving on the TOEFT Research
Committee. Among other things, it was a good opportunity to hear soul of the
"inside" stories about the TOEFL. One of those stories, as told by
Russ Webster (personal communication), illustrates the high-stakes nature of
this global marketed standardized test. A ring of enterprising
"business" persons organized a group of pretend the takers to take
the TOEFL in an early time zone on a given day. (In those days the the were
administered everywhere on the same day across a number of time zones. TOEFL
administrations ended in some East Asian countries as much as 8 to 14 hour before
they began in the United States.) The task of each test-taking "spy"
was not to pass the TOEFL, but to memorize subset of items, including the
stimulus and all of the multiple-choice options, a immediately upon leaving the
exam to telephone those items to the central or nizers. As the memorized
subsections were called in a complete form of the TOI was quickly
reconstructed. The organizers had employed expert consultants to great the
correct response for each item, thereby re-creating the test items and the correct answers! For an outrageous price of
many thousands of dollars, prearran buyers of the results were given copies of
the test items and correct responses a few hours to spare before entering a
test administration in the Western Hemisph. The story of how this underhanded
group of entrepreneurs were caught brought to justice is a long tale of
blockbuster spy-novel proportions involving the and eventually, international
investigators.
Test promoters commonly
use such findings to support their claims for the efficacy of their tests. But
several nagging, persistent issues emerge from the arguments about the consequences
of standardized testing. Consider the following interrelated questions:
a) Should
the educational and business world be satisfied with high but not perfect
probabilities of accurately assessing test-takers on standardized instruments?
In other words, what about the small minority who are not fairly assessed?
b) Regardless
of construct validation studies and correlation statistics, should further
types of performance be elicited in order to get a more comprehensive picture
of the test-taker?
c) Does
the proliferation of standardized tests throughout a young person's life give
rise to test-driven curricula, diverting the attention of students from creative
or personal interests and in-depth pursuits?
d) Is
the standardized test industry in effect promoting a cultural, social, and political
agenda that maintains existing power structures by assuring opportunity to an
elite (wealthy) class of people?
TEST BIAS
It
is no secret that standardized tests involve a number of types of test bias.
That bias comes in many forms, language, culture, race, gender, and learning
styles (Medina & Neill, 1990). The National Center for Fair and Open
Testing, in its bimonthly newsletter Fair Test, every year offers dozens of
instances of claims of test bias from teachers, parents, students, and legal
consultants (see their website: www.fairtest.org). For example, reading
selections in standardized tests may use a passage from a literary piece that
reflects a middle-class, white, Anglo-Saxon norm. Lectures used for listening
stimuli can easily promote a biased sociopolitical view. Consider the following
prompt for an essay in "general writing ability" on the IELTS: You
rent a house through an agency. The heating system has stopped working. You phoned
the agency a week ago, but it has still not been mended. Write a letter to the
agency. Explain the situation and tell them what you want them to do about it. While
this task favorably illustrates the principle of authenticity, a number of cultural
and economic presuppositions are evident in such a prompt, calling into question
its potential cultural bias.
ETHICAL ISSUES:
CRITICAL LANGUAGE TESTING
One of the by-products
of a rapidly growing testing industry is the danger of an abuse of power in a
special report on "fallout from the testing explosion," Medina and
Neill (1990, p. 36) noted: Unfortunately, too many policymakers and educators
have ignored the complexities of testing issues and the obvious limitations
they should place on standardized test use. Instead, they have been seduced by
the promise of simplicity and objectivity. The price which has been paid by our
schools and our children for their infatuation with tests is high. Shohamy
(1997, p. 2) further defines the issue: "Tests represent a social
technology deeply embedded in education, government, and business; as such they
provide the mechanism for enforcing power and control. Tests are most powerful
as they are often the single indicators for determining the future of
individuals." Test designers, and the corporate sociopolitical
infrastructure that they represent, have an obligation to maintain certain
standards as specified by their client educational institutions. These
standards bring with them certain ethical issues surrounding the gate-keeping nature
of standardized tests.
These
issues are not new. More than a century ago, British educator E Y. Edgeworth (1888)
challenged the potential inaccuracy of contemporary qualifying examinations for
university entrance. In recent years, the debate has heated up. In 1997, an
entire issue of the journal Language Testing was devoted to questions about
ethics in language testing One of the problems highlighted by the push for
critical language testing is the widespread conviction, already alluded to
above, that carefully constructed standard ized tests designed by reputable
test manufacturers are infallible in their predictive validity. One
standardized test is deemed to be sufficient; follow-up measures are con
sidered to be too costly A further problem with our test-oriented culture lies in
the agendas of those who design and those who utilize the tests, Tests are used
in some countries to deny citzen ship (Shohamy, 1997, p. 10).
Tests
may by nature be culture biased and therefore may disenfranchise members of a
nonmainstream value system. Test givers are always in a position of power over
test-takers and therefore can impose social and political ideologies on
test-takers through standards of acceptable and unacceptable items. Tess promote
the notion that answers to real-world problems have unambiguous right and wrong
answers with no shades of gray. A corollary to the latter is that tests presume
to reflect an appropriate core of common knowledge, such as the competence reflected
in the standards discussed earlier in this chapter. Logic would therefore tate
that the test-taker must buy in to such a system of beliefs in order to make
the c! Language tests, some may argue, are less susceptible than general
knowledge tes to such sociopolitical overtones.
The
research process that undergirds the TOEFL ga to great lengths to screen out
Western cultural bias, monocultural belief systems, other potential agendas.
Nevertheless, even the process of the selection of contes alone for the TOEFL
involves certain standards that may not be universal, and the fact that the
TOEFL is used as an absolute standard of English proficiency by most versities
does not exonerate this particular standardized test. As a language teacher,
you might be able to exercise some influence in ways tests are used and
interpreted in your own milieu. If you are offered variety of choices in
standardized tests, you could choose a test that offers least degree of
cultural bias. Better yet, you might encourage the use of multi measures of
performance (varying item types, oral and written production, other
alternatives to traditional assessment) even though this might cost money.
Further, you and your co-teachers might help establish an institution system of
evaluation that places less emphasis on standardized tests and emphasis on an ongoing
process of formative evaluation. In so doing, you mis be offering educational
opportunity to a few more people who would other be eliminated from contention
EXERCISES
[Note: (1) Individual
work: (G) Group or pair work; (C) Whole-class discussion.)
1. (C) Evaluate the
standards-based assessment movement. What are its advantages
and disadvantages? How
might one compensate for potential disadvantages?
2. (1) Consult the
California English Language Development Test websites listed.
From what you can glean
from that information, how would you evaluate the
CELDT in terms of
content validity, face validity, and authenticity?
3. (1) Consult the
TESOL website on teacher standards (page 109). What would
you say are a few
minimal standards that language teachers should measure
up to? In your own
institutional context, or one that you are familiar with,
how would you assess a
teacher's attainment of those standards?
(G/C) Look at the four
questions posed on page 111 regarding the conse-
quences of standardized
testing. In groups, one question for each group, or as
a whole class, respond
to those questions,
5. (1) Log on to the
website for the National Center for Fair and Open Testing
(see page 111). Report
back to the class on the topics and issues sponsored
by that organization.
6. (C) Explain the
claim that "test-takers are political subjects in a political con-
text" (page 113)
and Shohamy's assertion that large-scale standardized testing
is the "agent of
cultural, social, political, educational, and ideological agendas."
Draw up a list of Dos
and DON'Ts through which teachers might overcome the
potential political
agendas in the use of standardized tests.
Source:
Brown, H. Douglas.
2003. Language Assessment Principles and
Classroom Practices. San Francisco, California
