Study Questions Purported Academic Gains in Texas
A report released Tuesday by the nonpartisan RAND research
organization raises what it calls "serious questions" about
the purported math and reading gains made in Texas in recent years.
RAND researchers examined the results on the Texas Assessment
of Academic Skills (TAAS), the highest-profile state testing program.
It's report, titled "What Do Test Scores in Texas Tell Us?",
questions the validity of those gains.
The report also cautions about the danger of making
decisions to sanction or reward students, teachers and schools on the
basis of test scores that may be inflated or misleading.
It also suggests some steps that states can take to
increase the likelihood that their test results merit public confidence
and provide a sound basis for educational policy.
To investigate whether the dramatic math and reading
gains on the TAAS represent actual academic progress, the researchers
compared these gains to score changes in Texas on another test, the National
Assessment of Educational Progress (NAEP).
The NAEP tests were used as a benchmark because they
reflect standards endorsed by a national panel of experts, they are not
subject to pressures to boost scores, and they are generally considered
the nation's single best indicator of student achievement.
Both the TAAS and the NAEP tests were administered to
4th and 8th graders during comparable four-year periods.
The RAND team – Stephen P. Klein, Laura Hamilton,
Daniel McCaffrey and Brian M. Stecher – generally found only small
increases, similar to those observed nationwide, in the Texas NAEP scores.
Meanwhile, the TAAS scores were soaring. Texas students
did improve significantly more on a 4th-grade NAEP math test than their
counterparts nationally. But again, the size of this gain was smaller
than their gains on TAAS and was not present on the 8th-grade math test.
The "stark differences" between the stories told by
NAEP and TAAS are especially striking when it comes to the gap in average
scores between whites and students of color.
According to the NAEP results, that gap in Texas is
not only very large but increasing slightly. According to TAAS scores,
the gap is much smaller and decreasing greatly.
"We do not know the source of these differences," the
researchers state. But one reasonable explanation, consistent with survey
and observation data, is that "many schools are devoting a great deal
of class time to highly specific TAAS preparation."
While this preparation may improve TAAS scores, it may
not help students develop necessary reading and math skills.
The authors suspect that "schools with relatively large
percentages of minority and poor students may be doing this more than
other schools."
Other features of the TAAS also may contribute to the
false sense that the racial gaps are closing.
Problems with statewide tests are not confined to the
TAAS or Texas, the authors observe.
To lessen the likelihood of invalid scores on such tests,
they recommend that states:
- Reduce the pressure associated with high-stakes testing by using
one set of measures for decisions about individual students and another
set for teachers and schools.
- Replace traditional paper-and-pencil multiple choice exams with
computer-based tests that are delivered over the Internet and draw
on banks of thousands of questions.
- Periodically conduct audit testing to validate score gains.
- Examine the positive and negative effects of the testing programs
on curriculum and instruction.
The RAND study