While the dot com boom may have come and gone, investors these days may want to look at companies that write and administer standardized tests. With school systems falling all over themselves to test students (In Greenville, SC, "some eighth-graders could take as many as six tests this year".*), the testing business is growing fast. However, according to this article in the New York Times, the quality control on those instruments may be lacking.
Beginning in 1999, Ms. Rhoades and Professor Madaus conducted a systematic search for reports of testing errors and found more than 100 in the United States, Britain and Canada from 1976 through 2002, a period that saw extraordinary growth in school testing. One major testing company, for example, had its revenues rise more than tenfold during those years.
The study confirmed the rising number of errors cited in a series of articles in The New York Times in May 2001 . And more errors have been reported since the research for the study was completed, Ms. Rhoades said. All told, of the 103 reported errors and disputes over testing results, more than two-thirds occurred in the past four years. And only a quarter of those were detected by testing companies themselves, she said.
But testing errors are not just the result of the current testing companies expanding their wares too quickly. There’s also the pressure that comes from the customers.
The pressure does not ease when the tests are delivered. States want the tests scored quickly so they can give tests in May and have the results in time for summer school. "But giving a test, getting it right and getting it back in two weeks you’ve just multiplied the odds for mistakes," said Mark Musick, the president of the Southern Regional Education Board.
In the spirit of W’s job creating economy, his Secretary of Education Rod Paige (he of the we-have-no-dropouts-in-Houston "miracle") said "the opportunities created by the law would attract more companies to the testing business" thus relieving the pressure and the number of mistakes.
But industry experts say it is hard for new companies to come in because of shortages of specialized personnel, especially the psychometricians who devise tests and monitor their validity. Moreover, newcomers need an expensive computer infrastructure, and states demand a proven track record.
"You’re not going to be able to go to ‘Joe’s Truck Stop and Testing Service’ and get a test," Mr. Musick said. "You’ve got to go to a major provider that, in spite of its problems, is still respected."
Remember that most of the tests on which errors occured are multiple choice tests, largely scored by machines, which are themselves pretty poor assessments of student learning. However, testing students’ communications skills and their understanding of facts and concepts is more involved. Testing for mastery of knowledge, the kind of testing we should be doing, requires more time, more effort and more money. We seem to be out of all three.
*Thanks to Kim Swygert at Number 2 Pencil for the link.