The PISA results for 2015 were released this week. US students were ranked below the average of other countries. And their scores dropped from 2012, the last time the test was administered.
The stories, of course, are largely written by news staff who barely know what PISA stands for (Programme for International Student Assessment) and who have probably only read the executive summary. The stories I’ve read barely bothered to report precisely what the test was assessing (other than math, reading, and science) or none questioned whether it was a valid assessment.
On the other hand, Yong Zhao is someone who understands PISA and and other standardized testing systems, and he asks three questions that should be essential for anyone trying to put these numbers (or those of any standardized test) in context.
Does it accurately measure what matters?
How does it assess skills needed for an uncertain future with such certainty?
Does its results in reading, math, and science accurately capture the domains of expertise each individual needs for successful participation in the future society anywhere in the world?
Although Zhao’s reaction to the PISA results for the Post’s Answer Sheet blog is a little convoluted in trying to tie these test results to the failure of polling to predict the recent elections in the US and UK, he does finally arrive at the cautionary conclusion all those headline writers should have reviewed before pushing publish on their stories.
PISA’s confidence in the predictive power of its assessment also comes from the past. The subjects it chose to assess — science, reading, and math — have long been believed as important for success in life all over the world. They have been the core subject matter schools teach worldwide with the belief that they are essential for living in the modern age. But will these subjects turn out to help today’s 15-year-olds some 10, 20 or 30 years later? Are they the right candidates for all people in the future, or might different individuals need different sets of skills and knowledge?
PISA not only tries to use its test results in these subjects to predict what skills and knowledge 15-year-olds will need to succeed in the future, it also disseminates, based on these scores, education policies and practices it believes will equip children with these skills and knowledge. It has the potential to affect the livelihood of hundreds of millions of children, hence the entire world.
The consequences are serious. The stakes are so high. Therefore, we must question the quality of the PISA results before eagerly jumping to conclusions. Don’t read too much into it.
Whenever someone collects a large pile of data, especially about kids, question the numbers.
Leave a Reply