Looking for a Valid Test

The big supporters of school choice and charter schools always seem to have a lot of passion to back up their ideas.

What they lack is quality research to demonstrate these concepts actually work. (Isn’t that a fundamental requirement of NCLB?)

A national panel recently looked at the growing volume of research on student performance in charter schools and found that “the actual quality of that research doesn’t seem to be improving”.

Most of the studies reviewed were judged to have used fair or poor research designs. Most simply studied whether students in charters were learning more or less than they would have in “regular public schools”.

So, what’s wrong with that?

Of a range of approaches used to study charter achievement, among the strongest is a method that can approximate the conditions of a randomized experiment by comparing students who won a lottery for charter school seats with those who lost out in the lottery, the paper says.

One major limitation of that approach, though, is that it only works for charter schools that are overenrolled and fill seats via lotteries, the panel notes. That raises questions about how representative charter schools in such studies may be.

It said the weakest studies use such methods as comparing average test scores in charter versus noncharter schools using just one year’s test results.

Bringing up all this mediocre research is not meant to be a criticism of charter schools. I really like the concept – allowing experimentation with some controls based on public funding – and there are a small percentage that are doing a great job.

However, before diverting large chunks of public money for these schools, we need to set up a valid system to evaluate them. Something better than we’re using to assess the kids themselves.

charter schools, assessment, research