Not long ago the headline in the Washington Post (and in their now-defunct publication Newsweek) would claim to provide the ranking of America’s best high schools. These days their list presents merely the “most challenging”.

Other than the slightly altered verbiage, nothing has changed in fifteen years. This is the Post Company’s annual attempt to define school quality in the most simplistic, meaningless way possible using a system created by their education writer emeritus, Jay Mathews.

For those who have missed the annual ritual, here’s how the “challenge” index works:

We take the total number of Advanced Placement, International Baccalaureate and Advanced International Certificate of Education tests given at a school each year and divide by the number of seniors who graduated in May or June.

That’s it. Nothing about how well students actually scored on those tests. Or of how they may have been academically or intellectually challenged at their schools in other ways, using assessments other than standardized tests.

The list completely excludes very challenging schools like the Science Leadership Academy in Philadelphia which is entirely project-based, assessing student learning without these “higher level” standardized tests at all.

As I’ve ranted in this space in previous years, there’s nothing wrong with compiling lists based on arbitrary criteria, and I doubt many would even notice if the Post called this a ranking of the schools most oriented to pushing test-driven college prep programs.

But listing schools only based on their abilities to herd students into particular testing programs is a very narrow, superficial way to define an educational challenge.