Challenging Clickbait

Last week the education RSS feed from the Washington Post was spammed with at least seven stories about Jay Mathews’ “challenge” index. Of course, they were all written by Mathews, who never misses an opportunity to tell you how he created this annual list of the “most challenging” high schools in the US.

So, these posts were not so much news as general clickbait.

In one of the articles, Mathews lets us know that this year is the 30 anniversary of the day this idea first popped into his head. Next year will be the 20th year since the Post and a then paper-based Newsweek magazine first published his list.

And I’ve been ranting about it in this space for almost three-quarters of that time. So I’m not sure what’s left to be said about this simplistic, headline-grabbing, mess. But I’ll say it anyway.

For those not familiar with the “challenge” index, here’s how it works: for each high school that will send him the stats,1 Mathews adds up the number of Advanced Placement (AP) and International Baccalaureate (IB) tests taken and divides it by the number of seniors who graduate. Any school with a score of 1 or higher goes on the list.

What? You were expecting more? Maybe like incorporating the number of students who actually passed the exams? Or other factors that go into making a successful high school beyond pushing kids to take more tests?

Mathews seems to think that his index has improved American education by pushing more schools into adopting the AP curriculum (after complaints a few years back, he grudingly included the IB program). Which assumes that those very limited programs, largely dictated by colleges and framed around the idea that college is the only goal of learning in K12, are appropriate for every student. It also ties nicely into Mathews’ love of charter schools, especially KIPP, many of which tightly embrace AP.

Then there’s the general idea in the public mind that this is a ranked list of the “best” high schools. I know, both Mathews and the Post will say that’s not the intent. They simply want to spotlight the schools that are “working hardest to challenge students from all backgrounds”.

However, that’s not how it works in the real world. Since the start, schools, especially those in the upper levels of the list, local media, public school critics, and others have trumpeted this “challenge” index as THE list of top US high schools.

For the Post, that also helps sell newspapers and magazines, and in the internet age, generates clicks.

A Very Simplistic Challenge

In this morning’s paper, the Washington Post wasted valuable space in the Metro section on Jay Mathew’s annual promotion of the fraud known as his “challenge” index. If you’re not familiar with this artificial ranking of school quality, take a look at some of my regular rants on the subject.

His premise in the column is that schools need to be more transparent, mostly by giving him their data on how many students take AP tests each year. Mathews is pissed that many private schools still don’t want to play along with his index and this piece is little more than him explaining why they are evading the noble purpose of his list.

However, I do agree with Mathews’ overall idea: schools do need to be more transparent. But with more than just numbers. We need to open schools by involving our communities in some basic discussions around what school is and should be.

Is the basic assumption, made by Mathews and many others, that every student should attend college a valid one? If so, is the AP program, created and marketed by a non-profit organization run by the colleges themselves, the best way to prepare them for that goal?

Are AP courses the best way to challenge students in high school, which is the core concept of Mathews’ index? And is publishing a list based only on the number of students who take a standardized test, ignoring completely their scores, a valid way to judge school quality?2

There are many more questions that need to be asked, as well as including other people who are not currently part of the conversation. Like students, who are most impacted by the decisions made by politicians, administrators, and teachers.

Bottom line is that preparing students for their future after high school graduation is a very complex issue. One that requires more options for students than just college. An issue that is far more complex than the simplistic approach promoted by Mathews, a columnist who gave up being a journalist many years ago.

Jay Demands the Data

I haven’t ranted about Jay Mathews’ writing in a while, in part because he’s no longer the primary education columnist for the Washington Post (fortunately, they chose Valerie Strauss to fill that role), but mostly because there’s really nothing new to discuss.

He loves the Advanced Placement program and is responsible for the “challenge” index, possibly the worst measure of high school quality ever created.2 What else is there to know about Mathews? He certainly doesn’t address many other education-related topics in his writing.

Anyway, his most recent column continues this AP romance by chastising a prominent local private school for not releasing to him data on how many students take Advanced Placement tests. So he can flatter them by including the numbers in the next round of the index scores, of course.

And that’s about it. Most of the piece is just Mathews complaining about not being able to include private schools like this one in his lame ranking of mostly public schools.

If he had only accepted the headmaster’s very rational justification for their policy, he would have had nothing to write about.

“We believe that parents and students are not helped by rating systems which purport to evaluate school quality based on test data,” Sidwell head of school Tom Farquhar told me. Independent Education, the local private school association, encourages all of its members to keep their numbers to themselves.

Good for them.

Two other minor points.

First, calling Sidwell Friends “our most famous private high school”2 because the president’s children and those of other politicians attend is a prime example of inside-the-beltway pretentiousness. In keeping with Quaker tradition, Sidwell likes to keep a low profile and I’m not sure they would consider themselves the most famous private school in the DC area, much less the whole US.

Second, the fact that his daughter graduated from Sidwell also lends a bias, and not a small dose of privileged attitude, to this column. In addition to letting us know that the Post must pay him very well since the school has always been one of the most expensive in a very pricy market.

Ok, I’ll now return to ignoring Jay Mathews.

A Very Narrow Definition of Challenging

Not long ago the headline in the Washington Post (and in their now-defunct publication Newsweek) would claim to provide the ranking of America’s best high schools. These days their list presents merely the “most challenging”.

Other than the slightly altered verbiage, nothing has changed in fifteen years. This is the Post Company’s annual attempt to define school quality in the most simplistic, meaningless way possible using a system created by their education writer emeritus, Jay Mathews.

For those who have missed the annual ritual, here’s how the “challenge” index works:

We take the total number of Advanced Placement, International Baccalaureate and Advanced International Certificate of Education tests given at a school each year and divide by the number of seniors who graduated in May or June.

That’s it. Nothing about how well students actually scored on those tests. Or of how they may have been academically or intellectually challenged at their schools in other ways, using assessments other than standardized tests.

The list completely excludes very challenging schools like the Science Leadership Academy in Philadelphia which is entirely project-based, assessing student learning without these “higher level” standardized tests at all.

As I’ve ranted in this space in previous years, there’s nothing wrong with compiling lists based on arbitrary criteria, and I doubt many would even notice if the Post called this a ranking of the schools most oriented to pushing test-driven college prep programs.

But listing schools only based on their abilities to herd students into particular testing programs is a very narrow, superficial way to define an educational challenge.

But Don’t Blame The Index

Ok, I know I said I would quit my largely fruitless ranting about Post education columnist Jay Mathews’ “challenge” index but there’s too much irony in his latest defense of the list to let it pass.

In his blog Mathews responds to an email received from a local parent who is concerned about his child attending a local high school because of it’s low ranking in the index. Mathews starts by admitting “it was my fault” for the parent getting the wrong impression of the school, saying that it has “terrific teachers”.

But that contrition doesn’t last long as he quickly moves on to declare that the problem is not with the index. No way.

The school ranks low on his list because there are so many “challenging” high schools in our area. This particular one just looks bad by comparison to all the others. Not unlike a six year old Mercedes sitting next to a brand new luxury sedan*.

Mathews goes on to devote many more more words of praise for the school and in the process, completely misses the point, not to mention a heavy dose of irony in his justification.

As I and many others have pointed out, judging a high school by one, narrow, extremely simplistic criteria, one based only on how many students take a particular test, is wrong, misleading, and intellectually dishonest.

However, year after year, he and his high profile publisher (the Washington Post Company and their news syndicate) continue to promote this “challenge” index as a valid ranking of high school quality, one superior to all others. And year after year parents, students, school administrators, and the general public, fed by other media companies who reprint the list unquestioned, continue to believe it.

So, yes Jay, you are very much to blame for the confusion your correspondent has about the quality of the high school his child will be attending.

You should also take responsibility for misleading many other people when it comes to the many factors that contribute to a quality high school education, and that have nothing to do with AP.


* His analogy, not mine!