Since it’s December, it must be time for Jay Mathews to unleash this year’s edition of his educational triviality known as the “challenge” index of high schools in the DC area. (The rest of you will get yours in Newsweek sometime in the spring.)
It’s also a good time to remind everyone of just how meaningless this over-hyped ranking really is.
The process Mathews uses to come up with the list is pretty simplistic: “Divide the number of Advanced Placement, International Baccalaureate or Cambridge tests a school gave by the number of seniors who graduated in June”.
Of course, nothing about how students actually performed on those tests is factored in, although there is something new this year related to that.
The listing now includes a number called the Equity and Excellence rate, defined as “the percentage of all seniors who have had at least one score on an AP, IB or Cambridge test that would qualify them for college credit”.
If that statistic has any connection to the “challenge” index ranking, it’s not at all obvious.
The bottom line is that there is far more that goes into the quality of a high school than just how many kids take these college prep tests. None of that will be mentioned in the press that’s generated by this farce.
The article tries to address that issue by claiming that the index is “not a measurement of the overall quality of the school but illuminates one factor that many educators consider important”.
However, that essential disclaimer never seems to make the headline.
Plus Mathews seems to be the only person I’ve ever read who makes the claim that this is an important factor to “many educators”. And he’s not an educator.
Thank you for writing about this again. I hope you continue to do so.
I will keep holding out hope that the Post will someday find someone with an education background to write about education.
A brief response in my own defense: I begin every speech reminding listeners that I have never been a teacher, and would last no more than two days in that hard job. What I have been doing the last 25 years is listening to teachers, hundreds of them, as they explained to me what actually works, and doesn’t work, in classrooms. In every case I have shown those stories, and the three books I have done about challenge in high schools, to those teachers before publication to make sure every word is true. They are the ones who gave me the idea for the Challenge Index, a way to help us break away from the usual way we rate schools, by test scores, which is the same thing as rating schools by how rich the students’ parents are. The index is far from perfect, but it is better than any of the other ways we have tried to quantify high school quality. There are lots of other things that make a good high school beyond participation in challenging courses and tests, but sadly they cannot be easily measured so that befuddled parents can figure out which school might be best for their children. The Challenge Index is such a clear measure, and thus a useful tool, as many parents, as well as many of those teachers who have educated me, have said.
I’ve seen some of Mathews’ discussion of the Index, such as
http://www.washingtonpost.com/wp-dyn/content/article/2007/06/22/AR2007062201435.html
and I think the Challenge Index works well. If nothing else, it’s the worst measure…except for everything else.
Schools that are genuinely working for ways to help their students will see that 1) college-level coursework is almost universally beneficial to offer 2) college-level coursework is a very realistic goal for all but the most extreme schools.
There will always be people who use tools like the Index improperly – parents who insist that their child go to a top 25 school, etc. The magazine will always play the Index up a little because that’s just how media works. I can’t fault a useful tool just because some people don’t use it in conjunction with all the other available information.
I got an overly-excellent scholarship because I happened to contribute towards a number that my university was trying to improve in order to raise the US News/World Report ranking. The Challenge Index’s simplicity helps a little to keep that kind of silly recruiting from happening.