Jay Mathews’ annual “challenge” index ranks high schools based on a simple ratio of the number of AP/IB tests to the number of graduates with no regard for the scores.
One side effect of this heavily-publicized “best high schools” ranking (what he calls an “unconventional use of AP”) is that large numbers of unqualified students are being pushed into taking AP classes just to nudge their schools higher on the list.
So, what can you say about a quality index that assigns a high score to a school while only 2% of their students taking the tests on which it’s based actually pass them?
I would say that the method for compiling the index is fundamentally flawed. And I have. Many times.
Mathews, however, isn’t ready to abandon the concept.
Given the emergence of this unconventional use of AP, the Challenge Index has been split this year into two ranked lists, one for schools with college-level-test passing rates of 10 percent or higher, and one for schools with single-digit rates.
Ten percent is the dividing line? A school in which students take large numbers of AP tests but only with a one in ten passing rate could still be considered one of the best in the country?
Anyway, the new ranking of schools with high index numbers but single-digit passing rates will be called the Catching Up list.
Sorry, even with some minor tweaks, Mathews’ “challenge” index is still no way to judge high school quality.
And the AP program is still not a magic wand for school improvement.
Update: In his online Class Struggle column, Mathews offers a long attempt to support the methodology behind his “challenge” index. He doesn’t succeed.