wasting bandwidth since 1999

Fixing the Challenge

Jay Mathews’ annual “challenge” index ranks high schools based on a simple ratio of the number of AP/IB tests to the number of graduates with no regard for the scores.

One side effect of this heavily-publicized “best high schools” ranking (what he calls an “unconventional use of AP”) is that large numbers of unqualified students are being pushed into taking AP classes just to nudge their schools higher on the list.

So, what can you say about a quality index that assigns a high score to a school while only 2% of their students taking the tests on which it’s based actually pass them?

I would say that the method for compiling the index is fundamentally flawed. And I have. Many times.

Mathews, however, isn’t ready to abandon the concept.

Given the emergence of this unconventional use of AP, the Challenge Index has been split this year into two ranked lists, one for schools with college-level-test passing rates of 10 percent or higher, and one for schools with single-digit rates.

Ten percent is the dividing line? A school in which students take large numbers of AP tests but only with a one in ten passing rate could still be considered one of the best in the country?

Anyway, the new ranking of schools with high index numbers but single-digit passing rates will be called the Catching Up list.

Sorry, even with some minor tweaks, Mathews’ “challenge” index is still no way to judge high school quality.

And the AP program is still not a magic wand for school improvement.

Update: In his online Class Struggle column, Mathews offers a long attempt to support the methodology behind his “challenge” index. He doesn’t succeed.


  1. ~Tim

    I HATE that stupid challenge index and am very discouraged at all the attention it gets.

  2. Dave

    You’ve posted this post enough times that I feel like it’s OK to comment again. ;)

    I personally like the Challenge Index because;
    1) I think it helps force schools to offer college-level opportunities to as many students as possible.
    2) I know that the Index doesn’t include pass rate at all, but that doesn’t bother me, because I think that pass rate on a test is the one number that all parents and taxpayers already know to look for. (I don’t think that AP is a magic wand, but I think that education and community oversight of public education has improved to where the community can identify most basic problems with schools. The Challenge Index seems to aim at further improvement by highlighting an additional beneficial goal that much of the community doesn’t yet know to examine.)
    3) Not focusing on pass/fail stats lets students take responsibility for whether they pass or fail, rather than putting additional pressure on schools who would in turn put additional pressure on teachers and students. (That additional pressure isn’t motivational, it’s just stressful)
    4) Using AP tests recognizes a wider range of student abilities. A student may not be strong in math and science, but their ambitions in liberal and fine arts are still recognized.

    I feel like the Challenge Index’s detractors are upset because they don’t think it accurately indicates the best schools. And they’re right, it doesn’t, but I’m not sure that’s possible to do in an efficient or effective way.

    I’d like to see 10 years or so of data and see what really happens at the schools. My hypothesis would be that # of tests increases, and that after a couple years at the increased numbers, the pass rate starts to increase, too. The end result would be that a larger percentage of all high school students are doing at least some college-level work in high school.

    All that said, I’m not too concerned, and I don’t think anyone really cares very much about this outside of the relatively small number of us who read and write edublogs. :)

  3. Jay Mathews

    Thanks for providing the link. It is a long argument, but most people come to this fresh, and it takes some explaining. I would love your thoughts on the argument I make about the 10 percent mark . An inner city school that commits to exposing nearly every student to AP, and gets a 10 percent passing rate, will have about as many students passing AP as an average suburban school, something which will not happen if we accept your view that those inner city kids “are’t qualified” for AP. Except for math and foreign language, all that you need to be qualified for AP are an ability to read and write and a desire to work hard. Many inner city kids have those basic qualifications but are still told they just arent ready for AP., and are given some laughable remedial course instead.

© 2024 Assorted Stuff

Theme by Anders NorenUp ↑