Our Information Stinks

As a guest writer in the Post’s Answer Sheet blog points out, a good deal of the debate over education reform in the past two decades has centered around two concepts: choice and accountability.

Choice, of course, usually comes back to charter schools and vouchers, and accountability may as well be a synonym for standardized test since almost no other ideas of what it means to assess student learning seems to be considered.

Neither has done much to improve American education and probably have done a great deal of harm by narrowing the discussion of what public schools are and should be. But why have choice and accountability not lived up to their claimed potential?

Critics have a whole host of explanations, some of which are quite compelling, and some of which are burdened by political agendas. But the simplest answer, which also happens to be true, is that both movements are dependent on good information about school quality. And, frankly, our information stinks.

Both of these models, of course, are dependent on accurate information about school quality.  Whether parents have the power or accountability officers do, the central assumption is the same: that we can measure school quality precisely enough to make high-stakes decisions.

As the writer correctly points out “standardized test scores provide a very narrow picture of what happens inside schools”. As for charter schools and most private schools, they aren’t doing much if anything different from the public schools. They are working with a selected group of students whose parents are very motivated.

He concludes with a list of five criteria for rating schools that, while certainly not perfect, would be a much better alternative to test scores.

I especially love number one, how much time do students spend on art, music and other creative activities?, and number 5, which asks how well did the education they received help students five to ten years later.

However, back here in our real world, this is the unfortunate bottom line of our current education policy in this country.

Test scores, as many parents and policymakers already know, are misleading.  But they aren’t going away.  They aren’t going away in state or federal decision-making.  And they aren’t going away in the role they play in parental decisions about school choice.  In fact, the opposite is happening: test scores are insidiously taking hold in policy discourse and among the public as a perfectly acceptable measure of quality.  They aren’t.  And, as such, it is our job not only to resist narrow and simplistic measures of educational quality, but to demand access to the data we really need—information that allows us to make thoughtful decisions about our schools.

Thoughtful decisions about our schools. Wouldn’t that be a nice change?

A Mediocre Debate

The Post’s Answer Sheet blog today has a short piece concerning a recent live chat at the paper dealing with education issues, featuring three state governors.  At one point during an interview, the moderator asks them “how America became “so mediocre” in regard to educational outcomes”

The governor of Mississippi1 started by blaming parents. Specifically he said that our troubles began when both parents started working outside the home: “And the mom is in the work place.”.

Ok, that makes for a nice, sensationalistic headline that draws traffic, links, and comments (almost 2000 the last time I looked).

However, the real story here is not that yet another regressive politician yearns to return to a black and white, Ozzie and Harriet2 view of American life that really only existed on TV screens.

This is the state of discussion these days.

A reporter declares the American education system to be universally mediocre (or broken or failing or [insert negative phase here]) without ever explaining the claim, everyone nods in agreement, and a panel of politicians proceed to generate sound bites blaming teachers, tests, Finland, video games, moms in the work place – basically anything other than than their own antiquated policies.


1 Certainly a shining center of academic excellence in this country…

2 Look it up, kiddies.

Lecturing in High Def

At the risk of being declared obsessive, I have one more rant about the educational philosophy of Bill Gates based on the Fast Company interview with him. This time, his vision of that classic instructional tool: the lecture.

That’s one more goal: to revolutionize the lecture in terms of cost and quality. The idea that you can store video essentially for free should mean that anyone can watch the best lecturers in the world. Rather than a student getting one of 3,000 people across the country who try to teach beginning physics or statistics or remedial math, through a process of comparison, competition, and improvement, you get someone who is pretty special and has the budget to do something fantastic. Lectures should go from being like the family singing around the piano to high-quality concerts.

Transform lecturing by giving them better production values.

The high-budget video of a concert may look and sound better, but you’ll probably learn more about music by actually singing with your family around the piano. I’m betting Gates didn’t learn computer programming by watching someone talk about.

Howl About These Numbers Instead

In the article that triggered the previous rant, both the writer and the subject, Bill Gates, make reference to the frequent howl of politicians and corporate types, that students in US schools have fallen far behind their counterparts in other countries. The line has been repeated so many times that it has become accepted as fact.

Except Alfie Kohn has some evidence-based arguments to use in response to those claims that are far more clichéd talking points than truth.

As always, his essay is very good, well worth saving for your next discussion with someone from the all-testing, all-the-time fan club.

However, this is probably the most important point Kohn makes about improving student achievement in the US, no matter how you define that term.

4. Rich American kids do fine; poor American kids don’t. It’s ridiculous to offer a summary statistic for all children at a given grade level in light of the enormous variation in scores within this country. To do so is roughly analogous to proposing an average pollution statistic for the United States that tells us the cleanliness of “American air.” Test scores are largely a function of socioeconomic status. Our wealthier students perform very well when compared to other countries; our poorer students do not. And we have a lot more poor children than do other industrialized nations.

More than 20% of American children are living in poverty, a rate that puts the US 34th out of 35 industrialized countries, the same ones frequently used in test score comparisons.

That ranking should be far more upsetting politicians and corporate types than the numbers generated from largely irrelevant multiple choice tests.

Don’t Worry, Bill Will Be Fine

In their current issue, Fast Company, a business magazine that focuses on technology and design, presents an interview with Bill Gates, in which he “offers his cure for what ails the education system”.

Although the intro section includes a little criticism, it’s pretty clear from the start the writer has no intention of asking anything that might be considered push-back. The editors even include a sidebar with a list of Bill’s “favorite edtech startups”, all of which are more about the technology and data management than they are about learning.

The whole interview isn’t very long and offers none of those “cures” mentioned in the subhead. But Gates’ answers to two questions stood out as especially shallow.

At the top, the writer asks him what he sees as the “ultimate challenge in education”. Gates replies that we must “get more out of $600 billion a year”, the amount he says the US spends on education. Spoken like a true billionaire money manager.

Then towards the end of the article comes this excellent question.

You’ve said that when you were in high school, you followed your own interests, taking on independent study, working on computer programming day and night. Is there room for that kind of student-driven learning in a highly rigorous, metrics-based environment?

Gates’ response is both disingenuous and clueless.

People who are as curious as I am will be fine in any system. For the self-motivated student, these are the golden days. I wish I was growing up now. I envy my son. If he and I are talking about something that we don’t understand, we just watch videos and click on articles, and that feeds our discussion. Unfortunately, the highly curious student is a small percentage of the kids.

As so many education “experts” do, Gates’ is extrapolating his personal experience to every student in the country. But unlike him, I don’t believe the highly curious kids are a small percentage of the whole. There are many more than he can see who are very self-motivated, just not by the narrow goals dictated by a standardized test-driven system.

Let’s face it, we don’t give our students many reasons or resources to express their curiosity and self-motivation during the time they spend with us in the formal process we know as school. Maybe fixing that would be a better way to spend Bill’s money.

Being Annoyingly Skeptical is Required

Although the article Big Surprise: Yet Another Ed Reform Turns Out to be Bogus has been sitting in my Instapaper queue for several months, it is unfortunately part of a larger trend that’s not going away. That would be the increasing tendency of ed reformers to over-inflate the effectiveness of their simplistic ideas.

In this particular example, the San Jose, CA school district has been for many years a “poster child” for claims of incredibly improved academic performance by high school students who were required to take college prep classes.

The reality, however, didn’t even approach the hype: “In 2000, before the college-prep program took effect, 40% of San Jose graduates fulfilled requirements for applying to University of California and Cal State University. In 2011, the number was 40.3%.”

The writer of this piece is not an education writer but nevertheless does a great job of summarizing the state of school reform research.

The number of ed reforms that hold up when the evidence is looked at critically seems to be tiny. The number that continue to work when they’re scaled up seems to be tiny. The number that continue to show results all the way through high school seems to be tiny. The number that can withstand critical scrutiny seems to be tiny. And of the ones that are left, the cost* to keep them up usually appears to be prohibitive.

And he’s certainly correct in the approach he suggests when reading reports of these educational miracles in the future: “I don’t think you can go too far wrong by being almost boundlessly and annoyingly skeptical about this stuff.”


* I think you need to include both monetary and human costs in those calculations.

Evaluating Gates

Earlier this week, the Post gave Bill Gates some prime space on the op-ed page so he could offer “a fairer way to evaluate teachers“.*

And he begins with a really crappy analogy, comparing developing “fantastic teachers” with the way that football teams “identify and nurture” their players.

Completely ignoring that those players, although evaluated as individuals, do not work in isolation. Their success, even that of a star quarterback like Gates’ example, Tom Brady, is very much dependent on the great support of many other people in a well-funded organization.

Teachers, however, in the view of Gates, many politicians, and other education “experts”, are expected to be rated based only on a very narrow measure of their work with absolutely no regard for any other factors.

The remainder of the piece is a messy mix of clichés in which he talks about using “multiple measures” and how “teachers want an environment based on collaboration” without any specifics. Certainly he doesn’t offer anything to convince the reader that he has a clue about the teaching process or how to evaluate it.

Although Gates and his business friends have been accorded a great deal of influence in the discussion over education reform, they do so with no accountability whatsoever.  To correct that situation, Anthony Cody writing at Education Week Teacher proposes The Billionaire Philanthropist Evaluation, noting that the kind of accountability they demand from teachers “is a street that goes one way only”.

Needless to say, Gates does not fare very well, starting with his lack of awareness of the social conditions that impact the learning of many students and extending to his poor understanding of effective instruction.

The bottom line?

Mr. Gates falls below standards in all four of the areas that were observed. His philanthropic activities should be suspended immediately pending his completion of the recommended professional growth activities.

Of course that won’t happen. He has enough money to buy a higher grade.


* Just for good measure, the editors included a video of Gates giving expert opinion on cyber security (which he might know something about considering how buggy Windows was while he was CEO), and… the new pope?