Questioning Dubious Statistics

BBC More or Less Postcard

More or Less is a radio programme1 and podcast produced by the BBC World Service. The weekly show tries to make sense of the statistics presented in popular media (including the broadcasts of their own organisation1) in a way the average educated listener can understand.

As you might expect, a common thread in the podcast is whether the numbers reported in stories about studies, polls, and surveys are accurate and used appropriately. Spoiler alert: they often are not.

In a recent “bonus” podcast, the host offers a short debunking guide that would fit on a post card2 from his holiday at the shore. “How to question dubious statistics in just a few short steps.”

The whole thing is worth ten minutes of your time. If you teach math to high school students, you may even want to play it for them.

However, if you’re very short of time, the final step is, for me, the most important idea presented.

Number 6: Be Curious.

If a statistic is worth sharing, isn’t it worth understanding first?

Forget that nagging feeling that says you might just be spoiling a good story. Facts matter… but facts are also fascinating.

Treat them as puzzles. Treat surprising or counterintuitive claims, not with suspicion nor open arms but as mysteries to be solved. It’s fun.

And they close with this reminder.

Hopefully, with this postcard as your guide, you can step into a world of statistical adventure. Because it’s not just about winning arguments, it’s about being curious. The world, after all, is a fascinating place.

Whether you consider statistics “fun” or an “adventure”, the advice is solid. Be curious, some would say skeptical, about the numbers constantly being thrown at you in the news and your social media feed. Very often the story behind them is far more interesting, and different, from what has been presented in the headline.


If you listen to podcasts, More or Less is a good one to add to your playlist. I would have embedded a player here for the episode but the BBC doesn’t allow those of us outside the UK to do that kind of thing.

1. British show, British spelling. :)

2. For you kids out there, postcards were something your parents (maybe grandparents) sent from locations where they were on vacation in the days before Facebook, Instagram, Twitter, and the rest. It was a slower method of trying to impress their friends and relatives. Or maybe make them jealous.

Technology Changes Everything. Or Nothing.

This from a recent segment of the BBC World Service podcast, 50 Things That Made the Modern Economy, caught my attention.

Two economists… published research showing that many companies had invested in computers for little or no reward, but others had reaped big benefits. What explained the difference was whether the companies had been willing to reorganize to take advantage of what computers had to offer.

You couldn’t just take your old systems and add better computers. You needed to do things differently.

The program1 is about how the technology of electricity failed to improved businesses who remained organized around steam, in the same way that computers failed to improve businesses who remained organized around manual practices.

With that in mind, go back to that first paragraph of the pull quote and replace “companies” with “schools”, minus the published research part.

Over the past twenty years or so, many, if not most, schools simply added computers to the old systems. And then wondered why the promised revolution never appeared. It’s still happening today.

The presenter ends the podcast with a few lines to consider the next time you hear or read about some service, app, or system someone claims will “revolutionize” learning.

The thing about a revolutionary technology is that it changes everything. That’s why we call it revolutionary. And changing everything takes time. And imagination. And courage. And, sometimes, just a lot of hard work.

Instead of just accepting the statement as fact, dig deeper and look for that imagination, courage, and hard work required to produce genuine change.

What’s the Point of School?

In an opinion piece for the BBC, the Home Editor asks that question, one that we really need to discuss here in the US.

Even though our education system is designed and assessed upon its ability to get lots of children through state exams, very few people seriously argue that the fundamental point of schools is ensuring pupils pass tests.

Not sure he’s right about the “very few people” part.

We ascribe to schools a loftier ambition than academic success alone. We want them to prepare our children for adulthood and provide them with the skills to have the most fruitful and fulfilling life possible. Don’t we?

We would like to think so. A report from an “all-party parliamentary group” seems to reinforce the idea of students learning more than just “history and maths”.

There is, actually, a surprising amount of agreement on these ideas. Progressive educationalists tend to call it “emotional intelligence” or “emotional health”, while conservatives prefer words like “character” and “backbone”, but it amounts to pretty much the same thing.

Here in the US, you could probably get the same agreement, although only in terms of language and phrasing. When it comes to execution, we still have far too many “leaders” who advocate for schools that emphasize pupils passing tests ahead of those “loftier” ambitions.

Of course, that seemingly high-minded report was written by politicians, which means there’s going to be a point where it heads way off the rails.

The APPG report recognises the weakness in the UK evidence base but points to successful initiatives in Singapore and the United States. The American “Knowledge is Power Program’ (KIPP) is cited as a model of what is possible.

And, as far as I’m concerned, anyone who uses the terms “grit” and/or “rigor” in support of their ideas about what education should be, immediately loses all credibility.

Ok, so maybe this story is not a good starting point for a discussion of the purpose of public education.2 But that question is one we very much need to answer as a society.

Our Website is Broken and We Apologize

I always keep some news feeds from the BBC in my RSS aggregator, both because they offer a non-American slant on the news and because of the occasional odd story that is just so very British.

From a recent example of the later comes the revelation that “The BBC Trust has upheld a complaint that the clock on the BBC homepage was “inaccurate and misleading”.”

The BBC Trust is the governing body that oversees the operations of the British Broadcasting Corporation and is supposed to represent the interests of the people who pay licensing fees to own a television set, aka the viewers.

In this case one of those viewers was upset that “although readers assume the clock is correct, it merely reproduces the time on the user’s computer” and thus may not be accurate if the computer itself has the wrong time.

The Trust ruled in the complainant’s favor saying that “having a clock which does not state it derives its time from a user’s computer is not consistent with BBC guidelines on accuracy”, and that the clock will be removed in the next update.

And the BBC management offers a very British apology, probably not unlike this wonderful example from John Cleese…

New Assessment Idea: Trust The Teachers

The BBC’s education writer Mike Baker thinks he sees a “long-term change of direction” in the UK’s love affair with “accountability” (aka standardized testing).

The latest sign was this week’s report from the Commons Schools Committee. It delivered a message we don’t often hear from politicians: trust the teachers.

The MPs [Members of Parliament] argued that the “complexity” of the school accountability system in England is creating “a barrier to genuine school improvement”.

The report highlighted the “adverse effects” that often flow from a target-driven school culture and criticised Ofsted [Office for Standards in Education] for taking a narrow, results-based view of learning in schools.

A report from a government “Expert Group” suggests that improving the reliability of teacher assessments would allow the country’s schools to move away from their reliance of “externally marked tests”.

The report did not argue for an end to all external assessment. But it called for a shift toward more within-school, teacher-led assessment. This, it said, would not only save money but also a lot of the teaching time that is lost to exam preparation and administration.

And this is the key point: it is not about dropping school accountability altogether, but about making sure it does not obstruct teaching and learning.

Baker refers to the UK testing program as a “crude” way to assess learning, one that has had “unintended effects” such as narrowing the curriculum.

It’s hard to tell from this one article whether or not any of these trends away from standardized testing are actually going to take root in British schools (much less their political institutions).

However, it’s all stuff we also need to be considering on this side of the pond.