Assorted Stuff

wasting bandwidth since 1999

Tag: skepticism

Questioning Dubious Statistics

February 19, 2018 / 0 Comments

BBC More or Less Postcard

More or Less is a radio programme1 and podcast produced by the BBC World Service. The weekly show tries to make sense of the statistics presented in popular media (including the broadcasts of their own organisation1) in a way the average educated listener can understand.

As you might expect, a common thread in the podcast is whether the numbers reported in stories about studies, polls, and surveys are accurate and used appropriately. Spoiler alert: they often are not.

In a recent “bonus” podcast, the host offers a short debunking guide that would fit on a post card2 from his holiday at the shore. “How to question dubious statistics in just a few short steps.”

The whole thing is worth ten minutes of your time. If you teach math to high school students, you may even want to play it for them.

However, if you’re very short of time, the final step is, for me, the most important idea presented.

Number 6: Be Curious.

If a statistic is worth sharing, isn’t it worth understanding first?

Forget that nagging feeling that says you might just be spoiling a good story. Facts matter… but facts are also fascinating.

Treat them as puzzles. Treat surprising or counterintuitive claims, not with suspicion nor open arms but as mysteries to be solved. It’s fun.

And they close with this reminder.

Hopefully, with this postcard as your guide, you can step into a world of statistical adventure. Because it’s not just about winning arguments, it’s about being curious. The world, after all, is a fascinating place.

Whether you consider statistics “fun” or an “adventure”, the advice is solid. Be curious, some would say skeptical, about the numbers constantly being thrown at you in the news and your social media feed. Very often the story behind them is far more interesting, and different, from what has been presented in the headline.

If you listen to podcasts, More or Less is a good one to add to your playlist. I would have embedded a player here for the episode but the BBC doesn’t allow those of us outside the UK to do that kind of thing.

1. British show, British spelling. :)

2. For you kids out there, postcards were something your parents (maybe grandparents) sent from locations where they were on vacation in the days before Facebook, Instagram, Twitter, and the rest. It was a slower method of trying to impress their friends and relatives. Or maybe make them jealous.

Healthy Skepticism

November 2, 2015 / 0 Comments

I’ve always been very suspicious of research related to human behavior. Especially anything related to kids and learning, which presents observers with far too many variables and unknowns to control for.

However, that lack of certainty doesn’t stop media outlets from reporting each “new” study, poll and survey as the definitive final word. Most often, the writer of those stories only read the excutive summary and knows nothing about any other research that had been done on that particular issue.

It turns out we all have good reason to be skeptical, both of the reporting and the original material.

Recently a group of researchers tried to duplicate the findings from 98 papers published in the top three psychology journals. Two-thirds of the findings could not be replicated.

And this is not a problem confined to psychological research: “30 percent of the most widely cited randomized controlled trials in the world’s highest-quality medical journals have later been found to be wrong or exaggerated and that number rises to five out of six for non-randomized trials”.

Does that mean we should not believe any research? Probably that’s too extreme. But here’s a great piece of advice.

Whenever you hear the words “new study,” alarm bells should ring. It isn’t new studies that you should base your opinions on; it is old studies that have been replicated again and again, and the results reported in meta-analyses and systematic reviews.

Some large scientific issues – evolution, the general age of the earth, climate change, for example – have piled up enough research over the decades and centuries to accept, even if new details are still being discovered).

But when it comes to studying human behavior, and the process of learning in particular, we as educators must be very skeptical of “new” research, and “treat every new claim not as a problem solved, but as an open question”.

Essential Skepticism

February 4, 2013 / 0 Comments

I’ve always considered myself a skeptic, but have been a little fuzzy on how to explain to people what that means. Certainly I have a set of beliefs and convictions but always try to stay open to new information that may cause me to revise them. And I’ve certainly changed my mind about things over the years.

In a new joint project between the Huntington Post and the TED organization1, the editor of Skeptic Magazine2 addresses that question of what skepticism is and what being a skeptic means to him.

Skepticism is not “seek and ye shall find,” but “seek and keep an open mind.” But what does it mean to have an open mind? It is to find the essential balance between orthodoxy and heresy, between a total commitment to the status quo and the blind pursuit of new ideas, between being open-minded enough to accept radical new ideas and so open-minded that your brains fall out. Skepticism is about finding that balance. Here is a definition of skepticism:

Skepticism is the rigorous application of science and reason to test the validity of any and all claims.

He continues on to talk about having a “Baloney Detection Kit”, inspired by Carl Sagan, “which consists of a list of questions to ask when encountering any claim”. It’s a good list but certainly not a new idea.

I first encountered the concept of “crap detecting” in Neil Postman’s wonderful 1969 book Teaching as a Subversive Activity3. And he borrowed the idea from Ernest Hemingway who told an interviewer that “to be a great writer a person must have a built-in, shockproof crap detector.”

Here in the internet age, Howard Rhiengold carries forward the concept and preaches that the skills of crap detecting are even more important as we rely on the web for information.  

However, whether you call it “crap detecting” (my preference), “baloney detecting”, or simply skepticism, I agree with Postman’s view that this is an “essential survival strategy and the essential function of the schools in today’s world”.

At least it should be. The centerpiece of every curriculum should be to help kids understand how to question the world around them, not in a cynical way or simply to be contrary, but to clarify for themselves what is worth their time and what is crap.

1 This association may give the Post a little more credibility but I’m not sure it enhances the reputation of TED.

2 One more publication I didn’t know existed.

3 The book is still in print but you can download the full text from here (pdf).

We Need More Tech Skeptics

July 18, 2010 / 5 Comments

I’ve never liked the whole “digital native/digital immigrant” meme, and an administrator at the University of Kansas seems to agree we need to look at how people understand “technology” in new ways.

She says that many of those digital natives we call students, in both K12 schools and colleges, are actually technologically illiterate, at least under what she says should be an updated definition of “tech literacy”.

The assumption that today’s student are computer-literate because they are “digital natives” is a pernicious one, Zvacek said. “Our students are task-specific tech savvy: they know how to do many things,” she said. “What we need is for them to be tech-skeptical.”

Zvacek was careful to make clear that by tech-skeptical, she did not mean tech-negative. The skepticism she advocates is not a knee-jerk aversion to new technology tools, but rather the critical capacity to glean the implications, and limitations, of technologies as they emerge and become woven into the students’ lives. In a campus environment, that means knowing why not to trust Google to turn up the best sources for a research paper in its top returns, or appreciating the implications of surrendering personal data — including the propensities of one’s bladder — to third parties on the Web.

I think I like the idea of teaching tech “skepticism” instead of “literacy”, for adults as well as kids.

It ties right into helping people develop their crap detector, a concept Neil Postman wrote about in the 70’s and that Howard Rheingold is discussing now.

Thanks to Shaun Johnson for the link.

© 2023 Assorted Stuff

Theme by Anders Noren — Up ↑