I’ve always been very suspicious of research related to human behavior. Especially anything related to kids and learning, which presents observers with far too many variables and unknowns to control for.

However, that lack of certainty doesn’t stop media outlets from reporting each “new” study, poll and survey as the definitive final word. Most often, the writer of those stories only read the excutive summary and knows nothing about any other research that had been done on that particular issue.

It turns out we all have good reason to be skeptical, both of the reporting and the original material.

Recently a group of researchers tried to duplicate the findings from 98 papers published in the top three psychology journals. Two-thirds of the findings could not be replicated.

And this is not a problem confined to psychological research: “30 percent of the most widely cited randomized controlled trials in the world’s highest-quality medical journals have later been found to be wrong or exaggerated and that number rises to five out of six for non-randomized trials”.

Does that mean we should not believe any research? Probably that’s too extreme. But here’s a great piece of advice.

Whenever you hear the words “new study,” alarm bells should ring. It isn’t new studies that you should base your opinions on; it is old studies that have been replicated again and again, and the results reported in meta-analyses and systematic reviews.

Some large scientific issues – evolution, the general age of the earth, climate change, for example – have piled up enough research over the decades and centuries to accept, even if new details are still being discovered).

But when it comes to studying human behavior, and the process of learning in particular, we as educators must be very skeptical of “new” research, and “treat every new claim not as a problem solved, but as an open question”.