Science Says! Or Maybe It Doesn’t!

There are many reasons why science reporting in the popular media, especially television, is crap. But one major problem, according to a writer in Vox, comes from the fact that “half of the studies you read about in the news are wrong”.

At least it’s a fact according to a new study. Which means there’s a 50-50 chance that this particular research is wrong?

Breaking news blog

Maybe. But let’s continue anyway.

Yet as researchers in PLOS One recently found, journalists typically only cover those initial papers — and skip over writing about the clarifying meta-reviews that come later on.

What’s more, the study finds, journalists “rarely inform the public when [initial studies] are disconfirmed” — despite the fact around half of the studies journalists write about are later rebutted by follow-up studies.

What’s more, journalists really, really like to report on studies that deliver positive results — even though studies that deliver negative results are equally valuable.

And although journalists gravitate toward covering single studies concerning lifestyle choices such as diet or exercise, these were actually the least likely to be confirmed by a meta-review (as opposed to non-lifestyle papers on topics like genetics).

What do the researchers suggest could be done to improve the quality of the science reporting most people are likely to consume? The article offers one suggestion:

Pick up the phone, and ask researchers whether it is an initial finding, and, if so, they should inform the public that this discovery is still tentative and must be validated by subsequent studies.

Which is likely too complicated for both the journalists, most of whom probably studied very little science in school, and their audience’s attention span.

So, I have another approach to the matter: How about if we teach the process of analyzing scientific reporting to our students in K12? Not just in “science” class but as part of critical reading, media studies, and social studies instruction.

We want students to graduate with a fundamental understanding of the concepts of science. But that understanding should include the necessary skills to intelligently evaluate and question the reporting done on science issues presented in the media they consume. At the very least, they need to learn this very basic fact: “in science, truth takes time”.

Maybe, if we educate a more scientifically literate population, they would demand better quality science reporting.

However, that will also take time.

What Doesn’t Work In Education Reporting

School House1

Rather than asking what works in education, NPR asked “education researcher John Hattie” about ideas that don’t work. His answers are not based on his own work but on a review of “more than 1,000 ‘meta-analyses’”, whatever that is.

Anyway, I wish the writer had started by asking Mr. Hattie what he means by “works”. Of course, I know the answer. Works means improving scores on standardized tests, even though his number two non-working solution is standardized testing.

High-performing schools, and countries, don’t necessarily give more standardized tests than low performers. They often give fewer.

The alternative: testing that emphasizes giving teachers immediate, actionable feedback to improve teaching.

Having said that, his final item, in which he says the US spends too much money on public education, is based on the fact that other countries spend much less while their students score higher than American kids on an international standardized test (the PISA).

He also says that smaller class sizes don’t work because countries like Japan and Korea have relatively large classes and are “high performing”, again on those standardized tests.

It’s too bad NPR just transcribed the executive summary of Hattie’s paper, instead doing some research of their own and asking some informed questions.

The worst part about stories like this is the failure to recognize that there are major differences between American society and our approach to education with those of other countries. Start with the fact that, instead of national education goals, we have 51 educational policies, plus around 31,000 local school boards.

Then review the level of public support for public education in the US compared to Finland, Japan, and the rest. How many of their government leaders are working hard to demonize teachers and privatize schools?

Finally, look at the societal support systems for children in each country, especially rankings of child poverty rates (anywhere from 20% to 33% of all children, depending on the definition of “poverty”). I’ll bet “making America great again” has nothing to do with improving those dismal statistics.

And anyone who says poverty has nothing to do with learning, has never tried to teach math to a class of hungry middle school students.

Pay to Play

Speaking of the Post Magazine (as in the previous post), this week’s education edition also featured a profile of a local private school. A long, glowing story about an expensive institution for gifted students and their success with those kids.

On the previous page we find an expensive, nearly full page ad for the same school.

My wife, who works with arts organizations in the DC area, is always trying to get the Post to cover their activities. Often the groups also buy ads in the same section of the Post as the story. She calls it “pay to play”.

But I’m sure in this case, it’s just a coincidence.

Some Things Never Change

Don’t underestimate the value of doing nothing, of just going along, listening to all the things you can’t hear, and not bothering. – Winnie the Pooh

Ok, it’s been a week since I became unemployed and, while I haven’t quite been doing nothing, it’s been an interesting experience knowing that I could have done just that with no repercussions.

Along with plenty of reading and reflection, I’ve been cleaning out lots of the cruft, physical and digital, that has accumulated over many years. Including a few bags of stuff brought home from the former cubicle. No matter the origins, it’s a process that’s often quite cathartic.

One of the odder items that came out of the bottom of an office desk drawer was a folder containing magazine and newspaper articles from my college days. Stories featuring headlines like “What’s Wrong With Our Teachers?” and “Saving our Schools”.

The articles behind those inflammatory headlines called the quality of teaching “woefully inadequate”, related how kids didn’t work hard enough (or were not required to work harder), and declared that their learning was lacking compared to students in other countries (Japan being the big baddy at that time). Accompanied by one or two cherry picked examples of where schools are “working” in the US.

These declarations of a failing American education system were based on the now legendary A Nation at Risk report, which in its executive summary made this provocative claim:

If an unfriendly foreign power had attempted to impose on America the mediocre educational performance that exists today, we might well have viewed it as an act of war.

I remember that hyperbolic line being held over the heads of teachers for many years after that.

Anyway, A Nation at Risk was written by a federal commission, largely populated by people in higher education and corporate executives, and based on reports of poor student achievement from colleges and SAT scores.

As opposed to any number of negative reports about our current education system today, written by think tanks funded by billionaires with no education experience and based on a flood of data from largely meaningless standardized test scores.

It’s a little depressing. The impression of public education and the national discussion of school reform hasn’t really changed in thirty years. Except that back then one of the major solutions was to provide better pay and support for teachers. You won’t find many “reformers” supporting that idea today.

One more nostalgically fun item from the publications for edtech fans: the full-page ads by Apple, IBM, and Texas Instruments (complete with Bill Cosby) pushing their machines for home learning.

Some Serious Navel Gazing

By my count, one that is likely off a little bit, the Washington Post paper edition published more than 40 different stories about their own sale between Wednesday, the day after it was announced, and today. I have no idea how many variations of them appeared on the website.

That total includes feature stories (at least one on the front page on four of six days), specialized writing in the Style and Business sections1, multiple profiles of the new owner, regular2 and guest columns, and opinion page pieces. One at the top of the Sunday main page gave a byline to six different writers so they are devoting many people to this effort.

I understand this is a major local story, and potentially a significant shift for the business of journalism in general, but at what point does news reporting cross over into obsession3?

1 Never thought to check the Sports section or the real estate listings.

2 George Will has been writing in the Post for 40 years??? That’s one starting point for the new owner.

3 Fox counts as obsession that never began with news reporting.