wasting bandwidth since 1999

Tag: vam

US Education Policy is Screwed Up

To back that blanket statement, consider these two pieces of evidence found on the interwebs just today.

First we have a new report showing that “fraudulent charter operators in 15 states are responsible for losing, misusing or wasting over $100 million in taxpayer money”.

We found, as stated in the introduction, that at least $100 million in public tax dollars has been lost due to fraud, waste, and abuse. These instances of fraud and mismanagement, which are catalogued in appendixes A-F, fall into six basic categories:

  • Charter operators using public funds illegally for personal gain;
  • School revenue used to illegally support other charter operator businesses;
  • Mismanagement that puts children in actual or potential danger;
  • Charters illegally requesting public dollars for services not provided;
  • Charter operators illegally inflating enrollment to boost revenues; and,
  • Charter operators mismanaging public funds and schools.

Charter schools, of course, are the cornerstone of many school reform plans, including those from the current federal administration, many states, and billionaire education “experts” like the Gates and Walton Foundations.

This report is on top of other studies showing that student learning at charters is no better, and often worse, than the districts from which they are taking students and money. And, contrary to the claims of supporters that the competition provided by charters will improve schools for all kids, are actually undermining public education.

With all this data to review, what’s a politician to do? Why, push for more charters, of course.

A bipartisan group of senators plans to introduce a bill Wednesday meant to encourage the growth of charter schools across the country, mirroring legislation expected to be taken up in the House later this week.

Building on the “success of charter schools”, according to one Senator.

Congress critters must not use the same set of web-based tubes that I do.

Then there’s the matter of how to assess the quality of teachers.

Many states and districts have begun using some variation on the Value-Add Model, about which I’ve ranted recently, that assesses teachers based in part on the increase in student learning (aka standardized test scores) over time. In other words, how much value did the teacher add.

Some teachers in Florida objected to the system used in that state in which they would be “evaluated on the scores of students they haven’t taught and on subjects they don’t teach”. Seems like an unfair process, one that might even violate “the Equal Protection and Due Process Clause of the Constitution”. And a District Judge agreed that the process is ridiculous.

However, he also said it was legal.

“This case, however, is not about the fairness of the evaluation system,” Walker wrote. “The standard of review is not whether the evaluation policies are good or bad, wise or unwise; but whether the evaluation policies are rational within the meaning of the law. The legal standard for invalidating legislative acts on substantive due process and equal protection grounds looks only to whether there is a conceivable rational basis to support them,” even though this basis might be “unsupported by evidence or empirical data.”

Obviously I’m not qualified to be a judge because I cannot see how an unfair system can have a “conceivable rational basis” to support it.

Anyway, bottom line, the political posturing represented by these examples, not to mention the tens of millions wasted, does nothing to help students get a better education.


Statistical Crap

Speaking of data (as in the previous post), the members of the American Statistical Association (ASA) probably know something about that topic. And they recently released a statement about the Value-Added model (VAM) of teacher evaluation, a very popular reform among those top-level education experts we all love and respect.1

In theory, VAM is supposed to measure how much value a teacher has added to the learning of their students, using standardized test scores and some complex mathematics that is supposed to exclude other “non-relevant” factors from the final numbers. Many districts and states (including Virginia, to a small degree) are using or planning to use some variation of VAM as a teacher evaluation tool and to determine continued employment, pay raises, tenure, even whether to close schools.

The ASA statement is, as you might expect, very academic in it’s assessment of VAM but they are still quite clear in their conclusions that this system is… how shall we put this? – statistical crap (my very non-academic interpretation of their 7 page report).

A few very relevant statements from the executive summary.

VAMs are complex statistical models, and high-level statistical expertise is needed to develop the models and interpret their results.

Expertise that is quite lacking in most schools, not to mention in pronouncements from supporters of this concept.2

Estimates from VAMs should always be accompanied by measures of precision and a discussion of the assumptions and possible limitations of the model. These limitations are particularly relevant if VAMs are used for high-stakes purposes.

Too many advocates of VAM consider the numbers as fact, not “estimates”, and are not open to any “possible limitations”.

And my favorites,

VAMs typically measure correlation, not causation: Effects — positive or negative — attributed to a teacher may actually be caused by other factors that are not captured in the model.

Under some conditions, VAM scores and rankings can change substantially when a different model or test is used, and a thorough analysis should be undertaken to evaluate the sensitivity of estimates to different models.

Most VAM studies find that teachers account for about 1% to 14% of the variability in test scores, and that the majority of opportunities for quality improvement are found in the system-level conditions. [my emphasis]

In other words, teacher quality, while important, is only one factor to consider in the very complex process of student learning, and the far-less-than-perfect method of assessing that learning, standardized test scores.

That “majority of opportunities for quality improvement” will only come from making systemic changes to educational policies at the district, state, and national levels.

© 2022 Assorted Stuff

Theme by Anders NorenUp ↑