Wednesday, December 16, 2009

Statistically insignificant

The folks at Olentangy will do anything to spin a story. The latest is this headline from the district listserve: SIX OLENTANGY STUDENTS EARN PERFECT SCORES ON OHIO ACHIEVEMENT TESTS.

That certainly sounds like news. But wait. There is no statistical difference between the state average percentage of perfect scores and the percentage heralded by the district. None.

Some of you will cry foul: The Antipositivist is using statistics to prove a point. What's up with that?

You are right, it is problematic. But let's continue for now.

Students are not randomly distributed among districts. Parents choose school districts, but they do not randomly choose Olentangy or (say) Cleveland City Schools. Therefore, the data around student scores are not probabilistic. So that very same data converted to statistics are suspect.

But, in this instance, the statistics are biased for Olentangy.

We can assume that most of the students earning a perfect score on any test would be found in high demographic districts -- with Olentangy being in the top 10 to 15 districts in Ohio, depending on which demographic value(s) chosen.

So Olentangy has to obtain numbers well above the state average just to be considered average within its peer group.

Since Olentangy is not statistically better than the state average (in spite of its biased draw), one can easily assume that it is below its peer districts in reality.

"'As Ronald Coase says, "If you torture the data long enough it will confess.'" -- Gordon Tullock



Note: The non sequitor above was never addressed. Statistics are valid as a means to understand our world. But statistics as a science never tells the full truth -- it can't. Nevertheless, it can provide insight, especially if the data is probabilistic.

3 comments:

Anonymous said...

Brilliant. Puts it all into proper perspective. OLSD is underachieving in its per capita-income cohort.

Anonymous said...

Where's the article on how many didn't pass? I'll bet it is more than 6.

Anonymous said...

Third Graders? My, oh my. Six Third Graders scored a perfect...out of how many total Third Graders? Are they serious? Well, we're talking about dopey Jeff Brown, so that says it all.

So, some bright kids happened to score impressively above the state of Ohio's confirmed low standards (see NAEP mappings). Good for them, but this is not exactly something for the district to blow it's horn on. Then again, they blow their own horn when the lunch lady wins the "lunch lady of the year" from OASBO. What a joke. More ridiculous is the treasurer winning "treasurer of the year". She should have won the "I can't do a variance walk with a seeing eye dog so I need a budget officer" award.

The whole thing is just one sick, expensive joke. But we're paying for it.

Check out where Ohio stands:
http://www.ed.gov/news/pressreleases/2009/10/10292009.html