Friday, September 04, 2009

Hiding Behind IP

A recent post of mine over on the Blog at Mises.org:









Hiding Behind IP

Jim Fedako



According to data from the Ohio Department of Education, 82% of Ohio public school students attend school districts that showed academic growth (as defined by the state) during the last school year which was significantly (statistically speaking -- defined as one standard error) above expected (or average) growth.

Add in the students who attended districts within one standard error of the mean and the state can claim that 91% of students attended districts achieving at or above expected growth.

Amazing institutions, these government schools.

So you decide to question the results. Tough luck. The state's value added calculation is protected by IP laws. That's right. The state can make an outrageous claim without being challenged. Nice.

Note:

Here is the FAQ that has to calm the concerned -- value added is just like the CPI. Too complicated for the masses, but valid and reliable, nonetheless. Trust them. They are your government, you know.

From the service that assists Ohio with its value added calculation:

The value-added methodology seems complicated. How can people understand the measure?
While the statistical methodology used for value-added analysis is complex, the data produced are valid, reliable and presented in easy-to-read charts and graphs. Understanding this methodology can be compared to understanding the statistics behind the Consumer Price Index (CPI), which measures the change in the cost of living between two periods of time. Few people understand how to calculate the CPI, but many people take advantage of the information and use it to make decisions in their daily lives. If educators understand the information derived from the value-added reports, they can use the information to make sound decisions about improving student achievement.

4 comments:

Anonymous said...

what are IP laws?

Jim Fedako said...

IP = Intellectual Property.

Copyright, patents, etc.

Anonymous said...

According to my statistical calculations OLSD's new AYP + PI = Wade Lucas's CYA. Put all of these together and you have a SI (Sophistry Index) of 120.

jenkinsbrigade said...

Jim,

The school I teach at has fewer than 300 students. In a population so small, it is a fairly easy for the teachers to observe the general abilities of a class as a whole. Last year, the consensus was that the current group was, to put it mildly, not the sharpest group we'd ever had. Imagine our surprise, then, to learn that last year's test scores were up across the curriculum, showing gains of 10 to 15 percentage points in students identified as either 'advanced' or 'proficient'. Having some experience with designing and scoring standardized tests myself, I immediately thought something was fishy. After all, which is more likely, that a group of modest achievers suddenly showed dramatically improved test scores, or that the test itself was somehow modified (either the test was easier, or the scoring methodology was changed)? Either way, we the teachers are being set up for a fall; some will say that in spite of all the cuts made by the district, we boosted scores, and thus they will justify more cuts. Others will expect similar score increases on the next round of testing. It's rather like an artificially generated 'education bubble'. Can't wait to see what happens when it pops.