Sunday, September 24, 2006

Value-Added Analysis

You are going to here a lot about value-added analysis as an indicator of academic performance in the coming year as the state begins rolling out value-added as a new accountability indicator. Be careful.

Value-added is purported to show whether or not a child has achieved a year's worth of academic growth during the past school year. The year-for-a-year will become the new standard by which student achievement, as well as district, school, and teacher performance, is judged.

There are problems with this standard. It is important to understand that value-added only shows whether or not a student has remained in the same place on the bell curve from year to year. [1] In addition, value-added only compares the current level of student achievement against past levels of grouped, similar-situated student achievement.

If we agree that the education system needs to improve, basing the standard on a string of historic test scores will not drive improvements. To satisfy that standard, the school system only needs to continue producing the same product as has been historically produced. In other words, Detroit in the 80's and 90's could have satisfied a value-added standard by simply producing the same cars that it had produced in the 60's and 70's. Value-added would not have set the standard at the level of newer model Japanese cars.

A concept of total achievement is a much more valid standard. Instead of evaluating student progress against historical aggregates, total achievement looks at student performance against a measure set by students and their parents. The student who desires to attend college needs to graduate with skills that open college doors. Having met year-for-a-year growth, then subsequently receiving letters of non-acceptance, or letters of qualified acceptance (where acceptance is based on taking remedial college course-work), would constitute a failure of the system. The year-for-a-year does not guarantee a quality outcome – only an average outcome based on the historic academic achievement of similarly-situated students.

So, for example, if your child struggles for a few years, value-added will place you child in a path with other students who struggled. In this situation, a lower-level of achievement becomes the acceptable standard; acceptable to the school system, but not to the parent.

Keep in mind that a year-for-a-year is not what is appears to be. I have yet to see a source that details a year's academic growth as a normative standard and not a relative ones; a normative standard described by a rubric that defines in exact terms what a year's growth looks like.

In other words, a year's worth of growth is defined as x where x has no relationship to other similarly-situated students' historical pattern of test results; a rubric that answers the question, "What is the expected level of achievement, in absolute terms, of a given child at the end of fourth grade." Without such definitions, value-added is an empty concept when it is used as a standard of individual student achievement.

A look at total achievement: The fact that ACT is stating that the greater majority of students taking the ACT are not ready for college-level work shows that absolute levels of achievement are not being met - according on ACT's informed standards.

Basing a year's worth of growth on historic levels of achievement will never solve the education problem. In addition, telling parents that, "We did our job by advancing your child at the same pace as similarly-situated students (a year for a year), so your child's inability to get into college is not our problem," has a hollow ring. In this instance, the total achievement goal set by parents and child was to be college-ready upon graduation - a higher standard than a simple year's worth of growth.

I suspect most parents want more with regard to their child's education than a simple progression equal to the historic test patterns of similarly-situated children.

That's what value-added isn’t; now for what value-added is. Value-added is a statistical means to evaluate programs and teacher effectiveness; a tool that takes into account all attributes of a child so that no one can claim that a program or teacher's class achieved low-level of results due to the students being the product of poor demographics, or that simply meeting state proficiency levels is sufficient for high socio-economics districts.

Value-added is not the ends, it is the means to improve the system. Programs or teachers that cannot keep pace with average growth should, respectively, be discontinued or shown the door.

The solution must be growth patterns that continually exceed historic averages across the board. Anything less is an insult to children, and their parents.

Notes:

[1] This is a very simple description of a complex system, though it is accurate in a simplistic way.

No comments: