Rating schools: A look inside can tell more than reams of statistics

How do you know if your child is attending a really good school?

Here’s a way to find out. Take the school’s attendance rates for the past two years, add them together and divide by two.

Then look at the percentage of students who took state tests last year and put it on scale which indicates that less than 94.5 percent participation means “needs improvement” and more than 94.5 percent means “outstanding.” Then figure an achievement rate.

Do the same for the prior year, add the two together and divide by two. Now do all of this again with math test results. For high schools, calculate the average graduation rate.

Then apply the following rule:

“The Overall School Rating is based on the Achievement Index. If any of the Other Indicators (Attendance, Graduation or Participation) are lower than the Achievement Index, the Overall School Rating is reduced to the lowest rating of the Other Indicators.  However, if the AYP Status is Met, the Overall School Rating cannot be lower than Satisfactory.”

This is the system used by the state of Oregon to rate schools and school districts for their annual report cards, which have been required by the state legislature since 1999. The system has evolved each year. For example, the reference to AYP, which means “adequate yearly progress” as defined by the Federal No Child Left Behind legislation, was added last year. It was needed, because in the past some schools rated as “strong” or “exceptional” on the state system were rated “does not meet” according to federal standards. 

The state now reports data on low income, minority, and disabled students in order to coordinate with the AYP system. 

Under federal guidelines, if one of these subgroups fails to meet the goals, the school is considered as not making adequate yearly progress.

For example, schools in our area have failed to meet AYP based on the performance of disabled students or students who speak English as a second language. This provision is used as an incentive for districts to provide adequate support for special groups of student in order to reduce the “achievement gap” between the general population and minority students.

Changes occur about every year. Performance on writing assessments was included at one point, but the scoring was found to be too inconsistent. Science scores were also intended to be used until budget problems made the scoring and data collection too expensive.

Defining terms in a way that can be consistently measured has been a challenge from the beginning. The original state legislation required that the formula include a measure of behavior. Unable to identify a consistent measurement, the Department of Education used attendance as the indicator of behavior.

To illustrate the power of numbers in the formulas, we can look back three years to North Baker Elementary School. The staff was shocked to see the school’s report card rating drop from strong to “satisfactory.” Analysis showed that a fraction of a percent in attendance level accounted for the difference, even though test scores had actually gone up slightly.

State report cards and federal AYP reports grew out of an understandable desire for accountability. Billions of dollars in taxes go to support education. Ironically, the systems designed for accountability are costing huge amounts of money.

One example was a protracted debate between state and federal officials about using a “growth model.” The state wanted credit for showing progress; the feds just wanted to consider the current scores. After extensive discussion between Washington and Salem (including travel between the two cities), Oregon got permission to use growth in its criteria.

Many of the financial costs come in smaller denominations and are much harder to analyze. For example, every school district spends large amounts of staff time in training, reporting and analyzing data, and reconciling numbers with the state. The advent of online testing made numbers more available, but was expensive and resulted in a protracted legal battle with a testing contractor.

There are other costs besides the financial ones. When the federal government disallowed the adjustment of tests for most disabled students, they were not thinking of the students facing computer screens filled problems they didn’t understand. Teachers witnessed the frustration and tears, but bureaucrats did not.

Perhaps if we want to rate the quality of a school, we should walk in the front door, look around, and ask a few questions.

Our results won’t be perfect or scientific, but they will tell us more than complex mathematical formulas intended to measure students like parts on an assembly line.