August 16, 2013
by Michael J. Petrilli
Executive Vice President of the Thomas B. Fordham Institute
Let’s invent a game; it’s called “Rate This School!”
Start with some facts. Our school–let’s call it Jefferson–serves a high-poverty population of middle and high school students. Eighty-nine percent of them are eligible for a free or reduced-price lunch; 100 percent are African American or Hispanic. And on the most recent state assessment, less than a third of its students were proficient in reading or math. In some grades, fewer than 10 percent were proficient as gauged by current state standards.
That school deserves a big ole F, right?
Now let me give you a little more information. According to a rigorous Harvard evaluation, every year Jefferson students gain two and a half times as much in math and five times as much in English as the average school in New York City’s relatively high-performing charter sector. Its gains over time are on par or better than those of uber-high performing charters like KIPP Lynn and Geoffrey Canada’s Promise Academy.
Jefferson is so successful, the Harvard researchers conclude, because it has “more instructional time, a relentless focus on academic achievement, and more parent outreach” than other schools.
Now how would you rate this school? How about an A?
***
My little thought experiment makes an obvious point, one that isn’t particularly novel: Proficiency rates are terrible measures of school effectiveness. As any graduate student will tell you, those rates mostly reflect a school’s demographics. What is more telling, in terms of the impact of a school on its students’ achievement and life chances, is how much growth the school helps its charges make over the course of a school year–what accountability-guru Rich Wenning aptly calls students’ “velocity.” This is doubly so in the Common Core era, as states (like New York) move to raise the bar and ask students to show their stuff against a college- and career-readiness standard.
To be sure, proficiency rates should be reported publicly, and parents should be told whether their children are on track for college or a well-paying career. (That’s one of the great benefits of a high standard like the Common Core.) But using these rates to evaluate schools will end up mislabeling many as failures that might in fact be doing incredible work at helping their students make progress over time.
Let’s go back to Jefferson. As a middle school, it welcomes children who enter several grade levels behind. Even if these students make incredible gains in their sixth-, seventh-, and eighth-grade years, they still won’t be at grade level, much less “proficient,” when they sit for the state test. Furthermore, unless the state gives an assessment that is sensitive enough to detect progress–ideally a computerized adaptive instrument that allows for “out of grade level” testing–it might not give Jefferson the credit for all the progress its students are making.
Here’s the rub: There are thousands of Jeffersons out there: Schools with low proficiency rates but strong growth scores. (See figure, borrowed from this Shanker Blog post, and notice in particular the many schools whose “growth percentile” is above 50 but whose percent proficient is below 50.)
This is particularly the case with middle schools and high schools, serving as they do students who might be four or five grade levels behind when they enter. Is it any surprise that middle schools and high schools are significantly more likely to be subject to interventions via the federal School Improvement Grants program? They are being punished for serving students who are coming to them way, way below grade level.