By Christian D’Andrea
MacIver Institute Education Policy Analyst
The Wisconsin Legislative Audit Bureau has completed their review of the state’s School Choice Demonstration Project (SCDP) in order to better understand the impact of vouchers on education in Milwaukee. The results were mixed, though they seemed to validate the positive impact of the Milwaukee Parental Choice Program (MPCP) on the city’s students.
The good news? Children in voucher programs outperform children that remain in Milwaukee’s public schools at certain grades in math and science. The bad news? That a consistent method of comparing growth between students is still tough to come by.
That inconsistency was enough to raise the ire of Senator Chris Larson (D-Milwaukee). Sen. Larson, a longtime opponent of the voucher program, voiced his feelings on the audit last week.
“It’s foolish to continue funneling over $55 million annually out of our children’s public schools without any measure of the value of this educational diversion,” said Sen. Larson. “I hope that this report will motivate the Legislature to re-examine the voucher program.”
Senator Larson is using this audit as a tool through which he’d like to shut down the voucher program. However, he’s ignoring one major point – that the problem here is related to data collection, not the schools themselves. While the audit reported that results were inconsistent, the audit also found positive educational growth for voucher students in later grades. In short, the Senator is calling to stop the voucher program based on an audit that, at worst, shows no negative effects on students. At best, it shows that students in seventh, eighth, and 10th grade are learning more when it comes to reading and math.
The SCDP was mandated by then-Governor Jim Doyle as a method to create high-quality research behind the country’s first modern school choice program. The goal of the program, which was led by distinguished researchers from Wisconsin and across the country, was to examine the educational growth of students. These students were broken down into two groups – students that attended voucher schools and those that remained in public schools.
After five years and plenty of headaches when it came to tracking students, the SCDP finally uncovered a significant benefit in student growth for students that used vouchers to attend private schools in Milwaukee. There was no negative impact for students that remained in public schools.
The Legislative Audit Bureau found similar benefits – though at different grades. However, since these scores coincided with the implementation of state standardized testing in the voucher schools (the WKCE), auditors could not conclude that these gains were attributable to the schools themselves. Other concerns, such as the dynamic movement of students and the difficulty of tracking their results, cast concerns over the validity of the data, despite the positive results in favor of the voucher program.
High-quality data analysis is an issue across the state. The WKCE has been the most common standard for when it comes to measuring student achievement in Wisconsin. However, that data system is flawed when it comes to all but the most basic comparisons between students across the state.
That process becomes even more muddled when students in choice programs are added to the mix. Many of these students who choose better fitting schools lag behind their counterparts when it comes to educational concepts like reading and math. That’s something that we saw most recently in Racine with their blossoming voucher program.
The WKCE, a test that students take in October of every year, measured the achievement of students across Racine in 2011. Students in the choice program lagged behind their public school counterparts by significant margins in literacy, math, and science concepts. Some have pointed to this as early evidence that the program does not work. That is a dangerous assumption.
The students taking these tests had just over a month of private school instruction. In most cases, they had at least three years of public schooling before that. The imprint of the regular classroom is much larger than their time in these voucher schools. This data strongly suggests that the students entering the Racine PPSCP lag behind the students that remain in public school. In short, the students that are taking advantage of this new voucher programs are the ones who are struggling; they are the ones who need a new environment the most.
This example reinforces the need for high-quality studies in order to determine student growth as a result of Wisconsin’s choice programs. There is no apples-to-apples comparison that can be drawn using solely WKCE data. When voucher students start at a lower level than their peers, they can show greater growth than their counterparts but still lag behind on standardized testing. What’s most important isn’t the score itself, but the growth that a student made to achieve that score.
That’s what the SCDP and its Legislative Audit Bureau determined that voucher schools provided at a greater level than regular public schools. Students that attended private institutions through vouchers displayed greater growth than their counterparts that remained in public classrooms for all five years of analysis.
Audit shows that data collection is an issue. But how do you impose data collection regulations without interfering with the process at private schools? How do you avoid turning these unique institutions into bureaucratic quagmires? How do you keep these schools as the dynamic classrooms that are supposed to present a legitimate change for students that have struggled in their traditional neighborhood public schools?
The most important piece of the city’s audit was that the study found positive gains for voucher students. There were no negative effects amongst families and students that took vouchers to attend the school of their choice. There were, however, statistically significant benefits for students in grades seven, eight, and 10 across reading and math.
What does this audit tell us? It primarily points out the difficulty of tracking student growth over a cohort group of students over a five-year span. It also reinforces the need for high-quality longitudinal research in order to better understand the true impact of vouchers in Milwaukee. The SCDP and its audit both found positive gains related to the students that attended private schools with vouchers. They also found that these students were more likely to transfer from school to school in order to find the best fit for them. However, the study and the audit disagree on the scope and source of these impacts.
The School Choice Demonstration Project still remains the best research that we have on the impact of vouchers on students in Milwaukee. As it stands, that project has ended headed into the 2012-2013 school year. The Legislative audit of the project wasn’t a criticism of the MPCP, but of the study itself. In fact, that audit continued to find significant benefits for students that attended voucher schools over the past five years.
Its findings don’t discredit the voucher program – far from it – but instead calls for a stronger study to determine the value of the growth displayed by MPCP students. Legislators should use this evidence as a call for more longitudinal analysis of student achievement between schools in Milwaukee. Without it, the program will be vulnerable to sinking at the hands of partisan squabble and half-truths rather than empirical analysis and statistically significant evidence.