April 5, 2013
by Christian D’Andrea
MacIver Institute Education Policy Analyst
A recent analysis by a Madison think tank is trying to poke holes in the six-year work of the School Choice Demonstration Project (SCDP). The true discovery here, however, is that this report from the Forward Institute seems to be more interested in discrediting the SCDP’s results than providing meaningful statistical analysis on the data or the Milwaukee Parental Choice Program itself. Even in that aspect, it falls short thanks to a limited view of the project’s six years of analysis in Wisconsin’s largest district.
According to the Forward Institute, the SCDP fails to provide compelling data that voucher schools are the underlying influence behind greater graduation and college attendance rates for students that leave MPS through the Milwaukee Parental Choice Program.
While the Forward Institute raises interesting points about the overall effect of familial influences on a child’s education and their overall success, the group fails to examine the full scope of research that the SCDP has produced in the realm of high school attainment in Milwaukee’s public and voucher schools.
The study is only able to examine exposure to the voucher schools, as students who started in an MPCP school at 8th grade may have switched to an MPS school prior to graduation. That student would be counted as an MPCP graduate, and vice versa. There is no accounting for which students switched at what point and to/from what school.
– Forward Institute Analysis of the Milwaukee Parental Choice Project Attainment Study.
This point ignores one of the most important significant findings of year four of the School Choice Demonstration Project – that students who stayed for four years in a voucher high school had higher graduation rates than students that stayed for four years in MPS schools. These students were tracked through the four-year period and their educational choices were recorded over this span. According to the SCDP 2011 release:
As Table 3 indicates, the graduation rates for students who stayed in their initial status, regardless of sector, were much higher than for those who left–albeit the difference is much greater for MPCP. Of the 2006 9th grade students who stayed in the voucher schools for four years, 94 percent graduated. The rate for students who stayed in MPS, while lower (75 percent), is comparable to the graduation rate for students who were in MPCP in 2006 regardless of whether the latter stayed in the same sector (e.g. the rates reported in Table 2).
This same group of students was more likely to enroll in a four-year college as well.
Before Year Five’s contested breakthrough, this was the only real significant effect that had been seen on either side. While the rate of response is limited since these follow-ups and tracking were done via enrollment confirmations with local schools as well as phone interviews with participating parents, it still gives us a strong insight on what happened to students that remained in the same high school for four years. Unfortunately, the Forward Institute fails to address this finding in their review, focusing only on 2012 and 2013’s results.
In terms of how that attrition affects the SCDP’s overall findings; well, perhaps it’s best to let one of the study’s lead authors handle that. Here’s what Dr. Patrick Wolf had to say in a EdNext response to earlier criticisms that walked down a similar path to the Forward Institute’s analysis:
The difference between the 75% and 56% figure is largely immaterial because our “intention-to-treat” analysis exclusively measures the effect of starting high school in the voucher program on future levels of educational attainment regardless of how long you stayed in the program. Okay, let’s all say this together, “Program attrition has no effect on the internal validity of intention-to-treat analyses of program effects.” None. Period. Anyone who doesn’t accept that doesn’t understand the basics of program evaluation and shouldn’t be discussing studies that employ such scientific methodologies.
– Dr. Patrick Wolf, April 3, 2013
Of course, what the Forward Institute’s analysis fails to do is find any negative effect on Milwaukee Public Schools or its students as a result of the MPCP. In fact, no longitudinal analysis of any school choice program in America has found that vouchers or tax-credit scholarships have had any adverse affect on students. A study by the Friedman Foundation’s Greg Forster reviewed 29 empirical studies that used random assignment – the gold standard of comparative research in social science – to examine the effects of school choice on students and their local public schools.
Forster’s research turned up 28 studies that returned positive results for students or schools as a result of voucher or tax-credit scholarship programs. Only one study, which looked at the Washington D.C. voucher program, found no significant impact on public schools as a result of school choice. While many of these programs provided modest gains and benefits to classrooms and their pupils, the most important takeaway from Forster’s research is that school choice has yet to be the devastating blow to public schools and students that opponents have claimed that it would be since the early 1990s.