Ozimek points out that the lottery winners not only saw large increases in test scores, but they also showed improvements in other quality of life measures.
I think the statistical measures from the study actually understate the improvements, because they are carefully controlling for sample size, response rate variations, etc. But, the raw data is overwhelming. From page 17 of the study:
The authors note that this is a high-achieving charter school, and that some evidence shows that typical charter schools don't outperform typical public schools on test results. But, the more important factor is the effect from the right to exit on ongoing innovation and administration. Imagine all the pressures there are to improve the Harlem public schools. Public schools across the country are responding to calls for improvement. And, underperforming charter schools are simply closing. Compare that to the circus of self-interested advocacy that surrounds underperforming public schools. This is a subtle distinction that is counter-intuitive: Q: Has the charter school movement been successful? A: Yes! Just look how many have closed!Seventeen percent of female lottery losers report having been pregnant at some point...... Female lottery winners are 12.1 (4.6) percentage points less likely to report that they have ever been pregnant, a 71 percent reduction from the control mean.......
Four percent of male lottery losers were incarcerated during our sample period, compared to none of the male lottery winners.
Ozimek also makes the following point:
If these results were found for a particular high-performing pre-k program then the response by reform critics would be that we need to figure out how to increase access to high-performing programs like this, and not too minimize it by pointing to less impressive average program impacts and the difficulty of replicability. The difference is that universal pre-k would represent mostly new government spending and an opportunity for expanding the public sector workforce, while high-performing charters in the long-run are more likely to crowd out existing public sector workers and spending.