Where have all the KIPPsters gone?

By Caroline Grannan, San Francisco

[Charter schools] have a distinct advantage … Their families have already chosen to be at a charter and have often jumped through numerous hoops to get there. This makes it easier for charters to create their own cultures. They can define the length of their days, dictate exactly how children dress and enforce strict codes of conduct. Those students — scholars, in charter parlance — who fall out of line don’t last.

– Jonathan Mahler in the New York Times Magazine

I will never follow the lead of those who exclude the kids who need education the most so that their precious scores will rise.

– John Kuhn, superintendent of Perrin-Whitt Consolodated Independent School District, Perrin, Texas

The highly acclaimed charter school operator KIPP – the Knowledge is Power Program – wins widespread praise for the overall high achievement of the low-income students it serves. It’s no wonder that KIPP’s practices are watched closely. And that means asking an obvious question: Why do so many KIPP students leave the schools without being replaced, and how does that affect the schools’ achievement?

Why is attrition at KIPP schools an issue?

Several studies show that a high number of students who enroll at KIPP schools leave the schools early, and the numbers show that students who leave aren’t replaced by new, incoming students. At public schools that serve comparable demographics, students who leave are replaced by new, incoming students. A 2008 study of San Francisco Bay Area KIPP schools by SRI International found that it’s consistently the lower-performing students who leave.

How does this affect KIPP’s achievement?

The lower-performing students are no longer there to bring down averages – but it would also be valuable to learn more about the impact on the students who don’t leave after the lower performers have departed. Do they learn more and achieve more, unfettered by their less successful former classmates?

This is difficult to address, since the topic is so often met with denial and distortion. Would those students do as well at public schools if their less successful classmates left and weren’t replaced? KIPP schools’ achievement is regularly compared to public schools’ achievement, but the attrition question confounds those comparisons.

Richard Kahlenberg, senior fellow at the New Century Foundation, wrote: “The issue is important because if large numbers of weaker students drop out of KIPP’s rigorous program, it would be highly unfair to compare the test score gains won by the top KIPP students against the scores of all regular public school students – who include KIPP dropouts.”

Unfairness is only one issue. If we want to look at KIPP schools to see how they often achieve academic success, the high attrition confounds efforts to do that. We frequently hear that KIPP schools have waiting lists, but the attrition puts that supposed situation in a whole different light. If there are waiting lists, why aren’t all the departing students immediately replaced?

The movie “Waiting for ‘Superman’,” a PR tool for charter schools, presented the story of Daisy, who hoped to attend KIPP Los Angeles College Preparatory Academy and was devastated to lose out in the admission lottery. But in real life, the high attrition at KIPP LA Prep means Daisy shouldn’t have to wait long for an opening.

For example, KIPP LA Prep’s most recent 8th-grade cohort lost a third of the students who started in grade 5 by the beginning of 8th grade – figures aren’t publicly available for how many students finished 8th grade. For that class, which started grade 5 in the 2006-’07 school year and finished grade 8 in 2010, the number dropped from 97 students at the beginning of 5th grade to 81 by the beginning of 6th grade to 65 by the beginning of 8th grade.

Based on that attrition rate, the school would have room for Daisy and many more hopeful applicants if the administrators filled those spots from the waiting list. Why isn’t that happening? It’s a mystery.

Does this attrition happen at all KIPP schools?

The available research doesn’t provide that information.

  • In early 2007, I researched attrition at KIPP’s California schools as a volunteer project, using data publicly available on the California Department of Education website. At the time, KIPP had nine schools in California. My research found very high attrition at six of them. I also broke down the attrition by demographic subgroup. At all six of the schools with high attrition, the attrition was much higher in the subgroup that’s statistically likely to be the most academically challenged – either African-American males or Latino males, depending on the school. My blog posts about the issue appear to mark the first time KIPP attrition had been publicly discussed. What my findings refer to is the overall drop in the number of students – total and by subgroup – year by year in a grade cohort.
  • In fall 2008, the organization SRI International released a study of the five KIPP schools that existed at the time in the San Francisco Bay Area. SRI used data that went deeper than the publicly available data I’d used, and found high attrition at all five Bay Area KIPP schools. SRI reported that overall, 60% of the students who enrolled at the five KIPP schools didn’t finish at those schools. SRI also found that the students who left were consistently the lower-performing students. If SRI’s research broke the students down by demographic subgroup, that wasn’t included in the final report. (Interestingly, SRI International found the high attrition pattern even at a KIPP school – KIPP Heartwood in San Jose – that didn’t show high attrition in the figures I researched. As noted, SRI International had access to more complete data than I did; I used only publicly available statistics.)
  • KIPP supporters responded to my research and to the SRI study with this claim: “The San Francisco KIPP schools are outliers.” But that was an invalid, misleading and inapplicable response. I had researched all the KIPP schools in California (nine at the time), and SRI had researched all the KIPP schools in the Bay Area (five at the time) – not just the two San Francisco KIPP schools. The two San Francisco KIPP schools, KIPP San Francisco Bay Academy and KIPP Bayview Academy, did show high attrition, but not the highest among the Bay Area’s or the state’s KIPP schools. It wasn’t true that the San Francisco KIPP schools were outliers, and that claim didn’t negate my findings or SRI’s findings. It didn’t even make sense in context of those findings.
  • A 2010 study of KIPP schools by Mathematica Policy Research was framed in a manner intended to refute reports of high attrition at KIPP. The report found that of the KIPP schools studied, one-third showed lower attrition than comparable public schools, one-third showed higher attrition and the rest showed comparable attrition. However, the Mathematica report failed to address the issue of whether KIPP replaces the students who leave, which leaves a giant gap in the report.
  • A 2011 study by Western Michigan University researchers, published jointly with Columbia University, found significantly higher attrition at KIPP schools than at comparable public schools – 15 percent attrition per year at KIPP vs. 3 percent at public schools. The study found that 30 percent of KIPP students leave between 6th and 8th grades.

But many KIPP supporters say that public schools have the same high rates of attrition, and the Mathematica study said that too.

That’s not valid because of the critical difference that at public schools, the students who leave are replaced by new, incoming students. The numbers show that at KIPP schools, students who leave are not replaced. The cohort of students simply shrinks drastically. Even though Mathematica did indeed make that statement, it’s not valid. It flies in the face of logic.

The distinction puts the situations in an entirely different light. To give a clear picture: Let’s say 100 students start 5th grade at a KIPP school that serves grades 5-8. Sixty of the students leave along the way, before completing 8th grade, and those are the lowest-performing 60 students. They aren’t replaced with incoming students. So that cohort at the KIPP school winds up with only the 40 highest-performing students. At the public school down the street, 100 students start. Sixty of them leave along the way, but each time one leaves, a new student arrives to replace him or her. So that cohort at the public school consists of 100 students from start to finish. Clearly, those two situations are not parallel, equivalent or comparable.

Low-income, at-risk students are likely to be “high-mobility” – meaning that they move a lot due to the instability that tends to afflict the lives of impoverished families. Those students are also statistically likely to be low academic achievers. With the high mobility that characterizes low-income communities, if if a high-mobility student leaves a public school, he or she is replaced with a similarly high-mobility student. By contrast, if a high-mobility student leaves a KIPP school, the numbers show that KIPP is usually not replacing him or her with an incoming student.

Why do the students leave KIPP schools?

That’s not publicly known. KIPP spokespeople and supporters deny that KIPP expels or “counsels out” low-performing students.

Does KIPP have a policy of not enrolling new students after the starting grade? How do we know KIPP isn’t replacing the students who leave?

KIPP doesn’t appear to have an actual policy of not enrolling new students. But the enrollment numbers at the KIPP schools studied show that the students who leave are, overall, not being replaced with incoming students. It’s not clear why that is, given the widespread reports of “long waiting lists.” Even if some of the students who leave are replaced with incoming students, the numbers still show that a very high number are not, resulting in very high total overall attrition and significant shrinkage of the grade cohorts.

If schools receive state funding based on the number of students, doesn’t the attrition mean that KIPP schools lose funding as students transfer to other schools?

Yes, it must mean that, though there’s discussion of whether the students leave after they are counted for the year, so that KIPP still receives the funding. (This situation could vary state by state.) The KIPP organization receives an immense amount of private philanthropical funding, which may provide enough of a cushion against the loss of the public funding. Perhaps the tradeoff – losing the less-successful students and also losing the per-student funding – is worth it to KIPP schools.

But there’s a high dropout rate in public schools that serve low-income populations, so how can we say that public schools replace the students who leave?

Most KIPP schools are middle schools, serving grades 5 through 8. (Almost all of the KIPP schools that have existed long enough for their attrition to be tracked are middle schools.) Except for a very small number of extreme, problematic outliers, students don’t drop out of middle school, so public schools don’t suffer from dropout rates at those grade levels. Students who leave KIPP schools would be transferring to other schools. I did comparisons of grade cohorts in demographically comparable public schools, and those schools simply didn’t show a pattern of attrition at all.

Why would Mathematica make the misleading statement that attrition was comparable to public schools?

Research organizations are known to negotiate with the funders of the research about just how the findings will be presented. It is not publicly knowable what kind of negotiations went on with Mathematica, whose report was funded by KIPP. The 2008 SRI International report – the one that found 60 percent attrition at all the Bay Area KIPP schools and reported that it was the lower achievers who left – presented that finding as a secondary one in its report. The report, which was also funded by KIPP, announced KIPP’s high achievement as the primary finding. The attrition was the newsworthy finding and the one that is still extensively – and increasingly – discussed today.

Some reports have said it might not be that students are leaving KIPP schools but that they’re being required to repeat a grade. That might be true in some – or many – cases. But the overall numbers still show the grade cohorts shrinking, so many students are clearly leaving the schools.

So what’s the conclusion?

KIPP attrition isn’t comparable to the flow of high-mobility students in and out of public schools – it simply isn’t, no matter how many claims there are to the contrary. Why it happens and what it means are still hard to pin down.

Posted on by CarolineSF Posted in Uncategorized

12 Responses to Where have all the KIPPsters gone?

  1. Stuart Buck

    Unfortunately for Caroline, Mathematica released a much more definitive analysis, and her claims about KIPP and public schools have been distinctly refuted.

    Mathematica’s recent study looked at attrition and replacement rates. Not only does KIPP actually have lower attrition rates for black males than comparable public schools, KIPP schools do (on average) replace students who leave:

    “Pooling the results across all offered middle school grades, there is no systematic pattern of differences in the prevalence of late arrivals at KIPP schools and comparison district schools. . . . Averaging the results across sites, the overall proportion of late arrivals at KIPP (15 percent) is similar to the overall proportion in the district comparison group (14 percent). After testing for statistical significance in each site, we did not find any systematic pattern of differences in the proportion of late arrivals at KIPP, compared with district schools. Some KIPP schools admit significantly more late arrivals, other schools admit significantly fewer late arrivals, and several schools are not statistically different from the comparison group.”

  2. CarolineSF

    KIPP itself acknowledged the attrition problem once it became part of the public conversation. So while I cited one invalid response that they sometimes give, overall they aren’t trying to dispute the figures.

  3. Stuart Buck

    And in the face of the recent Mathematica report, even you aren’t trying to dispute the fact that nearby public schools have similar or higher attrition, and that the replacement rates are also similar. It’s misleading to keep posting stuff like this.

  4. CarolineSF

    The numbers are clear, and KIPP — in its honest moments – acknowledges the problem. My post is accurate. (And Mathematica is being paid by KIPP to spin its findings to make KIPP look good.)

  5. Stuart Buck

    ” if a high-mobility student leaves a public school, he or she is replaced with a similarly high-mobility student. By contrast, if a high-mobility student leaves a KIPP school, the numbers show that KIPP is usually not replacing him or her with an incoming student.”

    That claim is false. You can’t just keep saying this when you have zero evidence of it, and when the evidence actually proves you wrong.

  6. Stuart Buck

    As I said over on Joanne Jacobs’ blog (to no response), no one disputes that KIPP schools have some attrition, and that some of the kids who leave aren’t replaced. What the Mathematica report now shows, however, is that similar public schools have pretty much exactly the same issues with attrition and replacement. Complain about the funding all you want, but Mathematica is very independent and the numbers are what they are.

  7. CarolineSF

    That’s false. Public schools do not have the same attrition; the numbers simply do not drop in comparable public schools. In public schools, new students (including those dumped by KIPP schools) cycle in to replace those who leave.

    I’m not complaining about Mathematica’s funding; they can be funded in whatever way they want. But they are not independent — anything but. KIPP funded their study and directed the way in which the results would be reported.

    You can be as persistent as you want in insisting that up is down and black is white, Stuart Buck, but what you’re saying is still false.

  8. CarolineSF

    …I did a lot of spot-checking making comparisons of comparable California public schools when I did the original KIPP blogging and KIPP defenders made the same false assertions that Stuart Buck is now making. Public schools do not show comparable attrition or anything like it.

  9. Stuart Buck

    You’re directly contradicting Mathematica’s numbers regarding public school attrition and replacements. You are doing this even though Mathematica has much more systematic and nationwide data than you could have ever collected, and even though Mathematica is tracking individual students over several years (which you can’t even claim to have done).

    If you can come up with more thorough numbers than Mathematica, go to it. Otherwise, it’s just your say-so against an independent organization of highly trained scholars working with data that is unbelievably superior to anything that you ever had.

  10. Stuart Buck

    ” Mathematica is being paid by KIPP to spin its findings to make KIPP look good.”

    If you would actually read the latest Mathematica study, there’s no spin in sight. Just an evenhanded presentation of numbers. And there’s no way to spin numbers about how many students are in schools from year to year — the numbers are what they are. In order for Mathematica to alter anything to KIPP’s benefit in this type of study, they’d have to be making up fraudulent numbers.

    Are you seriously accusing Mathematica of fraudulent numbers? Keep in mind that their incentive to do so is much less than your own incentive to make up numbers (given your seemingly personal vendetta against KIPP). And their incentive NOT to make up numbers is much greater than yours: they have much more of a reputation at stake if they were caught doing anything fishy with student attendance numbers that are on public record.

  11. Pingback: The Best Posts About Attrition Rates At So-Called “Miracle” Schools | Larry Ferlazzo's Websites of the Day...

  12. Pingback: Today’s “Round-Up” Of Good School Reform Articles & Posts | Larry Ferlazzo's Websites of the Day...