Head Start's Gains Faded by Third Grade, What the Federal Study Found

There is a rule in Washington that a program may be evaluated by anyone except the people who run it, and that when the evaluation comes back bad, the right response is to ask for a bigger budget so the next evaluation can be better. Head Start is the rare federal program that went further than most. It hired independent researchers, randomized the children, waited, and published the answer. The answer was not what the brochure said.

This post is about that answer. It is not an attack on poor children or on the teachers and aides who work with them, who are doing a hard job for little pay. It is an examination of a promise that government made in 1965, a test that government itself ordered, and a lesson about what we should demand before the next, much larger, promise is made.

The terms

Three terms need defining before we argue.

• Head Start is a federal program that funds early-childhood education, health, nutrition and family services, mainly for children aged three and four from low-income families. It was launched in 1965 as part of the War on Poverty, initially as a summer program run through the Office of Economic Opportunity.
• A randomized evaluation assigns children by lottery to the program or to a control group. Because the groups differ only by chance, any later difference between them is attributable to the program. It is the gold standard because it removes the most common trick in social science: comparing children whose parents were motivated enough to enroll with children whose parents were not.
• Fade-out is the pattern in which an early advantage shrinks as the children who did not attend catch up, and eventually disappears from the test scores.

The promise

The idea behind Head Start was attractive and sincere. Children from poor families often arrive at kindergarten behind, the thinking went, and the gap persists. Reach them at three and four, with learning, food, health screenings and parent involvement, and you could close the gap before it hardened. The program grew from that summer in 1965 into a standing federal institution, funded year after year through the Department of Health and Human Services.

For decades the evidence for it was a mixture of observational studies, which compared enrolled children with others and could not rule out selection, and small, intensive model programs, which I will come to. The program had a strong constituency and no randomized test of itself at scale.

The test

In 1998, Congress, in reauthorizing Head Start, required a national evaluation. The result was the Head Start Impact Study: a randomized trial in which three- and four-year-olds who applied to participating centers were assigned by lottery either to Head Start or to a control group that could find other arrangements, including other preschools or none. Several thousand children at hundreds of centers took part, and the study followed them for years.

Its reports came in stages. Two matter here.

The 2010 report looked at children through the end of first grade. The picture was mixed to thin. The Head Start group had edged ahead on some measures of vocabulary and early literacy during the program year, and by the end of kindergarten and first grade most of those edges had narrowed or vanished.

The 2012 report followed the children to the end of third grade and reached the conclusion at the heart of this post. In the words of the evaluators' summary, there was little evidence of systematic differences between children in the Head Start group and those in the control group in cognitive, social-emotional, health and parenting outcomes at the end of third grade. Put plainly: the children who won the lottery and the children who lost it looked about the same.

I want to be careful, because the careful reading is the stronger one. The study did not find that Head Start harmed children, and it did not find that children learned nothing in the program year. It found that the measured advantage did not last, and that the program, as it actually operated, did not move the outcomes on which its promoters had staked its case.

The objection: "The control group was not empty"

The first thing a defender will say is fair, and a skeptic should say it too. In a randomized study of a program that serves most eligible children, the children in the control group do not sit at home. Many of them attended some other preschool, including state pre-K programs and other subsidized care. If a lot of the control group got a decent alternative, then the study measures Head Start against something real, not against nothing, and the difference will be smaller.

That is true, and the evaluators addressed it by looking at differences. What it implies, however, is awkward for the program's defenders. It means that the marginal federal dollar bought little beyond what other arrangements already provided. Taxpayers are entitled to ask about that margin. A program does not justify its budget by pointing out that the alternative was also pretty good.

Then Tennessee made it worse

The Head Start study is an old finding by now, and one might hope the next large randomized test would be kinder. It was not.

Researchers at Vanderbilt University, led by Mark Lipsey and Dale Farran, studied Tennessee's Voluntary Pre-K program. Tennessee had oversubscribed classrooms, so many children were admitted by lottery, which created a rare opportunity for a randomized comparison of a state-funded pre-K program of reasonable quality. The study reported its early results in 2015 and its later findings through sixth grade in a paper published in 2022.

The early result was a positive bump in the program year. The result at sixth grade was the reverse: the children who had been admitted to pre-K scored lower on state achievement tests than the children who had not, and they also had more disciplinary infractions. I will say what I know and what I do not. I know the direction of the finding and the source. I do not know the mechanism, and the authors were cautious about proposing one, so a careful reader should be too. Anyone who tells you with certainty why it happened is telling you a story, not a result.

What the Tennessee study establishes is smaller than a repeal of pre-K, and larger than a footnote. It establishes that a randomized, state-run program of the kind reformers propose to universalize can produce, over time, worse outcomes than doing nothing. That is exactly the possibility that a policy debate based on early gains never has to face.

The steelman: Perry, Abecedarian and the long run

The best case for early education does not rely on third-grade test scores. It relies on two famous experiments and on the long run.

The Perry Preschool Project began in Ypsilanti, Michigan, in the early 1960s. A small group of low-income Black children, around one hundred twenty in all, were randomly assigned to an intensive preschool program with a high ratio of teachers to children and regular home visits, or to a control group. The participants were followed into middle age. The reported results included higher rates of high school graduation, higher earnings and lower arrest rates, and the economist James Heckman has given the findings a wide audience and attached large estimated returns to them.

The Abecedarian Project in North Carolina, begun in the 1970s, enrolled children from infancy in a very intensive full-day program and found long-run gains as well.

Defenders also point to studies, including work by the economist David Deming, that compared siblings, one of whom attended Head Start and one of whom did not, and found long-run advantages even though test scores had faded. The suggestion is that the benefits may live in places test scores do not look: behavior, self-control, motivation.

That is a serious argument, and it is why I do not claim that early education never works.

The answer

Here is what the steelman actually proves, and what it does not. I will do it in numbered points.

1. Perry and Abecedarian were small and exceptional. One site, a few dozen to about a hundred children, run by researchers, with staff and intensity that a national system serving a million children cannot reproduce. Scaling is the entire problem. A recipe that works at the dinner party for twelve does not tell you how the cafeteria will cook for twelve thousand.
2. The experiment that tested the real thing at scale found fade-out. The Head Start Impact Study is the randomized test of the program as it actually exists. It is a better guide to what a national program does than a seminar's demonstration.
3. Sibling and long-run studies are not randomized. They compare families who made different choices, which brings back the selection problem the lottery was meant to remove. They are suggestive, and a conclusion drawn from them should be held more loosely than a lottery.
4. Fade-out plus a late benefit is a hypothesis, not a result. The Tennessee study looked for the late benefit and, at sixth grade, found the opposite. If benefits were hidden in behavior, one would expect fewer disciplinary problems later, not more.
5. Policy should be proportioned to evidence. A promise to close achievement gaps is a promise about outcomes. When the best randomized evidence says the outcomes did not move, the burden of proof shifts to the proponents, and it should shift before the checkbook opens, not after.

What it means for universal pre-K

All of this arrives at a practical question. Across the country, politicians propose to make pre-K universal: free, for every four-year-old, and sometimes three-year-old, at public expense.

Consider the arithmetic of proportion, with an explicitly made-up example. Suppose a state proposes to add a free pre-K year for every child. The cost is real and recurring; the benefit is, by the best randomized evidence we have, uncertain and possibly absent. A prudent legislator treats that the way a prudent family treats a large purchase of unproven value: pilots it, measures it with a lottery, publishes the results, and expands only if the results hold.

This is not a counsel of despair. It is the opposite. If early education works, a well-designed randomized test will find it, and we can then expand with confidence. If it does not, we will have saved billions that could go to something that does work, such as the reading instruction of children already in school (see The Reading Wars, How Whole Language Lost a Generation and Phonics Won Back) and the parental choice discussed in Milton Friedman's 1955 Voucher Essay and Its Seventy-Year Road.

Here is the question the coverage almost never asks. When a program fails its own evaluation, why is the headline "Experts Call for More Funding"? The facts in the evaluation are the facts. The "more funding" is an opinion. A news story that blurs the two is doing advocacy.

What a lottery can and cannot tell a legislator

Let me say a word for the reader who thinks all this is statistical hair-splitting. It is not, and a made-up example shows why. Picture two towns, each with a new reading program. In Town A, the parents who sign up are the ones who already read to their children every night; a year later their children are ahead, and the superintendent holds a press conference. In Town B, enrollment is by lottery, and a year later the winners and losers are even. Which town has learned something about the program? Only Town B. Town A has learned something about its parents, who would have done well with a cardboard box and a library card.

That is the whole value of the Head Start lottery. It is the difference between a program that looks good because of who walks through the door and a program that makes a difference to whoever walks through it. Every earlier claim for Head Start had Town A's problem. The Impact Study was the first at national scale to put Town B's design in front of the question, and it is a credit to the federal government that it ordered the test. It would be a discredit to ignore the result.

There is a second lesson, and it is about speed. A legislature can pass a universal program in a session. A lottery study takes a decade to follow children into third grade, and longer to reach sixth. That mismatch is why so many programs are enlarged on the strength of the first, flattering year of data. The remedy is boring and sound: write the evaluation into the statute before the first dollar moves, require the lottery, and require that the results be published in full whether they flatter the sponsors or not. Government that is confident in a program should welcome such a clause. If a bill's champions resist it, you have learned something about their confidence.

The wider pattern

Head Start is one instance of a pattern that Real Spending Per Pupil More Than Doubled Since 1970, So Where Are the Gains documents for schooling in general: money rises, hope rises, outcomes do not. It is the pattern of a government that funds intentions rather than results. The moral of the story is not to stop caring about poor children. It is that caring about them is measured by what happens to them, not by what we spend in their name.

A free society has a particular tool for this problem, which is the family. Parents of three- and four-year-olds already make decisions about care, with grandparents, churches, neighbors, small centers and employers. A tax system that left more money in their hands, or a child-care policy that funded the family rather than the institution, would respect a choice that no federal evaluation can reach. The conservative objection to a universal public pre-K is not that early learning is bad. It is that the state should not become the default parent of every four-year-old on the strength of evidence it has itself shown to be weak.

For more on how Washington tends to grow a program after its justification has been tested and found wanting, see The School Staffing Surge, Hiring Outpaced Enrollment for Decades.

The case in brief

1. Head Start launched in 1965, and Congress ordered a randomized national evaluation in 1998. Its third-grade report in 2012 found little evidence of systematic differences between the Head Start and control groups.
2. Tennessee's randomized pre-K study, reported in 2022, found the early gains reversed by sixth grade, with lower test scores and more discipline problems among the pre-K group.
3. The best case for early education, Perry and Abecedarian, rests on small, intensive, single-site programs that a national system cannot easily replicate.
4. Evidence-based policy means piloting with lotteries, publishing results and expanding only what works, before universal pre-K is made permanent.
5. The family, not the state, is the proper default decision-maker for a three-year-old.

What you can do: read the executive summary of the Head Start Impact Study on the federal Administration for Children and Families website, then ask your state legislators whether any pre-K proposal before them includes a randomized evaluation.