What the Nation's Report Card Measures and What Proficient Really Means

Every school wants a good report card. Only one test gives the nation a report card the schools cannot grade themselves. It is called NAEP, and almost nobody reads it correctly.

Let me tell you why that matters, because it is a story about measurement, and conservatives care about measurement. We believe in accountability, in checking the claim against the record, in not taking the producer's word for the quality of the product. The National Assessment of Educational Progress is the closest thing American education has to an audit. This post explains what it is, how it works, what its three achievement levels mean, and why a headline that says "only a third of students are Proficient" does not say what most readers think it says.

The claim

NAEP is the most reliable comparison of student achievement across states and over time, but the word Proficient is widely misread, and both the people who dismiss NAEP and the people who panic over it are usually misreading it. I will make the case in three parts: what NAEP is, what its levels mean, and why state tests tell a different story.

What NAEP is

NAEP stands for the National Assessment of Educational Progress, and it is often called the Nation's Report Card. It is a congressionally mandated assessment, run by the National Center for Education Statistics, part of the Department of Education's research arm. It does not exist to rank individual children or to rate individual teachers. It exists to answer a question a free people is entitled to ask: how well is the country educating its children, and is it getting better or worse?

The first assessments were given in 1969. The idea behind NAEP was to give the country an independent gauge of results, which is more than the schools alone could provide.

The program is governed by the National Assessment Governing Board, created by Congress in 1988. The Board includes governors, state legislators, teachers, school administrators, business leaders and members of the public. It sets the policy for NAEP: which subjects are tested, how the frameworks are written, and, crucially, what the achievement levels mean. Because the Board sets policy while the National Center for Education Statistics runs the tests, the people who write the standards are not the people who administer the assessment.

What it does not do: no child gets a score

This part surprises parents. NAEP does not give any student, teacher, school or, in the main assessment, most districts a score.

NAEP works by sampling. A carefully designed sample of schools is drawn, and within each selected school a sample of students takes part. No student takes the whole test. Each student answers only a portion of the questions, and statistical methods combine the pieces into estimates for groups: the nation, a state, and subgroups defined by things like race, family income and disability status. That is why NAEP can cover a wide range of material without burdening any one child and why it cannot tell you how your own daughter did.

This design has a consequence that conservatives should appreciate. Because there are no stakes for the school or the child, there is little reason to teach to the test or to game the results. A school cannot fire a principal over a NAEP score, and a child cannot be held back by one. That is the very reason it is trusted as a yardstick. A thermometer that the patient controls is not much use.

The two NAEPs, main and long-term trend

There is a second source of confusion. NAEP comes in two forms.

Main NAEP is the assessment most headlines are about. It tests students in grades 4, 8 and, in some years, 12, in subjects such as reading and mathematics (and others, like science, history and civics, on a less frequent schedule). Its frameworks are updated from time to time to keep up with what schools teach, and it reports state-level results, which makes it the tool for comparing Massachusetts with Mississippi.

Long-term trend NAEP, often shortened to LTT, is different. It tests students at ages 9, 13 and 17 in reading and mathematics, and it has kept its content largely stable over decades so that scores from different years can be compared. If you want to know whether a 17-year-old today reads better than a 17-year-old in the early 1970s, LTT is the instrument built for it. If you want to compare states in the same year, main NAEP is the instrument. Mixing them up is a common error; our post on Real Spending Per Pupil More Than Doubled Since 1970, So Where Are the Gains leans on the long-term trend for exactly this reason.

There is also the Trial Urban District Assessment, or TUDA, which began in the early 2000s. It reports results for a group of large urban districts that volunteered to be measured, which lets us compare Boston with Houston with Cleveland, and ask what separates the cities that improve from the cities that do not. It is voluntary, and the participating districts are not a random sample of American cities, so be careful with generalizations. But it has produced some of the most useful comparisons in education research.

The three levels and the word that causes trouble

NAEP reports results two ways: as scale scores, and as percentages of students at three achievement levels. The Governing Board set the levels, and the names are Basic, Proficient and Advanced. Students below Basic are sometimes described as below Basic, and the share at or above Basic is often reported alongside the share at or above Proficient.

Here is where parents go wrong. In ordinary speech, "proficient" sounds like "good enough," and "good enough" sounds like "on grade level." NAEP's Proficient is neither of those things. The Governing Board's own description is that Proficient represents solid academic performance for the grade assessed, with demonstrated competency over challenging subject matter, including subject-matter knowledge, application of that knowledge to real-world situations, and analytical skills appropriate to the subject. The Board has stated plainly that Proficient is not the same as being on grade level, and Proficient is a standard of competency over challenging subject matter.

Basic, by contrast, means partial mastery of the knowledge and skills needed for proficient work at that grade. A student who is at Basic is not failing. A student below Basic is failing to demonstrate even partial mastery.

So when a headline says "only about one in three eighth graders is proficient in math," it is describing the share who reached an ambitious benchmark, not the share who are failing. That is a fair statement, and it is also an incomplete one. If you want the more sobering reading, look at the share who are below Basic. That is the number that tells you how many children lack the basics.

I think the honest conclusion has two edges. First, the critics are correct that the label is easy to misread, and some defenders of the schools have used that fact to wave away every bad result. Second, the label is not arbitrary, and the benchmark is not a trick. Students who reach it are the ones who would likely be prepared for the next stage of schooling, which is a reasonable thing to ask of a school system. We may fairly debate whether the bar is placed in the right spot. We may not pretend that the underlying scale scores, which do not depend on the labels at all, say something different.

Why the scale scores matter more than the labels

Because the levels invite argument, serious readers look at the scale scores and at the trend in them. NAEP reports a numerical scale for each subject, and the scales differ by test, so check which one a table uses. A score is not a grade. It is a point on a measurement scale, and a change in that score tells you whether students are learning more or less than before.

Here is the practical advice. When a headline gives you a percentage at a level, ask three questions:

1. Which NAEP is it, main or long-term trend?
2. Which grade and subject?
3. Has the scale score moved, and by how much, relative to earlier years?

If the answer to the third question is "barely," then arguments about whether the cut line is generous or harsh miss the point. The line can be wrong and the trend can still be flat. And the trend is the thing the country should worry about.

State tests and NAEP, a very different story

Now we come to the part of the argument that every parent deserves to understand, because it affects how you read your own child's report from school.

Since the No Child Left Behind Act of 2002, and again under its successor, every state has had to test students in reading and mathematics and report the share who reach "proficiency" on the state's own test. But the law left each state free to write its own test and to decide for itself where the proficiency line sits. The result was a patchwork. A child could be "proficient" in one state and not in another on the same level of skill. And in many states, the share of students labeled proficient by the state was far higher than the share labeled Proficient by NAEP.

That gap is not a mystery. Each state sets its own bar, and every state has a political incentive to set it where its schools look good. Education researchers have studied this for years, comparing state standards by mapping them onto the NAEP scale. The consistent finding has been that many states set their proficiency bars well below NAEP's. Some states raised their standards after those comparisons became public. Others did not.

What does this mean for a parent? If your child's school tells you she is proficient on the state test, that is a piece of information, but it is a piece of information produced by a system with an incentive to look good. NAEP is produced by a body with no stake in any one school's rating. When the two diverge, the divergence itself is a story.

I want to be careful, because there is a tempting overreach here. A mismatch between state results and NAEP results does not prove that any individual teacher or school is hiding anything. Often it reflects the plain fact that the state test is aligned to the state's curriculum and designed for a different purpose, as a gate for a student rather than a survey of a population. Still, when 70 percent of a state's students are called proficient and a much smaller share reach Proficient on NAEP, the honest conclusion is that the state's label means something less demanding than the national one.

The conservative stake in an honest yardstick

Why do conservatives, of all people, care about a federal assessment? Isn't education a state and local matter?

It is, and nothing about NAEP changes that. The Constitution does not list education among the powers of Congress, and the Tenth Amendment reserves what is not delegated to the states or the people. NAEP does not control a curriculum, it does not set a standard that any school must meet, and it does not penalize anyone. It is information. A federal government that tests nobody and gathers a sample of data so that states, parents and legislators can judge their own performance is exercising a modest, almost clerical power, and the information it produces is a public good.

Information is exactly what a self-governing people needs when it chooses between school systems. Parents choosing a school, legislators deciding whether a reform worked and voters judging a superintendent all need a measure the school cannot manipulate. If you want to know whether an education policy has helped, whether it is a new reading curriculum, a voucher program or a union contract, you need a stable yardstick that does not change when the policy does. NAEP is that yardstick.

Consider what it has shown. It has recorded the long flatness of high school results, the widening gap between the strongest and weakest students, and the sharp drops in performance after the extended school closures that began in 2020 (on which see What the Long School Closures of 2020 and 2021 Taught Parents). It has also recorded real success stories, including state-level gains in early reading that came from decisive changes in instruction, and we cover one of them in The Reading Wars, How Whole Language Lost a Generation and Phonics Won Back. Without NAEP, such claims would rest on each state's own word.

What a good reader does with a NAEP headline

Let me turn this into a short guide. It is not advice about your family's finances or your child's diagnosis, only how to read a published table.

• Find the original. Start from the tables published by the National Center for Education Statistics, not from the news story about them. The state profile pages show your state's scale scores and achievement-level shares across years.
• Look at both ends. Check the share at or above Proficient, and also the share below Basic. A policy that lifts the top while leaving the bottom untouched is not the same as a policy that lifts the floor.
• Read the subgroup rows. The gaps between groups of students are shown in the data, and so are the cases in which a group in one state outperforms the same group in another. Those comparisons often point to what works.
• Compare like with like. Compare your state's students to students in other states with similar demographics, not just to the national average. State averages move with the makeup of the population.
• Note the confidence interval. Because NAEP is a sample, small differences are not meaningful. A one-point change in a state's average is usually statistical noise, not news.

The strongest objection, answered

The best argument against leaning on NAEP runs like this. NAEP is a single test, taken in a few hours, by sampled students who have no incentive to try. The Proficient bar is, by the Board's own description, a demanding one. Treating its results as a verdict on schools distorts the picture, and the focus on test scores narrows what schools are for.

That objection has some force, and I will answer it in three points.

First, the low stakes cut both ways. Students who do not care might try less, but there is no evidence that low effort has gotten systematically worse in a way that would explain decades of flat results, and the trend comparisons are made between groups taking the same low-stakes test.

Second, the test is not the whole of education, and nobody serious says it is. Character, civic knowledge, craft and faith matter. But reading and mathematics are the foundation of everything else, and a school that cannot teach them has no business claiming it is teaching the rest.

Third, if you doubt NAEP, you owe the public a better measurement, not less measurement. Dropping the yardstick does not shrink the room.

Where this connects to the rest of the argument

The NAEP is the evidence behind several of the arguments on this site. When we show that spending rose while results did not, NAEP is the source. When we argue that a reading reform worked, NAEP is the proof. When we ask whether a state's tests are honest, NAEP is the comparison. And when parents ask who is responsible for results, the answer starts with understanding Who Actually Runs Your School District, Board, Superintendent or State. Other posts in this series, such as The School Staffing Surge, Hiring Outpaced Enrollment for Decades, ask where the money went. NAEP is the reason we know to ask.

You can also see the arithmetic of public spending on the national numbers page, and the Constitution's reservation of powers to the states in the text of the Constitution.

The case in brief

1. NAEP, begun in 1969 and overseen since 1988 by the National Assessment Governing Board, is a sample-based assessment that reports group results, never individual scores.
2. Main NAEP compares states and grades; long-term trend NAEP compares ages across decades; the urban district assessment compares large cities.
3. Proficient is a demanding standard that the Governing Board says is not the same as grade level, so the right figures to watch are the scale scores, the trend and the share below Basic.
4. State test pass rates are often far higher than NAEP results because each state sets its own bar.
5. A free people choosing among schools needs a yardstick the schools cannot move, and NAEP is the best one we have.

What you can do: open your state's NAEP profile on the National Center for Education Statistics website and compare it with the pass rate your school district reports on the state test.