Seven Things Crime Statistics Cannot Tell You, and Two Things They Can

Every politician who has ever stood in front of a crime chart has been right about exactly one thing: the line goes up or down. After that, the speech is mostly improvisation. The chart does not say why, it often does not say where, and it sometimes does not say what it is counting. The audience nods anyway, because a line on a chart carries the authority of arithmetic.

Conservatives have a particular reason to get this right. We say that public safety is the first job of government and that policy has consequences. Both claims are testable, and they deserve better than slogans. So here is a method: seven popular claims about crime data, each stated, tested against how the numbers are built, and given a verdict. Then the two kinds of evidence I would bet on.

A word on method. Three sources run through this post. The FBI's Uniform Crime Reporting program records crimes known to police, and since 2021 it has been moving to incident-based reporting, as I explain in Why the FBI's Switch to NIBRS Made National Crime Numbers Harder to Read. The National Crime Victimization Survey asks households about their experiences, whether or not they called the police, as described in The Crime Survey That Asks Victims Instead of Asking Police Departments. And public health death records count deaths by homicide independently of both. Each source is built differently, and each claim below falls apart in a different place.

Claim one: "Crime fell last year, so our policy worked"

The claim. A mayor or a sheriff announces that crime dropped after a new program began, and credits the program.

The test. A one-year change tells you almost nothing about cause. Crime moves for reasons that have nothing to do with any local policy: demographics, the economy, national trends, weather, changes in how offenses are recorded. One year can be noise. And there is a statistical trap that catches the most sincere officials: regression to the mean. Cities tend to adopt a new program after a bad year, and a bad year is often followed by a better one whether or not anyone did anything. The program gets the credit for a tide that was already going out.

To say a policy worked you need a comparison: the same city before and after over a long window, set against similar cities that did not adopt the policy, with attention to whether the trend had already started. Without that, you have a story.

The verdict. A one-year drop proves nothing. This cuts both ways. When critics say a drop in a city that adopted a policy they dislike was luck, they have a fair point, and when supporters of the policy say the same about a rise, they have the same point.

Claim two: "Arrests measure crime"

The claim. More arrests mean more crime, or fewer arrests mean less.

The test. An arrest is a measure of what the police did, not of what criminals did. It reflects staffing, priorities, resources, the cooperation of witnesses, and the policy of the prosecutor. A department told to focus on drug offenses will record more drug arrests without any change in drug use. A department short of officers will make fewer arrests while crime climbs. One arrest may clear a dozen burglaries, or none.

Arrest data is useful for what it is: a record of enforcement. It is a poor proxy for the underlying volume of crime. Use it to understand police activity, and use other sources, especially victim reports and homicide counts, to understand crime.

The verdict. Arrests measure police work. They do not measure crime.

Claim three: "Reported crime equals actual crime"

The claim. The number in the FBI's table is the number of crimes that happened.

The test. It is the number of crimes that were reported to police, recorded by police, and passed on to the FBI. Each step is a filter. A victim may decide that calling is pointless. A department may record a complaint in a lesser category, or not record it. A reporting agency may not send its data at all, since reporting to the FBI is voluntary. The victimization survey exists precisely because police-reported data leaves out what victims do not report, and its findings show a large unreported share across many offenses.

This matters politically. If a city launches a campaign to encourage reporting, recorded crime can rise as the dark figure shrinks, and the city looks more dangerous when it has become more trustworthy. If a city discourages reporting, in whatever way, recorded crime can fall while real victimization does not.

The verdict. Reported crime is a floor, and a shifting one. Treat it as the crime the system heard about.

Claim four: "The national average tells you about your city"

The claim. "The national violent crime rate is down, so you have nothing to worry about," or the reverse.

The test. A national rate is an average over more than three thousand counties and many thousands of agencies. Crime is concentrated. Within a city, a few blocks often account for a large share of violence, and between cities the differences are enormous. The average can fall while your city rises, and the reverse. The victimization survey is designed to give national estimates, not city or neighborhood rates.

The FBI itself has long warned against ranking cities by crime totals, because a ranking offers no insight into the many local factors that shape crime in a particular town or city, and one agency can record more crime than a comparable one because its proactive efforts identify more.

The verdict. The national number is a weather report for the whole continent. It does not tell you whether to carry an umbrella in your town. Look up your own agency's data, and check whether it reported.

Claim five: "Crime rose by 100 percent"

The claim. A percentage change makes a dramatic headline.

The test. Percentage changes from small bases are close to meaningless. Here is a labelled hypothetical. A town of 20,000 people has two murders one year and four the next. The percentage change is 100 percent: (4 minus 2) divided by 2, times 100. Is that a crime wave? The town's murder rate per 100,000 residents went from 10 to 20, which is exact arithmetic: 2 divided by 20,000 is 0.0001, times 100,000 is 10, and 4 divided by 20,000 is 0.0002, times 100,000 is 20. That is a real increase, and also one that a single additional incident, or a single incident with several victims, can produce. A town of that size sees swings of that magnitude from chance alone.

Now the other direction: a city of 500,000 with 300 murders that falls to 270 has dropped by 10 percent, which is 30 fewer people dead. That is a bigger human difference than the 100 percent jump in the small town, and a much more reliable signal.

The verdict. Always ask for the count, the population and the rate. A percentage from a small base is a rumor with a decimal point. And that applies equally to the activist who says a category of offense has "tripled" and the official who says one has "plunged."

Claim six: "Clearance rates are a footnote"

The claim. What matters is how many crimes happen, and what fraction police solve is a detail for specialists.

The test. The clearance rate, the share of reported offenses the police close by arrest or by what the FBI calls "exceptional means," is among the most important numbers in the system. Deterrence depends on the probability of being caught. A criminal who believes arrest is unlikely will behave differently from one who believes it is likely, and the evidence from economists on policing, which I summarize in What Economists Found When They Counted Police Officers and Crimes, points toward certainty of apprehension as a stronger deterrent than severity of punishment.

A low clearance rate has a second cost: it tells victims that reporting is pointless, which feeds the unreported share that the victimization survey finds. So a falling clearance rate is a warning about both deterrence and trust. A city can have a falling crime count and a falling clearance rate at once, and a reader who looks only at the first misses a problem in the second.

One caution. "Cleared" does not mean "convicted." A case can be cleared by arrest and then dismissed. And clearance can be exceptional, which the FBI allows when the offender has been identified and there is enough evidence to support an arrest, but something outside the police's control prevents it. Read the definition.

The verdict. Clearance rates are a core indicator of whether the system works. Ignoring them is like judging a restaurant by how many people walk in and never asking whether anyone gets fed.

Claim seven: "The statistics tell us why"

The claim. Crime fell because of policing, or because of removing lead from gasoline, or because of the end of a drug epidemic, or because of the legalization of something, or because of whatever the speaker already believed.

The test. A table of offenses has no causes in it. It has counts, rates and dates. Causal claims require a research design: comparison groups, controls, a plausible mechanism and replication. Some of the big claims about the great crime decline of the 1990s are interesting hypotheses and nothing more, and serious researchers disagree about the weight of each. Anyone who says the numbers prove a single cause is selling something.

This applies to conservatives who credit tough policing for every decline and to progressives who credit social programs. Both may be partly right. The honest statement is the narrow one: here is what the evidence shows about this specific intervention, in this specific setting, and here is the uncertainty.

The verdict. Statistics describe. They do not explain, and the explanation has to be built separately and held loosely.

The two things they can tell you

After that list of limits, you might conclude that crime data is useless. It is not. Two kinds of evidence are solid enough to stand on.

One: homicide counts

A body is hard to hide. Murder is the most consistently reported and recorded offense, because the death is recorded by a medical examiner or coroner, by public health authorities in the death certificates that feed national mortality statistics, and by the police. Two independent systems count the same thing, and when they roughly agree, you can trust the trend. Homicide also tracks other violence. A city with a rising murder count almost always has rising shootings and aggravated assaults.

Its weaknesses are few. A murder count is a small number in small places, so a rate for a tiny jurisdiction swings wildly, as in claim five. And homicide is not all crime, so it says little about burglary or fraud. But as a reading on the worst thing that can happen to a person, it is the most reliable gauge we have.

Two: victimization surveys

The National Crime Victimization Survey does what police statistics cannot: it asks the people who were hurt. It does not depend on whether the police recorded the crime, and it uses a consistent design, so changes in its estimates are less likely to reflect changes in police practice. It also records why victims did not report, which is a direct measure of public confidence in the system. Its limits are real: it does not cover homicide or business crime, it does not interview children under twelve, and it carries sampling error. Used within those limits, it is the best check we have on the police numbers.

When these two sources agree with each other and with the police data on direction, you have found something durable. When they diverge, you have found a question worth asking.

What to do with all this

I would offer a short set of habits, in order.

1. When you see a crime claim, ask what is being counted, by whom, over what period, and for what place.
2. Ask for counts, populations and rates, not just percentages.
3. Check whether the agency reported all twelve months, and whether it changed systems.
4. Look at homicide first, and at the victimization survey second.
5. Distrust any claim of cause that comes with a single chart.

These habits do not favor either party. They favor accuracy, which is where a conservative who wants to win the argument over policy ought to want to be. A case against a policy such as the one in Defund the Police Was a Slogan Cities Tried and Then Quietly Walked Back is strongest when it relies on counts a hostile reader can verify, and it is weakest when it relies on a number that falls apart under a skeptic's first question.

The case in brief

1. A one-year change cannot prove a policy worked, because crime moves for many reasons and a bad year tends to be followed by a better one.
2. Arrests measure police activity, and reported crime measures what the system heard about, so neither is the same as actual crime.
3. National averages do not describe a particular city, percentage changes from small bases mislead, and clearance rates are a central indicator, not a footnote.
4. Statistics describe trends and do not explain causes.
5. Homicide counts and victimization surveys are the most solid ground, and when they agree with each other, trust the direction.

What you can do: the next time someone cites a crime statistic to you, ask for the source, the place and the time period before you accept it, and offer your own figures in the same form.