How often Americans use guns in self-defense and why the estimates differ

Two careful teams of researchers can ask Americans the same kind of question and publish answers that differ by a factor of more than twenty. One says defensive uses of guns number in the tens of thousands a year. The other says they number in the millions. Each side then quotes its favorite and calls the other side's number a fantasy, rather like two men arguing over the size of a fish neither of them weighed.

If you want to argue about guns without being fooled by anyone, and that includes me, you need to know why numbers this far apart can both come from honest work. This post explains it. Along the way you will pick up a toolkit that works on any statistic in any debate. The conclusion is one I will argue plainly: even the lowest credible estimate describes a real and substantial benefit, and a policy that ignores it has not done its arithmetic.

What counts as a defensive gun use

Start with a definition, because most of the dispute lives here. A defensive gun use, or DGU, is an occasion on which a person uses a firearm to protect himself, another person or property against what he believes to be a crime or an attack. Everything depends on the words "uses" and "believes."

Does "use" mean firing the gun? Almost certainly not, because in most defensive incidents no shot is fired. The attacker sees the weapon and leaves. Does it mean brandishing, or showing, or saying "I have a gun"? That is a defensive use to the person who did it and a non-event to a crime statistician. Does "believes" count if the intruder turns out to be a neighbor's drunk teenager? Researchers have to decide, and different decisions produce different numbers.

I raise this because the question is not "how many people shot an attacker." That number is small in any version of the data. The question is how often a gun's presence and use changes the outcome of a threatened crime, and that is a harder thing to measure.

Source one: the National Crime Victimization Survey

The first source is the National Crime Victimization Survey, the NCVS, run by the Census Bureau for the Bureau of Justice Statistics. It is the nation's principal survey of crime victims. Interviewers contact a large sample of households and ask about crimes the members experienced in the previous period, whether or not they reported them to the police. It is a serious survey with a long track record, and its estimates of burglary, robbery and assault are what most scholars use.

Its estimates of defensive gun use are low. Depending on the years and the analysis, the NCVS puts DGUs in the range of tens of thousands to roughly a hundred thousand per year in most analyses, and somewhat higher in a few. Put in daily terms, even sixty thousand a year is more than 160 a day. I will come back to that.

Why so low? Here the design matters. The NCVS is a survey about victimization. A respondent is first asked whether he or she was the victim of a crime. Questions about what the victim did to protect himself come afterward and only for people who reported a crime. That structure has consequences:

1. A defensive use that prevents a crime may never register as a crime. If a man shows a gun and the burglar runs, the man may not describe himself as having been burgled, and so never reaches the self-protection questions.
2. The survey does not ask the general public about gun use. It asks about crime victims, and it counts a gun use only when the person was already identified as a victim.
3. The reference period matters. The NCVS asks about a limited period, and people forget, or place events in the wrong period. This is a general feature of surveys, and I return to it below.
4. Some respondents are reluctant to tell a government interviewer that they used a gun, especially if the gun was unregistered or carried without a permit, or the incident was never reported to the police. That is a hypothesis about behavior, and a plausible one, but it is not a fact I can prove.

These are reasons to think the NCVS undercounts. They are not reasons to think it counts nothing.

Source two: Kleck and Gertz, 1995

The best-known high estimate comes from criminologists Gary Kleck and Marc Gertz. In 1995 they published "Armed Resistance to Crime: The Prevalence and Nature of Self-Defense with a Gun" in the Journal of Criminal Law and Criminology. Their National Self-Defense Survey was a telephone survey of nearly five thousand adults in the early 1990s, and it asked directly whether the respondent, or someone in the household, had used a gun to protect against a person during the previous year. From the answers they estimated about 2.5 million defensive uses per year, or about 2.1 million after an adjustment for telescoping.

That figure has been quoted for thirty years. It has also been attacked for thirty years, and the critics were not stupid. The central critique, made most forcefully by the public-health researcher David Hemenway, concerns what statisticians call false positives in rare-event surveys.

The rare-event problem, with arithmetic

Here is a way to see it. This is a hypothetical, labeled as one, and the arithmetic is exact.

Suppose a survey interviews 5,000 adults and the country has about 200 million adults. Each respondent then stands in for 200,000,000 divided by 5,000, which is 40,000 people. If 50 respondents say yes to a question, the survey's national estimate is 50 times 40,000, or 2 million.

Now suppose the true number of yes answers was 10, and 40 of the 50 were mistakes: a respondent who misunderstood the question, who was joking, who counted something that did not meet the definition. Then the true national figure is 10 times 40,000, or 400,000, and the survey overstated it by a factor of five. Forty mistaken answers out of five thousand is less than one percent of the sample.

That is the problem in a nutshell. When the event you are measuring is rare, a very small rate of error in a large majority of respondents can swamp the signal. Every "yes" is multiplied by tens of thousands. Hemenway argued that false positives of this sort explain the very high estimates.

Kleck and his defenders answer that there are false negatives too, people who used a gun and will not say so, and they argued that their survey's design, the screening questions, and follow-up checks limited the errors. Each side can point to real features of the data. Neither can prove the other's number wrong, which is exactly what makes the dispute durable.

Why the same question yields different answers

Pull the threads together. The estimates differ for reasons that each researcher can name.

• Definition. What counts as a use: a shot, a display, a threat? The NCVS counts a gun used for self-protection in an event that qualifies as a crime. Kleck's survey counts any use against a person, as the respondent understood it.
• Population. Victims only, or all adults? The NCVS counts among those who report a victimization, and general surveys ask everyone.
• Recall period. A six-month window and a twelve-month window produce different forgetting. People tend to pull events forward in time, a distortion called "telescoping," that makes old incidents look recent and inflates annual counts.
• Interview mode and context. A government interviewer who has just asked about crime victimization is a different prompt from a telephone survey about self-defense, and respondents react to the framing.
• Rarity. Because the underlying event is rare, small error rates produce large swings, as the arithmetic showed.
• Weighting. A few cases are weighted up to a national total, so the estimates have wide margins of error, and the published ranges reflect them.

If you remember one thing from this post, remember that a statistic about a rare event is only as good as the definition behind it and the survey method that produced it. I would apply the same rule to every number on crime and on guns, including the favorable ones. The post on nine myths about gun shows, private sales and the loophole that is not one shows the same habit at work in a different debate.

The 2013 National Research Council report

In January 2013, after the Sandy Hook school shooting, President Obama directed the Centers for Disease Control and Prevention to commission a review of the research on gun violence. The CDC asked the Institute of Medicine and the National Research Council, and the committee published Priorities for Research to Reduce the Threat of Firearm-Related Violence later that year.

The report is a useful referee because it was written by a committee of scholars at the National Academies, not by an advocacy group. On defensive gun use, it said that most national surveys produce estimates ranging from about half a million to more than three million per year, while the NCVS produces a radically lower estimate. It also said that most national surveys suggest defensive uses are at least as common as criminal uses of firearms, called the variation in the estimates "a controversy in the field," and noted that the 3 million figure rests on an extrapolation from a small number of responses and that the NCVS figure of 108,000 is hard to interpret because respondents were not asked specifically about defensive gun use. I would ask you to read its discussion rather than take my paraphrase, since the wording matters, and the report is free online.

What the committee did not do was say defensive gun use is a myth. What it said was that the numbers are uncertain and the research is thin, and that better data are needed. That is a fair statement of where things stand.

Even the lowest estimate matters

Now to the argument I promised. Suppose the NCVS is exactly right and the critics are exactly right about the high estimates. Suppose, for the sake of argument, that defensive gun uses in America number about sixty thousand a year. What follows?

1. Sixty thousand a year is over 160 a day. Every day, somewhere in the country, about 160 people use a gun to protect themselves or someone else from a crime. That is not a rounding error. Fill a modest football stadium every year and you are in the neighborhood, and no one calls a stadium a myth.
2. Each of those is a person. A statistical benefit is a woman who was not raped, a shopkeeper who was not beaten, a family that was not harmed. The cases that never become headlines are the ones in which the defender did his job so well that nobody was hurt.
3. The deterrent effect is not captured. A criminal who does not attempt a burglary because the house might be occupied by an armed resident does not appear in any survey, since no crime occurred. The NCVS cannot count the crime that was never tried. I do not claim a number for this, because nobody has one. I claim only that the direction of the omission is clear: the true benefit exceeds any count of incidents.
4. A right is not a statistic. The Second Amendment protects the right to keep and bear arms, and District of Columbia v. Heller (2008) held that the core of the right is self-defense in the home. The Constitution does not ask whether defensive uses exceed some threshold. But where policy is being weighed, the benefit side of the ledger is not empty.
5. Policy must weigh both sides. A person proposing to restrict guns has to account for the costs of the restriction as well as the benefits. If defensive uses are even in the tens of thousands, a policy that reduces them has a cost that should appear in the analysis, and it rarely appears. When gun restrictions are discussed, the costs are usually described as "inconvenience." Whether that is fair to a person defending a life is the question.

The best counterargument

A fair critic replies that guns in the home are also used in suicides and in accidents, and that the total harm outweighs the benefit. That is a serious claim, and it belongs in the ledger. It is not, however, an answer to my point. My point is only that the benefit is real and must be counted, not that the harms are zero. Honest accounting puts both columns on the page. The suicide and domestic-violence claims deserve their own treatment, and the argument about who benefits most from a firearm is in the equalizer, why a firearm matters most to the smallest person in the room.

I would add a modest observation about the conservative approach to this. We say we prefer policy built on evidence. Then let us hold ourselves to it. We should not repeat the 2.5 million figure as though it were established, because the criticisms of it are serious. We should say what is true: the best estimates are uncertain, the range is wide, and the low end of it is still large. A person who understands the data has a stronger argument than a person who quotes the biggest number.

How to judge any gun statistic

Here is the toolkit. Ask these questions of any figure you meet, from either side.

1. Who counted, and how? A survey, a police record, a hospital database or a news search each captures a different slice.
2. What does the number include? Defensive uses and shootings, suicides and homicides, accidents and intentional acts are different categories, and merging them is a favorite trick.
3. How big is the sample, and how rare is the event? Rare events and small samples produce wide error margins.
4. What period and population? A figure for one city in one year does not describe a country.
5. What is missing? A crime prevented, a death that was not recorded as a gun death, a use that went unreported.
6. Who is quoting it, and what do they want? Everyone has an interest. Read the source, not the press release.

These are the same questions I try to ask in the post on What the Justice Department's own studies found about the 1994 ban, and I think they would improve most arguments, on any subject.

The case in brief

1. A defensive gun use is a use of a firearm to protect against a perceived crime, and most such uses involve no shot being fired.
2. The National Crime Victimization Survey finds tens of thousands to roughly a hundred thousand a year, and Kleck and Gertz's 1995 survey found about 2.5 million, and the difference comes from definitions, populations, recall periods and the arithmetic of rare events.
3. The 2013 National Research Council report commissioned through the CDC found the estimates ranged widely, called the evidence limited and did not call defensive use a myth.
4. Even at sixty thousand a year, defensive use means more than 160 people a day, and the deterrent effect on crimes never attempted is unmeasured.
5. Policy must weigh the benefit alongside the harm, and a person who knows the data is more persuasive than one who quotes the largest number.

What you can do: when someone quotes a gun statistic at you, ask for the source and the definition, then look up the Bureau of Justice Statistics report on the National Crime Victimization Survey and read how it asks the question.