Why is the greatest nation in the world shockingly bad at recording crime data?

Read a longer, technical methodology of how we worked with the UCR data here.

When the head of America’s top law enforcement body admits to not having enough data to stay on top of violent crime, you know you’ve got a problem.

“We face a data shortage on the violent crime front,” James Comey, the FBI’s director, told the International Association of Chiefs of Police at a conference late last year. “We can’t tell you on a national level how many shootings there were in any particular city last weekend.” Comey went on to say that without good data, every single conversation about policing, reform, and justice is uninformed.

It’s an argument criminologists and social justice advocates have been pushing for more than two decades. Criminologists like Charles Wellford at the University of Maryland have argued that inaccurate crime statistics lead to skewed policing, which in turn leads to further inequality and discrimination and, in some cases, neglect.

At the moment, there is just one relatively comprehensive, widely used system for collecting national crime statistics. It’s called the Unified Crime Reporting System, or the UCR database. This data is the most official and representative public snapshot of policing throughout the U.S., but it has deeply troubling shortcomings.

The UCR was first implemented in 1929, purportedly as a way to prevent the media from making up stories about “crime waves.” Not much has changed in the 85 years since: The UCR is still being used to determine policies and control public discourse about crime, even though it’s known to be bad data.

Why use it, then? Simply put, because there’s nothing else. The UCR and a component of that database called NIBRS are the only official national public resources for analyzing crime patterns (down to cities and counties). NIBRS is also voluntary (only about 35% of police departments participate), slow, and prone to manipulation.

Here are the reasons we think the UCR has outlived its usefulness:

  1. The UCR program is voluntary.

About 30% of police departments in the country don’t contribute to it, leaving massive holes and making it virtually impossible to accurately analyze the data. In addition, police departments aggregate their own numbers and then submit them to the FBI. This makes it easy to downgrade serious crimes or manipulate the statistics for individual agencies’ benefit.

  1. The UCR is archaic.

The whole thing is formatted in 1970s mainframe binary type, making it impenetrable—even for those of us who are mildly data savvy. One must fire up one’s Mac IIE system in order to process or analyze anything. This is a key explanation for why the FBI takes so long to release the data, as well. The 2014 UCR stats were only released in September 2015, and that release was incomplete. That means we’re all working with outdated information.

  1. The UCR isn’t consistent.

The whole database is divided into two sections. Part 1 offenses are more severe and violent—they include forcible rape, robbery, and homicide—and Part 2 offenses are minor crimes, like disorderly conduct. The issue here is that while Part 1 offenses include all related incidents—arrest or no arrest—Part 2 offenses don’t. Another issue is the lack of specificity in certain categories. Often crimes are shoehorned into general categories—carjacking, street muggings, or breaking-and-enterings are all just recorded as “robberies.” Similarly, there is little differentiation between weapons used in an assault. An attack with a knife is recorded the same way as an assault with a semi-automatic gun.

So if the data is that bad, why did we choose to do a project on it? It’s all we have. We would like this project to reflect the publicly reported activities of America’s police forces. But we also want to highlight the glaring need for real, up-to-date data about what our police are really doing to protect us.

This content was made possible in part by a grant from the Ford Foundation and produced independently by Fusion’s editorial staff. To find out more, explore our interactive map and read more of our coverage of America’s dysfunctional system of policing.