As anyone who’s been following the news over the past few years knows, police killings of civilians, especially unarmed black ones, has become a highly contentious issue. Questions about the extent to which these shootings were justified, whether police are racially biased in their dealings with blacks, whether the chant, from those protesting these events, that “black lives matter” is itself a racially biased slogan, and a host of others have created a fissure in our social fabric. And, starting in Dallas, we’ve had a few high profile shootings of police officers which seem to have “fanned the flames.” This is the context within which a working paper by economist Roland Fryer appears.

Fryer, in a paper entitled An Empirical Analysis of Racial Differences in Police Use of Force, makes a valiant attempt to bring data to bear on this highly contentious issue. The paper is long and technical, but I’ll provide a short overview of what it says and, as importantly, what it doesn’t say.

The first thing to note is that Fryer’s data comes from several sources: 1) NYC’s Stop and Frisk data 2) The Police-Public Contact Survey (PPCS) and 3) data on police shootings from Dallas, Austin, Houston, six Florida counties, and Los Angeles. Only one of these sources, PPCS, is based on a nationally representative sample.

Using NYC’s Stop and Frisk data, as well as those from the PPCS, Fryer’s statistical models found that blacks are more likely to suffer non-lethal force from police than whites are. Non-lethal force includes such things as pushing/shoving, forcing someone onto the ground or up against a wall, drawing (but not using) a weapon, using pepper spray, etc.

When it came to police shootings, however, Fryer found something which surprised him: the probability of being shot by police at all, as well as the probability of being shot by an officer before having attacked that officer did not vary by race. That is, blacks weren’t statistically significantly more likely to be shot by police than whites. As I said, this finding surprised Fryer and may be as surprising to the rest of us who’ve heard the names Michael Brown, Tamir Rice, Philando Castile, and other black people who, over the past few years, have been shot by police.

We should be careful, however, before concluding too much from these findings, something Fryer is well aware of but which may get lost in media discussions of his work. One thing we should definitely not do is conclude that there’s no racial bias in the U.S. when it comes to police shootings. We shouldn’t do this for several reasons.

One goes back to Fryer’s data sources. As I said earlier, the only nationally representative data set Fryer used was the Police-Public Contact Survey. His other sources, and the ones on which his conclusions about police shootings are based, were from police precincts in 10 cities; and those precincts chose to make their data available to him. The problem this raises is that what’s happening with police shootings in these cities may not be representative of what’s happening in the nation as a whole.

A related problem is the possibility that only the more “enlightened” precincts made their data available. In other words, those precincts more likely to have been engaged in racially biased shootings may have been less likely to grant Fryer access to their data. If this were the case, his findings could provide a distorted view of what’s really going on.

Assuming precincts which granted Fryer access to their data are no more or less enlightened than those which didn’t, another possibility is that the data are inaccurate. Those data on which the conclusions about police shootings are based came from extensive reports written by officers involved in the shootings. As Fryer notes, the negative consequences of being involved in an unjustified shooting are serious enough to provide a powerful incentive for police officers involved in shootings to misrepresent what actually happened, including any racial bias that may have affected their decision making.

The nation is currently facing a difficult period when it comes to relations between police and many members of the black community. Tensions are high, and it’s very easy for intense emotions to dominate public discourse. I applaud Fryer’s attempt to bring data into the conversation. But, given the limitations of the data sets he used,  Fryer’s findings ultimately raise as many questions as they’ve answered; they certainly aren’t the last word on the subject of race and police shootings, something  he readily acknowledges.

This post appeared as part of a Roosevelt House series on race and law enforcement. To read the rest of the series, click here.

Michael A. Lewis is a social worker, sociologist, and former community organizer. He currently teaches courses in economics and public policy and the Silberman School of Social Work at Hunter College.