Monday, October 27, 2008

Lies, Damn Lies, and Statistics

FBI report: Anti-gay crimes up - USATODAY.com You may have seen several news stories around today (USA Today is pretty typical) that scream about anti-gay hate crimes being up in 2007 over the previous year.
There were 7,624 hate crimes reported in 2007, down 1% from 2006. Crimes based on sexual orientation — 1,265 in 2007 — have been rising since 2005. A hate crime is one motivated by bias against a person's race, religion, sexual orientation or other status.
There is however a small problem on the data collection side.

These reports are interesting in that they help law enforcement understand the magnitude and proportion of these crimes, but year-over-year comparisons are mostly meaningless, and location-to-location comparisons are problematic to say the least. The FBI refrains from such comparisons and warns against them.

Very few law enforcement agencies report hate crimes. It was even fewer just a few years ago. Of those that do report, the mechanisms whereby crimes are reported as bias crimes can be a bit questionable. These make some of the numbers suspect, and cross agency comparisons meaningless.

For example, the 2006 UCR Hate Crimes report (released in late 2007 - the FBI is always about a year late) recorded one (yes 1) hate crime in Alabama, the infamous noose-hanging of the "Free the Jena Six" incident. Do you think there was only 1 race-related, and no religious or sexual-orientation hate crimes in Alabama in all of 2006? Really? There is this bridge you might be interested in....

In 2007 only about 15% of law enforcement agencies reported hate crimes. As that number increases (or if it were to decrease) the total number of crimes will probably change accordingly. And even when the agency reports, it is sometimes questionable how certain crimes do or don't get classified as a hate crime. This can be a genuine disagreement about classification, or a simple paperwork error. In any event, this systematic problem with the data makes - as the FBI usually tries to make clear - comparisons between locales questionable, and in this case makes cross-year trending difficult or impossible.
Valid assessments about crime, including hate crime, are possible only with careful study and analysis of the various conditions affecting each local law enforcement jurisdiction. (See Variables Affecting Crime.) In addition, some data in this publication may not be comparable to those in prior editions of Hate Crime Statistics because of differing levels of participation from year to year. Therefore, the reader is cautioned against making simplistic comparisons between the statistical data of this program and that of others with differing methodologies or even comparing individual reporting units solely on the basis of their agency type.
But that hasn't stopped the mainstream press from making those "simplistic comparisons" and screaming about the results.

Now I have no idea whether hate crimes are increasing or decreasing. I know they are under reported, (or at least categorized incorrectly) in many areas. And I know that back in the day, some cops didn't think beating up gays was a crime, and that there are probably still some on the job today. Do you think they are careful about filling in the Hate Crimes reporting forms? What I do know, is that you can't make simple comparisons about complex statistical reports unless you are only interested in the propaganda factor. But hey, this is the mainstream media. Of course I don't imagine that most journalism majors took any statistics classes in college. (Bias or stupidity? Tough call.)

Oh, the good news about that 15% of agencies that report hate crimes? They cover a little more than 85% of the US population. See the whole thing at the FBI website. It is a bit fragmented, but it is an improvement over a few years back when all they published was a single, very large PDF file.

No comments: