Fatal Encounters is a complex and rigorous project that uses several processes of data collection to ensure a high level of validity. Media news sources have predominantly focused on the crowdsourcing aspects of our project. While some of our data is crowdsourced, we have three main methods of collecting information. They are listed below in order of numbers of records in the database:
1) Paid researchers; 2) Public records requests; 3) Crowdsourced data.
Out of the 6,900 documents we have on June 15, 2015 around 85 percent have been submitted by researchers we pay to log data. Column Q lists who logged the data.
Our paid researchers have several methods of getting information into the verification queue. First we aggregate data from other large sets like KilledByPolice.net or the Los Angeles Times’ The Homicide Report and individuals like Carla DeCeros who have contributed their data to FE. They then research the missing information and double-check the information that’s included. When the record is complete, it’s moved over to the verification queue, where it is again checked against published sources yet again by the Principal Investigator of FE.
When an incident is reported by a volunteer—the crowd—every fact presented is compared to published media reports or public records to verify its accuracy. This information from any source–a hometown newspaper, for example–and submitted it through our form. Once submitted, it goes to a separate spreadsheet, where we verify its information against media sources.
We have also been conducting research by state and by date. These methods are intended to be redundant so that we catch as many incidents as possible. However, we know from experience that incidents have been missed, sometimes because the death was not reported at the time it happened, through human error, or just because of the vagaries of the internet. To address this issue, FE and our sister project, EncuentrosMortales.org, have made more than 2,300 public records requests of state, federal and local law enforcement agencies. This part of the process is extremely expensive, but the documents are useful as yet another level of redundancy. Other researchers, such as Lance Farman have also been testing the completeness of the database against FOIA requests and have found that this method yields a 97% completeness rate for 11 of the states that have been logged so far.
D. Brian Burghart is the principal manager of FE. He is a newspaper editor with more than 20 years of experience. Walt Lockley is the lead researcher for FE. Most researchers are paid $3.50 per record. Since nearly all of our data is an analysis of media sources, a random sample of the data will be intercoded, and we will apply a Krippendorff’s intercoder reliability test to ensure that the observed differences between coders are caused by chance and not through systemic researcher bias. Once complete, the FE database will not only be the largest and most complete in the United States, but also the most methodologically rigorous.