On the heels of a peer-reviewed study that concludes ShotSpotter may be of little benefit to police agencies and does not reduce violent crime, VICE News has taken aim at the gunshot detection system, alleging that police are altering evidence—a claim ShotSpotter has vehemently denied, calling the allegations false and misleading.
The study
Published in June in the Journal of Experimental Criminology, Dennis Mares and Emily Blackburn evaluated the ability of St. Louis’ ShotSpotter to reduce gun violence.
“Despite rapid growth and substantial investments in acoustic gunshot detection systems (AGDS), remarkably few academic evaluation studies on gunshot detection systems have been published,” write Mares, a professor of criminal justice studies at Southern Illinois University, and Blackburn, Manager, Crime Analysis Unit with the St. Louis Metropolitan Police Department (SLMPD).
In fact, the two collaborated on one of the first papers in 2012. (This lack of supportive evidence is referenced by VICE News.)
In the 2021 study, Mares and Blackburn used data from SLMPD for the first implementation of ShotSpotter from August 2008 to March 2013, as well as the expansion from March 2013 to February 2018.
The most striking find of the study is the effect of AGDS on citizen-initiated reports for shot fired. According to the study, St. Louis saw a 32 percent decline of citizen-initiated reports throughout the first AGDS implementation, and 27 percent after the expansion of 2013. However, during AGDS interruption in 2016, citizen reports for shots fired bounced back 33 percent.
This is important because the researchers found that citizen-initiated calls for service are over seven times more efficient in uncovering and responding to criminal behavior than ShotSpotter notifications.
“We calculate that for every 100 citizen-initiated calls for shots fired, 7.6 cases are generated using 42.7 hours of labor; for each 100 AGDS hits, .9 cases are generated at a cost of 36 hours,” the researchers write.
Piggybacking off those findings, Mares and Blackburn found little to no evidence that AGDS calls in St. Louis uncover crimes or criminals, nor do they help reduce the number of crimes.
“Given the proportionate difference in generating actionable police work between citizen-initiated shots fired and AGDS calls for service, we believe a much better and cost-efficient approach would be to educate the public on how and when to call police for gunfire, and to train 911 operators to sort gunfire in response hierarchies,” the researchers conclude.
But, they are careful to say this does not mean AGDS is without merit in other cities. St. Louis, for example, has a relatively barebones version with notifications relayed through its dispatch system. Meanwhile, Chicago integrates its ShotSpotter with real-time notification of officers in the field and risk terrain modeling.
And it’s Chicago that is at the crux of the VICE report.
VICE vs ShotSpotter
The VICE article centers on the shooting death of 25-year-old Safarain Herring. Michael Williams brought Herring to the ER, claiming he was shot in a drive-by. However, Chicago police eventually arrested the 64-year-old Williams for Herring’s murder, saying ShotSpotter could prove there were gunshots in the area in which Williams’ car was captured on camera.
According to VICE, the night of the murder, 19 ShotSpotter sensors detected a percussive sound at 11:46 p.m. about a mile away from the site where prosecutors say Williams committed the murder.
“But after the 11:46 p.m. alert came in, a ShotSpotter analyst manually overrode the algorithms and ‘reclassified’ the sound as a gunshot. Then, months later and after ‘post-processing,’ another ShotSpotter analyst changed the alert’s coordinates to a location on South Stony Island Drive near where Williams’ car was seen on camera,” Todd Feathers writes in the VICE article.
When the defense requested a Frye hearing, the prosecutors withdrew all ShotSpotter evidence against Williams.
ShotSpotter says the prosecutor didn’t withdraw the evidence because he knew it wouldn’t win a Frye trial; rather, the prosecutor’s theory of what happened could not be supported by ShotSpotter evidence, thus the prosecutor chose to withdraw said evidence.
Had the Cook County State’s Attorney’s office not withdrawn the evidence in the Williams case, it would likely have become the first time an Illinois court formally examined the science and source code behind ShotSpotter, Jonathan Manes, an attorney at the MacArthur Justice Center, told VICE.
“Rather than defend the evidence, [prosecutors] just ran away from it,” he said. “Right now, nobody outside of ShotSpotter has ever been able to look under the hood and audit this technology. We wouldn’t let forensic crime labs use a DNA test that hadn’t been vetted and audited.”
(If this sounds familiar, it’s because the issue of source codes has come up in relation to probabilistic genotyping software like STRmix and TrueAllele. However, in these cases, many peer-reviewed studies have been undertaken, although the controversy is alive and well.)
In ShotSpotter’s case, evidence was deemed admissible in 10 Frye challenges and 1 Daubert challenge. Thus far, ShotSpotter evidence and expert witness testimony have been successfully admitted in 190 court cases in 20 states.
Still, trying to show a “pattern of alterations,” VICE referenced another case. In 2016, Rochester (New York) police looking for a suspicious vehicle stopped the wrong car and shot the passenger, Silvon Simmons, in the back three times. They charged him with firing first at officers. According to VICE, the only evidence against Simmons came from ShotSpotter.
Initially, the new organization says, Shotspotters sensors didn’t detect any gunshots, and the algorithms ruled that the sounds came from helicopter rotors. But after Rochester police contacted ShotSpotter, an analyst ruled that there had been four gunshots—the number of times police fired at Simmons. Then, Rochester police asked again and Paul Greene, SpotShotter employee and expert witness, testified that he found a fifth shot.
The audio featuring the fifth shot, however, was lost, leading a jury to acquit Simmons of attempted murder. The judge also overturned his conviction for possession of a gun, stating ShotSpotter was unreliabile in ths case.
SpotShotter accuses VICE of “misconstruing” the facts of the Simmons case.
“The audio files ShotSpotter recorded and used during the trial were secured and preserved using industry-standard forensic procedures. Audio files submitted as evidence were reviewed by our forensic analysts to create a court-admissible forensic report. They were never altered by ShotSpotter. We are currently engaged in a lawsuit and are vigorously defending our position,” reads a statement from the company.
VICE vs ShotSpotter’s Expert Witness
In addition to the Williams and Simmons cases, VICE details the Ernesto Godinez case, which occurred in 2018 in Chicago. But, the news organization specifically mentions the case in relation to the role Greene played.
The evidence against Godinez, who was charged with shooting a federal agent, included a report from ShotSpotter stating that seven shots had been fired at the scene, including five from the vicinity of a doorway where video surveillance showed Godinez to be standing and near where shell casings were later found.
During the trial, Greene testified under cross-examination that the initial ShotSpotter alert only indicated two gunshots—those fired by an officer in response to the original shooting. But after Chicago police contacted ShotSpotter, Greene re-analyzed the audio files.
“An hour or so after the incident occurred, we were contacted by Chicago PD and asked to search for—essentially, search for additional audio clips. And this does happen on a semi-regular basis with all of our customers,” Greene told the court, according to a transcript of the trial. He later ruled that there were five additional gunshots that the company’s algorithms did not pick up.
While not specifically addressing the Godinez case, ShotSpotter said its court-admissible analysis of a gunfire incident as an additional service the company provides.
“The detailed forensic report is never altered because it is a completely separate process from the alerts. Forensic analysis may uncover additional information relative to a real-time alert, such as more rounds fired or an updated timing or location upon more thorough investigation by forensic analysis. We respond to requests to further investigate an incident for a forensic report only to provide the facts that we can determine and not to fit a predetermined narrative,” the company said.
Lastly, the VICE article claims the stated accuracy of ShotSpotter has increased over the years and may not be entirely scientific. The news organization quotes Greene telling a San Francisco court in 2017: “Our guarantee was put together by our sales and marketing department, not our engineers. We need to give them [customers] a number … We have to tell them something. … It’s not perfect. The dot on the map is simply a starting point.”
However, ShotSpotter says that is a twisting of words and their stated 97% accuracy rate is a figure derived directly from police department reports throughout 2019 and 2020.
In May, the MacArthur Justice Center analyzed ShotSpotter data and found that over a 21-month period, 89 percent of the alerts the technology generated in Chicago led to no evidence of a gun crime, and 86 percent of the alerts led to no evidence a crime had been committed at all.
And while the St. Louis peer-reviewed study examined something different—ShotSpotter’s ability or inability to reduce gun violence—recent stats from Chicago show the city may follow St. Louis. The number of shootings in the windy city in 2021 is on pace to be the highest in four years.