In a recently published Letter to the Editor in his own journal, Michael Peat, Editor-in-Chief of the Journal of Forensic Sciences, said he is disappointed by the “emotional” nature of letters submitted in response to a February 2021 study by Itiel Dror et al., that claims to show the potential for racial cognitive bias in the forensic pathology field.
The controversy over the study has continued to escalate and turned personal with ad hominem attacks against Dror and his co-authors. The National Association of Medical Examiners (NAME) even filed a formal complaint to the University College London where Dror is employed. The complaint was investigated and ultimately dismissed.
“It was not disappointing to me that this article received such scrutiny; however, the ‘emotional’ nature of some of the letters was,” Peat writes in his letter. “I certainly recognize that we all have different opinions and we all have biases. However, it is important that we discuss these issues openly and professionally and we encourage all concerned to do so. In my opinion, discussion is the best means for resolving issues, both in science and elsewhere in today’s world. We should not lose sight of that.”
The original article by cognitive expert Dror and a handful of senior forensic pathologists received 9 commentaries, all of which the authors answered. The Journal of Forensic Sciences (JFS) then allowed Dror et al., and authors of two of the original commentaries to respond back a second time, resulting in a total of 22 letters. You can find all of them, in addition to Peat’s letter, here.
Three of the commentaries demanded JFS retract the Dror study, with Peterson et. al., going as far as to say the journal risked “ceding its reputation from advancing objectivity and rigorous scientific method in forensic science to promoting agenda-driven editorial content disguised as medical literature.” Two letters from NAME demanded retraction of the study, as well. In his letter, Peat confirms that the journal took the retraction demands seriously, asking Wiley’s Integrity in Publishing Group and the Committee on Publication Ethics, as well as the University of Alabama at Birmingham’s Institutional Review Board, to investigate. None of the investigations found grounds to support a retraction of the original article.
The original
To best understand the controversary, we must start at the beginning—the original study published on Feb. 20, 2021 in the Journal of Forensic Sciences. The purpose of the study was to examine if medically irrelevant information can influence cognitive bias, affecting forensic pathologists’ decision-making.
To do this, Dror and team looked at manner of death on all death certificates issued during a 10-year period in the state of Nevada for children under the age of 6. They also sent out an electronic survey featuring a hypothetical case to 713 pathologists on the NAME mailing list, of which 133 answered (18.6% response rate).
Survey participants read a vignette describing a 3.5-year-old child who arrived at the hospital with diminished vital signs, and died shortly thereafter. In the vignette, the caretaker described finding the toddler unresponsive on the floor of a living room. Postmortem examination determined that the toddler had a skull fracture and subarachnoid hemorrhage of the brain.
All survey respondents read the same vignette with the exception of two pieces of “medically irrelevant” information: 65 pathologists were told the child was Black and the caretaker was the mother’s boyfriend, while 68 pathologists were told the child was white and the caretaker was the child’s grandmother.
Of the 55 participants who reached a conclusive determination about the manner of death (78 ruled the manner undermined; 37 vs 40 for Black vs white victim, respectively), those who read the victim was Black were about 5x more likely to rule the death a homicide rather than an accident. Those who read the white-child vignette ruled the death a homicide only about half as often as they ruled it an accident.
Dror et al., concluded that these findings demonstrate how extraneous information—the race of the child victim and who their caretaker was at the time—can result in cognitive bias in forensic pathology decision-making.
The responses
Anatomical pathologists vs forensic pathologists
While examination of all 22 letters is outside the scope of this Forensic article, we will look the criticisms that were repeated in more than one letter. Most of the negative response commentaries discuss problems with the survey.
For example, letters from Gill (the president of NAME) et al., and Speth et al., claim the study’s findings are inaccurate since the survey was sent to 713 anatomic pathologists, rather than 713 forensic pathologists. But Dror et al., say the study is interested in showing bias in manner of death decisions, regardless of who makes them. And even if all the responses were from board-certified forensic pathologists, the bias could still be there. Dror and co-authors point out that believing forensic pathologists are less biased than coroners and other manner of death decision-makers falls into the “fallacy of expert immunity.”
“Expertise has no protection from bias, and forensic experts have been shown to be just as susceptible to bias as novices,” Dror and co-authors write in their response letter to Speth et al.
Medically irrelevant vs medically relevant
Responses by Peterson et. al., and Oliver (who is a co-author on the Peterson et al. response) highlight the issue of medically relevant vs medically irrelevant information. In the hypothetical scenarios relayed to pathologists in the original study, one Black child victim was in the care of the mother’s boyfriend, while the second white child victim was in the care of a grandmother when the deaths occurred. Dror and his team classify the caregiver relationship to the child victim as “medically irrelevant contextual information.”
Peterson et. al., adamantly disagree, citing literature and studies that show stepparents and biologically unrelated caregivers are much more likely to abuse a child than a biological grandmother. While caregiver relationship may not be relevant, they argue, it is important in manner of death determination.
However, Dror and his co-authors say this viewpoint is actually a known bias itself—the ecological fallacy, which results from making a casual inference about individual phenomena on the basis of observation of groups.
“Our main point is that all else (e.g., injuries, case details) being identical, it is in our view wrong, unacceptable and biased to determine that the death of a child in an equivocal case was a homicide or accident purely by the nonmedical irrelevant information of who the caregiver was,” write Dror et. al, in their response to Peterson et. al.
Race vs relationship
Peterson and his co-authors have an extremely strong stance on the fact that Dror and his team made one victim Black and one white in the hypothetical cases, going so far as to claim the original authors are trying to paint the entire forensic pathology field as racist.
“The focus on race moves the construction of the study from inexplicable to absurd,” they write in their commentary. “To introduce race in an obscure fashion appears as an effort to label the survey responders, and their colleagues by proxy, as racists.”
While Peterson and his co-authors acknowledge there may be unconscious race bias in the field of forensic pathology, they say the caretaker relationship in the Dror study does not provide evidence of such.
To an extent, Dror et al. agree with that statement, as they say their research question was whether forensic pathology decisions were biased by nonmedical, irrelevant information in general, not race specifically. In fact, the authors acknowledged this in their original paper.
“The data do not allow us to ascertain whether [the pathologists] were biased by the race of the child or/and characteristics of the caretaker,” they wrote. “The important finding is that their decisions were noticeably affected by medically irrelevant contextual information. The degree and the detailed nature of these biasing effects require further research, but establishing biases in forensic pathology decision-making—the first study to do so—is not diminished by the potential limitation of not knowing which specific irrelevant information biased them (the race of the child or/and the nature of the caretaker).”
The take home message
While there are other responses and responses to responses, they all mostly fall into criticisms of the survey, intended audience, race, and manner of death, as described above. Obenson, Graber and Duflou authored (mostly) positive responses to the original study, with Duflou agreeing with JFS editor Peat on the motivations behind some of the passionate responses.
“Although many negative comments have been made in relation to this study, these are largely unwarranted and not based on a detailed analysis of the material presented and conclusions drawn,” writes Duflou. “The important take-home message of the paper is that as forensic pathologists who consider a large amount of information, much of which is not specifically medical in nature, we need to acknowledge to ourselves that bias exists and that it is the very knowledge of its existence and our attempts to minimize its impact which will make us better and more impartial expert witnesses.”
Photo credit: Wiley