Some Thoughts on Quality Concerns in Crime Laboratories
According to the much anticipated crime laboratory study report released by the National Academy of Science (NAS) on February 18, 2009, a “badly fragmented” forensic science system needs an overhaul. The report raises questions as to the scientific validity of fingerprint identification, firearms identification, and bite mark identification in particular, pointing to the exhaustive baseline testing and the stringent training requirements and examination protocol associated with nuclear DNA testing.
The report states that it is beyond the scope of the courts to be able to effectively regulate the scientific reliability of scientific testimony. Recommendations include mandatory accreditation of crime labs and certification of individual examiners. Removing crime labs from law enforcement agency control so as to reduce confirmation bias is proposed. Overall control of crime lab policies and procedures should be standardized through the creation of a National Institute of Forensic Science, according to the report.
Some of the Issues
One of the determining factors relating to the admissibility of scientific testimony in court proceedings, according to the Daubert decision (Daubert v. Merrill Dow Pharmaceuticals, 509, U.S. 579, 1993), is the error rate associated with a particular procedure. This has proven to be elusive within many disciplines for a number of reasons. First and foremost, it is virtually impossible to know when a finding is in error and when it is not since it is only very rarely possible to independently establish the true source of evidence (i.e. fired bullets, fingerprints, footwear impressions, etc.).
The latent fingerprint community has long subscribed to a zero rate of error for fingerprint identification. The basis for this declaration has simply been that no two fingerprints from different sources have been identical. This line of reasoning has been challenged both in the literature and in court. As pointed out in the NAS report, adequate research is lacking in this area.
With the advent of formal proficiency testing programs, such as that offered by Collaborative Testing Services beginning in 1971, some laboratories have attempted to relate error rate to proficiency test results. Given the fact that examiners know when they are being tested, it is pretty naïve to presume that the results of such testing are in any way representative of actual error rate. At best, such rates might be categorized as the minimum error rate, especially given the potential for test compromise. Clearly, blind tests are needed.
The obvious problem with constructing totally blind tests in general is the associated paper trail that is required. That becomes more difficult to pull off in smaller labs, especially those in which there is typically interaction between the analyst and the case officer. Basically, a conspiracy of sorts would be required, due to the typical number of involved individuals in a real case, for blind testing to work in those situations.
Laboratory accreditation is mostly voluntary and is available through the American Society of Crime Laboratory Directors Laboratory Accreditation Board (ASCLD-LAB). Accreditation is granted to the organization, not the individual analysts. Texas, Oklahoma, and New York have required accreditation. When a forensic scientist leaves an accredited lab and goes to work in an unaccredited lab (approximately 80% of the crime labs in this country are accredited by ASCLD-LAB according to the NAS report of 02/18/09), the accreditation does not go with them. This serves as the impetus for certification of individual forensic scientists in addition to accrediting the organization to which they belong. Certification, like accreditation, is not currently required by most crime labs in the country.
Certification is available through a variety of specialty certification programs such as the American Board of Criminalistics (ABC). These specialty certifications are overseen by their voluntary membership in the Forensic Specialties Accreditation Board (FSAB) created by the American Academy of Forensic Sciences (AAFS).
The concerns raised in the NAS report have been recognized by laboratory personnel for many years. Unfortunately the solutions are, in many instances, tied to requisite funding. Some reduction in the frequency and ongoing nature of some of these issues might be possible by taking some different approaches to quality control/quality assurance methods currently in place.
One possible consideration is to implement true blind proficiency testing, utilizing samples previously examined by a particular examiner where feasible. This could supplant current testing protocol. The cost and effort associated with this type of testing may well prove worth it in the long run, especially when weighed against the potential remediation costs that result from long term faulty analyses.
A corollary to blind testing is truly independent verification. Like blind testing in general, this is not easy to accomplish and has a negative impact on productivity. However, reducing confirmational bias in the verification of fingerprint, firearms, and tool mark identifications and other areas is an important quality assurance aspect that needs to be emphasized.
A third possible consideration is the random review of trial testimony transcripts for monitoring court testimony in conjunction with traditionally used methods. Added costs might again be justifiable in view of potential long term gain.
Mandatory accreditation and certification are important first steps for improving crime lab operations and their employees, as well as for single service/limited service entities. Accreditation should either be an all or nothing proposition with regard to service offerings or steps should be taken to ensure that testimony in non-accredited areas is identified as such in court. The separation of crime labs from law enforcement agency chains of command is an important step toward minimizing incidents of contextual/confirmational bias.
The NAS crime lab report of 02/18/09 once again raises awareness of the same issues that have plagued crime laboratories’ quest for excellence since their very inception: the need for more personnel, better facilities/equipment, and funding for ongoing training. Given the current economic situation and the history of shortcomings in those areas during much better economic times, it is doubtful any real relief is in sight for the immediate future, but one can still hope.
- http://topics.nytimes.com/top/reference/timestopics/people/g/joyce_ gilchrist/index.html
- www.in.gov/ipdc/general/Baltimore Fingerprint Ruling.pdf
- http://www.bioforensics.com/news/HPD_proficiency_test_10- 07.html
- Dror, David C. and Peron, Alisa E., “Contextual Information Renders Experts Vulnerable to Making Erroneous Identification,” Forensic Science International, Vol. 156, pp. 74-78.
- Cole, SimonA., “More Than Zero:Accounting for Error in Latent Fingerprint Identification,” The Journal of Criminal Law &Criminology, Vol. 95, No. 3, 2005.
- DeLillo, Sandy, “AMatter of Due Process:An Examination of How State Mandated Accreditation Has Impacted Texas Crime Laboratories,”Master’s Degree Thesis,University of North Texas, 2006.
Ed Hueske is the Criminalistics Program Coordinator, Department of Criminal Justice at the University of North Texas in Denton, Texas. He is a former Crime Lab manager, former ASCLD-LAB inspector, and former member of the ASCLD-LAB board of directors. He can be contacted at Edward.Hueske@unt.edu.