Observation of a Validation
The Full-Scale Implementation of Automation in a Small Casework Laboratory
In late 2008, the Allegheny County Office of the Medical Examiner Forensic Laboratory (ACOME FL) began the full-scale conversion to automation with increased use of information technology within the workflow of the Forensic Biology Section. As part of a federally-funded grant, through the National Institute of Justice, to enhance efficiency within forensic laboratories, the Forensic Biology process was evaluated for bottlenecks which could be alleviated through the application of automation and computer systems. Several points in the process which introduced inefficiency were identified. Those areas included microscopic examination of sexual assault samples; sample extraction for DNA processing; and set-up for quantitation, amplification, and capillary electrophoresis, as well as DNA data analysis. All of the identified bottlenecks possessed the characteristics suitable for automation: work intensive and highly repetitive. The addition of new chemistries was also planned; they were intended to optimize the amount of information obtained from a sample, while also identifying the most advantageous inlet into the DNA workflow for the samples being processed. The chemistries included a Y-STR kit and a male/autosomal quantitation kit. Information technology and data manipulation were addressed through the implementation of a DNA-specific Laboratory Information Management System (DLIMS), the creation of a dedicated computer network for the DNA workflow, and a genetic calculator to aid in the time-consumptive data analysis. Through the course of executing the restructuring of the DNA process in the small-casework laboratory of Allegheny County, a number of observations were made on how to improve laboratory efficiency in an efficient manner.
Careful attention must be paid when creating the process map of the new automated paradigm and how different aspects of the new instrumentation and chemistries fit into the intended workflow. The impact of areas beyond casework analysis must all be considered when selecting the appropriate platforms for your laboratory. The selection of the robotics platform must reflect on the technical capabilities of the staff. Some automated systems require a great deal of technical knowledge to build methods as well as maintain the instrument. The technical capabilities required to successfully operate the more involved platforms can be quite extensive, and a lack of technical aptitude can lead to a difficult validation process and result in a learning curve which slows the implementation of robotics, negatively impacting efficiency. The robotic needs of the laboratory must also be factored into the selection of automation. The robotic platform chosen should also be useful for the analysis performed within the laboratory; the type of samples which the laboratory processes must be taken into consideration, so that a suitable automated platform can be selected to address all sample types within the laboratory. In this laboratory, the increased processing of low level DNA samples requires a robotic system not subject to contamination, while still producing high quality results. A robotic system that was capable of adequately processing low level samples, which constitutes approximately 30% of our DNA caseload, as well as stain level samples, would have been more beneficial to the efficiency of the forensic biology workflow. The choice of robotics should also remain consistent in the platforms utilized in the laboratory, when possible. Selecting multiple robotic platforms requires time-intensive validation for each one, as well as more training for each analyst to gain competency on the robots. This effect does not end with the implementation of robotics but continues to affect the training of each new DNA analyst. Moreover, maintaining multiple systems creates layers of time consuming quality control steps that must be performed at regular intervals (i.e., weekly, monthly, annually, biannually). Each robotic platform requires quality actions that remove analysts from participation in casework, decreasing the efficiency of the process, while eating away at costly reagents and consumables.
In the implementation of automation, the most profound impact was observed at two points in the process: microscopic examination of slides for spermatozoa and extraction. If a laboratory must selectively implement automation, the areas where the greatest benefit would be achieved are at these points in the process. The microscopic examination of slides for the presence of spermatozoa performed by an automated sperm detecting microscope system has multiple benefits. The system utilizes an algorithm to identify spermatozoa through staining density and the characteristic morphological structures of human sperm. The microscopic examination of samples in a typical sex assault kit submitted to the Forensic Biology Section of ACOME FL consumes an average of two days of analyst time. The automated sperm detecting microscope can perform the same examination overnight, freeing the analyst to perform other tasks unsuited for automation during the work day and completing the kit faster. Additionally, the capabilities of the system result in higher levels of spermatozoa being observed due to superior optics, as noted in Table 1, as well as documentation on the cells identified for case records. This increases the sensitivity of the microscopic examination while also providing useful information for the courts and submitting agencies. The reproducibility of the system is at approximately 60%, as can be observed by comparing the values in columns 1 and 2 of Table 1, but all spermatozoa identified by the system are confirmed by a trained analyst, resulting in a dependable system. The extraction stage of the Forensic Biology workflow experienced a significant increase in efficiency as well, as observed by the greater number of samples and less time demonstrated by Figures 1 and 2. Previously, manual organic extractions, the most common extraction performed in this laboratory, consumed 20–24 hours of analyst time to successfully extract 12–15 samples. The implementation of automation reduces the procedure to an average of 4 hours to successfully extract approximately 50 samples, while requiring significantly less hands-on manipulation by the scientist. The combined effect of decreased time with increased batch size acted to significantly improve the extraction throughput in the DNA portion of the forensic biology workflow.
The execution of the validation study for complete conversion to automation is best performed in a step-wise fashion, rather than as a whole system. In an attempt to streamline the validation process, ACOME FL validated all of the new systems together. Instead of achieving the intended result of efficient realization of the automated system, it inexorably slowed the ability of the laboratory to bring any new improvements online. The combined validation made troubleshooting issues observed in the course of testing difficult and added to the time required to determine optimal settings and procedures for each of the new processes.
When performing a validation as extensive as full-scale conversion to automation, it is important to consider the effects of personnel shifts created by loss and gain of scientists, as well as the impact of time dedicated to validation on casework and backlogs, as illustrated by the data in Table 2. The best approach to an extensive validation study is the participation of multiple members of the laboratory. Not only does the involvement of multiple people aid in time management, but it also prevents a concentration of knowledge on the technique under validation. The concentration of knowledge in a limited number of scientists within the process negatively impacts the efficiency of the laboratory when members of the laboratory are lost or gained. Dedicating one individual to the validation process removes them from the day to day operations of the laboratory, reducing the number of analysts performing casework. At the outset of the validation, it can be observed in Periods 1 and 2 of Table 2 that even while the number of samples per analyst increased, the turnaround time for samples also significantly increased, due to the limited number of individuals performing casework at that time, while two scientists were dedicated to the validation process. Additionally, the departure of a member of a dedicated validation team will result in the loss of knowledgeable familiarity, which was gained through hands-on experience with the procedure and instruments. In Table 2 the loss of individuals from this laboratory can be observed in the sharp increase in turnaround time and backlog samples in Period 4. By having several individuals execute the validation study there is a sharing of information and experience. This insulates against the loss of skill and knowledge, as well as decreasing the amount of training required to gain competency with the procedure due to expertise. The training analysts who perform the procedure can also be negatively impacted by using a small validation team.When training commences, few members of the laboratory will have had exposure to the process, creating a steeper learning curve and lengthening the training process.
Information technology can greatly facilitate the process in a forensic biology laboratory.2 A dedicated network, comprised solely of DNA instrument computers and data analysis computers, was created for all forensic biology in this laboratory. The dedicated network simplified the data transfer steps required throughout the process. The movement of data between instruments can add multiple steps at every level of processing. The presence of data in one central location, accessible at all points of the process, removes the need to transfer information from system to system. In addition to the dedicated network, a dedicated DNA Laboratory Information Management System was purchased with the intent of assisting in the transfer of data, while aiding in the tracking of evidence and quality assurance as well as control performance. When selecting a DLIMS, it is important to consider any existing LIMS utilized laboratory-wide, as well as the demands of the process the DLIMS will be servicing. The implementation of a secondary system within ACOME FL, different from the existing system, led to redundant steps within the information tracking procedure. Information which is originally entered into the laboratory wide system must be re-entered into the DLIMS when it enters the DNA process, creating multiple steps to track the same information and reducing the efficiency it is intended to enhance. Additionally, it is important to implement an information management system which meets the needs of the process. A DLIMS which does not meet the requirements for the information tracking and data manipulation of the process requires time consuming and costly customization and the implementation of additional information technology, as experienced in the number of additional hours and customizations this laboratory invested into the DLIMS, as well as the creation and utilization of an Excel workbook for DNA processing. The implementation of additional levels of information technology can help the management system to meet the needs of the laboratory but also added further layers of superfluous data entry if not approached with existing systems and future changes in mind. Added layers of data entry and incompatible features will negate the efficiency of implementing a DNA Laboratory Information Management System but still allowing for the laboratory to perform.
Automation and information technology is a recognized method to increase efficiency in forensic laboratories.1-3 It has been utilized at various steps in the forensic biology process in laboratories world-wide. But the full-scale implementation of robotics is not as common as piecemeal utilization. As time progresses, the number of laboratories that will completely automate their workflow will increase. The experiences of this laboratory can be used by others to make a more graceful leap into full-scale automation, rather than an ungainly tumble resulting from the many downfalls that can be experienced in the planning and validation processes. The most important points to be gained from the experiences of this laboratory are careful consideration of the existing system and intended improvements when planning the validation and the distribution of knowledge when performing the validation. Both measures will ensure the successful execution and completion of a full-scale validation of automation.
- Crouse C, Yeung S, Greenspoon S, McGuckian, Sikorsky J, Ban J, Mathies R. Validation of Robotic Assays and Evaluation of Microcapillary Array Device. Croat Med J 2005;46(4):563- 577.
- Frégeau CJ, Lett M, Elliott J, Bowen KL, White T, Fourney RM. Adoption of automated DNA processing for high volume DNA casework: A combined approach using magnetic beads and real-time PCR. International Congress Series 2006;1288: 688-690.
- Greenspoon SA, Sykes KL, Ban JD, Pollard A, Baisden M, Farr M, Graham N, Collins BL, Green MM, Christenson CC. Automated PCR setup for forensic casework samples using the Normalization Wizard and PCR Setup robotic methods. Forensic Science International 2006;164(2): 240-248
Sara E. Hochendoner is a scientist at the Allegheny County Office of the Medical Examiner Forensic Laboratory in the Forensic Biology Section. Allegheny County Office of the Medical Examiner; 1520 Penn Avenue Pittsburgh, PA15222; (412)350-3742; Sara.Hochendoner@AlleghenyCounty.us; http://www.alleghenycounty.us/me/index.aspx