Facial recognition can be a valuable identification tool when fingerprint identification is unavailable or impracticable.
Just like automated fingerprint identification, facial recognition can provide law enforcement agencies with a valuable tool for multiple public safety applications. Whie fingerprints assure higher rates of accuracy than face recognition can, facial recognition provides benefits when fingerprint data does not exist, is not easily shared between agencies, or when multiple independent verification methods are desired. Additional applications include identity verification in the field and intelligence gathering, as well as crime prevention and investigation.
A Brief Background
Since the advent of photography, both government agencies and private organizations have kept photo collections of people. Indeed, photos have made their way onto personal identification documents, from passports to informal membership cards issued by schools and athletic clubs.
Before the use of computers to recognize faces was even considered a possibility, facial recognition was already the subject of a great deal of research. Examples include:
- Development of identification line-up techniques in which a witness is confronted with a group of physically similar people, one of whom is a suspect. The witness must then decide whether one of the persons in the group was present at the scene of the crime or not.
- Work done by Bertillon1 on face classification. In order to recognize individuals who were repeatedly arrested, Bertillon developed means by which portraits could be sorted by common morphological characteristics—the specific shapes of the different parts of the face—and thus an individual’s prior photo could be found without having to resort to browsing through large collections of portraits. This classification is known as the “portrait parlé” or spoken portrait.
Facial recognition with good quality images
The first attempts to automate facial recognition started in the 1960s in semi-automated mode. The approach essentially consisted of checking the correspondence of measurements between different facial feature locations (the corners of the eyes, the hair line, etc.) These first attempts were not very successful as faces are by nature very animated and measurements between characteristic points are also affected by viewing orientation.
Toward the end of the 1980s, the development of the eigenfaces2 technique prompted more intense research efforts. This technique is used to find a face in a photo and to compare images of faces. Researchers quickly found that facial recognition was complex but could be simplified by only taking into consideration images that are similar in terms of orientation, lighting, expression, and image quality. Research focused on this problem and, as a means by which to mitigate these challenges, the International Civil Aviation Organization (ICAO) defined criteria to obtain controlled portraits and created meaningful test sets.
In early 2007, the National Institute of Standards and Technology (NIST) published the results of its Face Recognition Vendor Test (FRVT) 2006. The conclusions were clear. Research had reached a point where the operational use of facial recognition on high-resolution frontal images taken in a controlled environment was now feasible. Naturally, these results did not put an end to work on the recognition of facial images captured in controlled conditions. And while more improvements are expected, facial recognition has become a biometric technique in its own right.
General facial recognition
Since 2007, research has been focusing on significantly more difficult problems where faces are not viewed frontally, resolution is low, or the image quality is poor.With the Multi Biometric Grand Challenge (MBGC) series of benchmarks, NIST is again seeking to assess performance and has provided researchers with data that is representative of real world problems (images and videos of faces under non-controlled conditions). This heralds the start of a new era in facial recognition and we can expect to see significant progress in the coming years.
Criminal Justice Applications
Just like automated fingerprint identification, facial recognition provides law enforcement and government agencies a way to manage the records of people of interest. Among other tasks, it provides an alternate, or additional,method to make sure that databases do not contain multiple records for a single individual. While this task is routinely performed by police agencies using fingerprints, facial recognition provides benefits when fingerprint data either does not exist or is not easily shared between agencies:
- It allows people to be identified, even when it is not possible to take fingerprints for physical or legal reasons.
- By combining the two biometric techniques, the workload involved in the verification process is reduced and the efficiency and accuracy of the process increases.
Indeed, Pierce County Sheriff’s Office in Washington state has demonstrated a previously unimagined accuracy rate of 94% in the first candidate spot, when using mug photos for automated facial identification of individuals during the booking process.
Identity checks in the field
With a simple camera and a means of transmission, it is also possible to check the identity of a person in the field using face recognition. Police officers equipped with PDAs can quickly capture facial images and submit search requests to remote facial recognition systems, quickly determining whether an individual is known to law enforcement.
ID checks can be carried out on just the face or on both the fingerprints and the face, if the officer has the required equipment to take fingerprints. The combination of the two biometric techniques increases the accuracy of searches and allows reliable, automated decisions to be sent to the field, without requiring expert analysis.
Criminal investigations and information
Images are often made available for investigations. These can come from surveillance videos, a witness’ camera, Internet sites, or copies of identification cards.
The first step is to extract the facial images from the available evidence. In some investigations, hundreds of hours of video footage are analyzed. The “manual” approach of searching for video frames in which faces are visible is a long and painstaking job. This is why automated assistance is necessary. Current automated face extraction techniques work well with almost full frontal views of faces when the video quality is sufficient. The extraction of side and three-quarter views of faces is an active area for current research.
Even if the quality of the extracted facial images is highly variable, it is still possible to compare them with photographs of persons known to the police. Experience shows that it is already possible for these searches to solve and correlate crimes. Investigators in the U.S., Europe, and Australia have scored numerous hits with high-quality images, such as authentic or false ID documents or images posted on the Internet. It is interesting to note that criminal cases have even been solved using low quality images.
Operational examples where surveillance videos are used exist but are less common. Indeed, by way of example, images of individuals caught on video by ATM cameras have been used to solve crimes. However, they cannot be used to successfully close investigations if the video only shows the top of the suspect’s head or if the images are blurred. Hence, combined advances in the deployment of video surveillance systems along with improvements in facial recognition technology should enable more crimes to be solved using video data in the next few years.
Facial recognition can also be used for preventative purposes. For example, if a database of pedophiles is available, then ID photos can be used to check whether people who work with children are in the file, and hence should not be in that environment.
In some cases, facial recognition can also be used to interactively locate wanted persons in video footage. However, this application is subject to controversy for several reasons, including concerns that it may be an infringement of civil liberties. In any case, it is not currently suited to instances in which a very small number of persons need to be identified in a large crowd. Even if this technique were to reach an accuracy level of 90% of persons actually found with just 0.1% false identification rate, looking for one person amongst a crowd of 100,000 passers-by would operationally generate 100 false identifications; this would have a negative impact on the vigilance of control operators.
On the other hand, interactive facial recognition is already in use under controlled situations. For example, when travelers approach border police for a travel document check, facial biometrics can easily check travel documents and watch lists, providing passport control officers an opportunity to spend more time on higher risk travelers.
- Alphonse Bertillon, 1853–1914, was the criminologist who developed judicial anthropometry in France.
- Eigenfaces: a facial recognition technique that consists of learning the distinctive characteristics of faces from a broad sample of portraits using each complete image rather than local characteristics (e.g. the eyes, nose, or mouth).
Eric Hess is a Senior Biometric Product Manager at MorphoTrak. He specializes in facial recognition, symbol, and tattoo matching technologies. Eric has extensive experience in product management and has been with MorphoTrak since 2007. He can be reached at firstname.lastname@example.org.