By Marcus Borengasser, Ph.D.
Introduction to Forensic Image Processing
Forensic image processing (FIP) involves the computer restoration and enhancement of surveillance imagery. The goal of FIP is to maximize information extraction from surveillance imagery, especially imagery that is noisy, incomplete, or over/under exposed. Although this definition is with respect to surveillance imagery, FIP techniques can be applied to other types of images, such as retinal images, shoe impression images, UAV (unmanned aerial vehicle) infrared images, and more.
Often, for a variety of reasons, the quality of surveillance imagery is very low. The low imagery quality can be caused by poor lighting, poor media quality (analog systems), excessive motion of the subject, a camera in need of calibration, and noise introduced by the imaging/recording system. With digital filtering, image restoration, de-noising, and enhancement techniques, information can often be extracted from low quality imagery.
Forensic imaging processing is a method of improving a digital image (surveillance, closed circuit TV, infrared, etc.) using a variety of computer techniques. These techniques often involve digital “filters” that can suppress noise in the digital image, aid in the extraction of detail from shadow, and provide image sharpening.
Limitations of FIP
Forensic image processing cannot restore image quality beyond the original information content, i.e., it cannot add information to the image. Like any other technology, there are limitations to what can be done with forensic image processing. The most fundamental limitation is related to the information content. No amount of image processing and enhancement can add information that is not present in the image. This is different from a situation where an image is degraded by noise, for example. The desired information may be present in a noisy image and the proper sequence of forensic image processing techniques may allow the extraction of that information.
Computer Vision
Computer vision is a technology that uses computers to develop a high-level of understanding of a digital image or video. This process involves the automatic extraction, analysis, and understanding of information from an image or video. The image data can be produced by video surveillance cameras, image streams from multiple cameras, or images derived from medical scanning systems.
Computer vision applications include scene reconstruction, video tracking, object detection, object recognition, 3D pose estimation, motion estimation, 3D scene modeling, image restoration, and more. Some specialized applications of computer vision include event detection, activity recognition, learning, indexing, motion estimation, 3D scene modeling, and image restoration.
OpenCV
OpenCV (Open Source Computer Vision), initially developed by Intel, is a free computer vision library for real-time image processing. The OpenCV software is widely used for most computer vision applications, such as image processing, video capture, and object detection.
OpenCV is a large library of more than 2500 optimized algorithms that can be used for many different computer vision applications, such as:
- Face detection and recognition.
- Object identification.
- Object tracking.
- Image registration and stitching.
- Augmented reality.
Image Contrast
Contrast refers to the color or grayscale difference between various image features in both analog and digital images. Images with higher contrast usually have more color or grayscale variation than images with lower contrast.
Because of sub-optimal lighting conditions, optical systems, cameras, and image capture systems may produce low contrast in a captured image. These conditions, and the resultant degraded images, can adversely affect photography, forensics, surveillance, and image analysis. Image contrast can be improved by modifying the histogram of pixel values, a technique to increase the dynamic range of pixel intensity and enhance image detail.
Figure 1 shows an example of image pixel redistribution of a digital image and the subsequent improvement in contrast using an algorithm from OpenCV.
Figure 1. Original low contrast image (left) and image with enhanced contrast by histogram modification (right).
Image Filtering
Image filtering is a process of passing or attenuating specific spatial frequency components in an image. A digital filter can be used to either suppress or eliminate spurious data or enhance features that are not visibly apparent in the image. Image filters can be classified as low-pass or high-pass filters. A low-pass filter passes low-frequency spatial features and attenuates high-frequency features, resulting in a blurring or smoothing effect on an image. A high-pass filter does the opposite, it passes high-frequency spatial components and attenuates low-frequency features. The result is an image with enhanced detail, referred to as a sharpened image.
Low-pass filtering is often used to remove high-spatial frequency noise from a digital image. This improves image quality by suppressing high-frequency noise without loss of important image detail.
Figure 2 shows an example of noise suppression using an OpenCV algorithm.
Figure 2. Image with speckle noise (left), and the result of low pass filtering to reduce noise (right).
A high-pass filter is used for image sharpening. Image detail is sharpened when contrast is enhanced between adjacent pixels that have minimal variation in brightness or darkness. A high-pass filter is designed to retain the high-frequency information in an image while simultaneously suppressing low-frequency information. As a result, high-pass filters emphasize boundary pixels between contrasting pixels and perform as an edge detector or edge enhancement filter for image features.
Figure 3 shows an example of edge sharpening with an OpenCV high-pass filter.
Figure 3. Frame 175 of the President John F. Kennedy motorcade, from Abraham Zapruder’s Bell & Howell camera, showing before (left) and after (right) application of an edge sharpening filter.
Conclusion
Forensic image processing can be a useful tool for image enhancement and analysis, especially when used in conjunction with OpenCV.
About the author: Marcus Borengasser, Ph.D., works as an Image Scientist. He has expertise with remote sensing and image analysis, and proficiency with machine learning, simulation, and data mining. Marcus lives in Florida and dreams about flying. You can contact him at [email protected].