Why Missing Evidence is More Dangerous than a False Positive

False positives due to AI are quick to get attention in investigations. They’re visible, disprovable and it’s easy for people to point to AI as the point of failure.

But even when an AI system gets it wrong, there’s a trained forensic specialist with strong foundational skills who can identify the error and still crack the case.

Missing evidence, on the other hand, is more dangerous as there’s typically nothing to act on. It doesn’t announce itself as the problem, which leaves an investigator with nothing to interrogate since there’s no anomaly to figure out or error to disprove.

No evidence can still be evidence

AI doesn’t have the investigative instinct or the decades of training of a specialist which becomes clear when there’s a lack of evidence.

Unlike a false positive, AI systems don’t challenge that missing evidence because there’s nothing to reference in the first place. With everything AI, there’s got to be a source behind it, whether it’s DNA, digital evidence or a verbal witness statement. Every AI-assisted judgment needs to have a clear base of evidence, but if there is no base to begin with, then the AI system can’t move ahead.

AI systems are currently trained to tell an investigator when a device has come back clean, but it can’t tell them if that’s significant within the case, or whether a clean device was engineered by a criminal.

In a case that I worked on previously, one of the scans came up empty handed. Somebody had gone through the phone and had selectively removed exactly what a forensic examiner would expect to see. And that was deliberate because doing a full wipe would have been instantly noticed.

If AI had scanned, it would have stalled the case because the evidence didn’t exist. The same pattern shows up across many cases with 56% of devices arriving locked according to Cellebrite’s Industry Trends Survey, meaning that restricted access can be misread as a dead end early in the investigation.

It requires persistence and critical thinking to understand what to look for in evidence, and that includes what a lack of evidence could indicate.

Foundational skills remain vital

The greatest tool for a forensic analyst is stillwill always be foundational skills, the ability to investigate and validate AI outcomes, and to recognize false positives as well as what missing evidence could actually point to.

This is the hill I’ll die on. Without foundational skills, applying AI to a case can put an entire investigation at risk.

AI adds power and enhances existing knowledge, but if an inexperienced investigator relies on AI at face value, without any kind of deep search, sources or context, the whole case can unravel. What does matter is the expertise of the person behind the prompt and the keyboard, and what they know.

That’s why the forensic teams and investigators leading cases must have the core skills in place, including the ability to notice when key evidence is missing, and to have the instinct to ask if that’s by design.

The AI trust problem

No matter how skeptical an investigator is, AI is a tool that we need to get comfortable with. However, if it’s being used to make important decisions or determinations about a case, then there must be a critical lens to independently verify everything it produces to understand where information came from and how the AI made its conclusion. 

One of the main issues with AI can be confirmation bias, since AI often defaults to ‘yes’ as an outcome. That risks the AI not looking at an alternative angle or for different evidence once it’s come to a conclusion and confirmed what an investigator is thinking.

AI still has shortcomings for digital forensics, but it can be so efficient at supporting tasks that make an investigator’s life easier. AI tools can help with summarizing evidence and triaging large volumes of device data to fast-track what needs human review.

Trust in AI systems should be built up the same way that trust in a junior analyst is, regularly checking outputs and having a constant feedback loop to train what good looks like.

Foundational skills are still vital and that cannot become a lost art among the forensic community. Missing evidence is a clear example of why instinct and experience remain critical in the world of AI, and the investigators that prioritize these skills will be able to get the most out of AI as a supporting tool.

Subscribe to eNewsletters

Stay up to date on the forensic industry with the latest news, cold cases, technologies, webinars and more delivered straight to your inbox.