Fans of Law & Order or CSI Miami have been duped! Rarely does scientific evidence prove to an absolute certainty that the defendant committed the crime, and sometimes, such evidence is flat out scientifically unsound and unreliable. So sayeth the President’s Council of Advisors on Science and Technology (“PCAST”). The group’s 174 page report, released last week, analyzes impropriety and irregularities in the use of forensic evidence, and lays out recommendations to improve the use of science in the legal system.

Persistent Forensics Problems

As the report lays out in its executive summary:

Developments over the past two decades—including the exoneration of defendants who had been wrongfully convicted based in part on forensic-science evidence, a variety of studies of the scientific underpinnings of the forensic disciplines, reviews of expert testimony based on forensic findings, and scandals in state crime laboratories— have called increasing attention to the question of the validity and reliability of some important forms of forensic evidence and of testimony based upon them.

Too many stories, Like Keith Harward’s, proved the need to conduct this study. Harward languished in prison for 33 years, having been convicted of rape and murder, but was exonerated in 2016 based on DNA evidence that proved his innocence. Harward’s erroneous conviction was based in part on expert testimony that his teeth “matched bite marks on the victim,” that there was “a very, very, very high degree of probability [his] teeth left that bite mark,” and that “there is just not anyone else that would have this unique dentition.” Similarly, a July 2014 Report issued by the Inspector General of the Justice Department regarding irregularities at the Federal Bureau of Investigation Laboratory revealed, among other things, that in one capital case, false testimony about a hair sample led to conviction and execution of the defendant. And evidence tampering by a single former chemist at the Hinton State Laboratory Institute in Massachusetts may have tainted as many as 40,000 cases. Unfortunately, many similar stories exist of improprieties with respect to forensic evidence presented in criminal cases.

As one of us has stated and spoken about previously, while scientific evidence in a courtroom “can be very powerful,” it is subject to various forms of abuse. “[W]hen forensics experts tamper with or fabricate evidence, utilize shoddy testing procedures, testify about areas outside their expertise, overstate the value of their testimony, or present unsupportable scientific conclusions in order to obtain a conviction, this will not only likely lead to an injustice in the particular case; it will also hasten the arrival of the day when judges and jurors no longer trust the government’s experts.”

White House Science Advisers Debunk “Infallibility” of Forensic Evidence

The PCAST report goes farther. It says that there is good reason not to trust certain forensic methodologies—specifically, “methods for comparing DNA samples, bitemarks, latent fingerprints, firearm marks, footwear, and hair.” The White House science advisers, in consultation with legal experts including federal judges and Justice Department officials, concluded that each of those kinds of evidence suffers to varying degrees from an overarching lack of clarity surrounding what makes for scientifically valid and reliable evidence.

The good: DNA testing emerges relatively unscathed, as the report concludes that it is scientifically valid and reliable, and can be used to convict the guilty and to exonerate innocent individuals who were convicted on the basis of shoddier scientific evidence. When it comes to scientifically validating complex DNA samples (“mixtures of biological samples from multiple unknown individuals in unknown proportions”), the report concludes that some methods work in certain cases, but that scientists have much more work to do if they want to “establish foundational validity” (meaning that the scientific method involved is shown, based on empirical, peer-reviewed studies, to be repeatable, reproducible, accurate, and capable of being reliably applied when utilized in particular situations).

The bad: The Council concluded that while latent fingerprint analysis is “a foundationally valid subjective methodology,” not every partial print can serve as reliable evidence. Just ask Brandon Mayfield, an American lawyer who was initially identified by the FBI based on a partial fingerprint and held as a suspect in the March 2004 Madrid train bombings which killed 191 people and injured roughly 2,000 others. After Spanish authorities provided additional information casting doubt on those findings, the FBI conducted additional tests, cleared Mayfield, formally apologized to him, and ended up paying him a large sum of money. The Council also stated that “many jurors” would be surprised to learn, in light of “longstanding claims about the infallibility of fingerprint analysis,” that the number of false matches “could be as high as 1 error in 306 cases” according to an FBI study, or “1 error in 18 cases based on a study by another crime laboratory.” Again, more research was recommended.

The ugly: The Council’s report concludes that some forensic methods, especially bitemark analysis, are outright bunk. The report states that “available scientific evidence strongly suggests that examiners not only cannot identify the source of [a] bitemark with reasonable accuracy, they cannot even consistently agree on whether an injury is a human bitemark. The report goes on to state that, “PCAST finds that bitemark analysis is far from meeting the scientific standards for foundational validity.” Bite-mark analysis is, in fact, so shoddy that the PCAST advises “against devoting significant resources” to validating bite-mark analysis.

Obama’s science advisers state that jurors tend to believe that so-called scientific evidence is infallible (or darned close), but that such unquestioning belief is unwarranted. The group picks up where a 2009 report by the National Research Council, Strengthening Forensic Science in the United States: A Path Forward, left off, seeking to “overhaul” the national “forensic science community.” That report concluded that “much forensic evidence—including, for example, bitemarks and firearm and toolmark identifications—is introduced in criminal trials without any meaningful scientific validation, determination of error rates, or reliability testing to explain the limits of the discipline.” It was this report that led President Obama to ask his executive science squad, what could his Administration do “on the scientific side” to “help ensure the validity of forensic evidence used in the Nation’s legal system”?

PCAST’s Recommendations

The PCAST report lays out several recommendations to both government officials and private research entities to improve the current system. Many of these recommendations address the need for further research by academics (with suggested appropriations included) and federal agencies to fill existing gaps in the validity of forensic science. And where the rubber will eventually meet the road, the report also provides recommendations to judges and prosecutors.

Judges must decide what scientific evidence is valid and invalid in the course of deciding what gets admitted into evidence and what doesn’t. In making this determination, judges are guided by the Federal Rules of Evidence and the standards laid down by the U.S. Supreme Court in its 1993 decision in Daubert v. Merrell Dow Pharmaceuticals. Under the rules, judges may also, if they so choose, procure the assistance of their own experts to help advise them. The PCAST report supplies “the appropriate scientific criteria for assessing scientific validity,” and urges judges to reject scientifically invalid evidence. The report also recommends that the Judicial Conference of the United States create a best practice manual on feature-comparison methods (the process of comparing an evidentiary sample, usually seized from a crime scene, to a “source” sample from a suspect to see whether a scientifically-valid “match” exists between the two), and that the Advisory Committee notes that accompany the federal rules of evidence be revised to reflect the group’s findings.

Two of the federal judges who served as Senior Advisors to the PCAST Working Group, 9th Circuit Judge Alex Kozinski and Senior Judge Harry T. Edwards of the U.S. Court of Appeals for the D.C. Circuit, have championed the report’s findings. According to Judge Edwards and Prof. Jennifer Mnookin, another Senior Advisor on the project, the PCAST report “puts forward a plausible, workable test for validity: Forensic disciplines should pursue empirical studies designed to test error rates and accuracy in conditions akin to those found in the real world. . . . The integrity of our criminal-justice system deserves no less.” It remains to be seen what affect, if any, these findings will have on the traditional gatekeeping function that judges play in controlling the admission of scientific evidence in their courtroom.

The report also gives some strong advice to federal prosecutors. The report states that claims by experts that certain evidence has a “zero,” “essentially zero,” or “microscopic” error rate are “not scientifically defensible,” and urges prosecutors not to allow (and courts not to permit) experts whom they call to the stand to make such statements to the jury. The report also advises that “[i]n testimony, examiners should always state clearly that errors can and do occur, due both to similarities between features and to human mistakes in the laboratory.” PCAST recommends that DOJ generally facilitate improvements in the state of forensic science by, among other things, working with the forensics units of the applicable federal agencies and other interested stakeholders to help identify what is and is not reliable scientific evidence, and conducting (and encouraging others to conduct) research where it is needed.

Not everyone in the law enforcement community, however, is pleased with the report’s findings. An FBI statement claims that the PCAST report “makes broad, unsupported assertions regarding science and forensic science practice” and reaches “troubling generalized conclusions about all forensic science disciplines.” A DOJ spokeswoman, somewhat defensively, stated that while it welcomed “continued review,” the Department had already taken “unprecedented steps” to remedy past problems and that it remained “confident that, when used properly, forensic science evidence helps juries identify the guilty and clear the innocent.”

As Judge Edwards and Prof. Mnookin have correctly noted, “[r]equiring that the forensic methods we use in court have a reasonable modicum of scientific validity is neither pro-defense nor pro-prosecution; it is, rather, both pro-science and pro-justice.” One should hope and expect that government attorneys and forensic scientists share the same perspective.

The Obama administration and the PCAST members deserve credit for their efforts and insights to address longstanding and persistent problems in forensic evidence, and for providing a way forward to help jurors adopt a more accurate perspective on what weight to give to forensic evidence that is admitted at trial, to help preserve the liberty of innocent parties who might otherwise be convicted of crime on the basis of faulty scientific evidence, and to ensure that those who actually commit crimes get their just deserts.

* * * * *

John G. Malcolm is the Director of the Edwin Meese III Center for Legal and Judicial Studies, and the Ed Gilbertson and Sherry Lindberg Gilbertson Senior Legal Fellow, at The Heritage Foundation, and is also the Chairman of the Federalist Society’s Criminal Law Practice Group.  John-Michael Seibler is a Legal Fellow in the Edwin Meese III Center for Legal and Judicial Studies at The Heritage Foundation.