Forensics in crisis

The assumption that forensic science provides unequivocal answers for the criminal justice system was seriously undermined several years ago. Two US chemists who worked at Massachusetts state labs in the US separately admitted to, and were convicted of, faking drug evidence. Although they’ve both served their prison sentences, the story is not over.

The severe misconduct of the now infamous scientists Annie Dookhan and Sonja Farak caused massive reverberations, leading to tens of thousands of overturned drug convictions. Since the scandals erupted, several other smaller scale forensic fraud cases have surfaced in the US, as well as in the UK and Canada.

Beyond upending the legal system, these incidents have shaken confidence in forensic science. To navigate this complicated situation, well-established forensic techniques must be clearly separated from the less proven ones and lawyers require more substantial scientific training.

It all began with the September 2012 arrest of Dookhan, who had worked in a Massachusetts public health lab for more than nine years. She had apparently been falsifying evidence used in criminal drug cases. Although Massachusetts’ inspector general concluded that she was the ‘sole bad actor’ at the lab, concern soon grew that Dookhan’s actions revealed system-wide failures. This led to calls for major forensic science reform in the US.

Those calls were amplified when the Farak case became public only months later. She was arrested in January 2013 and convicted a year later, after pleading guilty to tampering with evidence, stealing illegal drugs from her lab that had been seized as evidence and possessing cocaine.

Farak worked for the Massachusetts public health department from 2003 to 2012, and then for the state police until January 2013. She and Dookhan worked in the same drug analysis lab for six months, but their motivations to commit fraud were very different: Dookhan wanted to appear as productive as possible, while Farak was driven by a drug addiction and her desire to cover it up. It has taken several years to flesh out the real damage resulting from their separate misconduct, and the full harm is still being assessed.

Vacating convictions

In April 2017, the Massachusetts supreme court directed the dismissal more than 21,500 criminal drug cases potentially tainted by Dookhan’s actions The move was described as the largest dismissal of wrongful convictions in US history.

A year later, the same court ordered that about 8000 drug convictions affected by Farak’s misconduct be dismissed. The American Civil Liberties Union of Massachusetts (ACLUM) estimates that the decision could result in more than 11,000 criminal charges being dismissed.

In fact, ACLUM and other groups are pushing to get more criminal drug cases thrown out in the wake of Farak’s actions, and they petitioned the Massachusetts Supreme Judicial Court on the matter in May. The court has not yet issued its decision as this article goes to press. However, the Massachusetts attorney general expressed support for dismissing the charges against additional defendants since Farak tampered with drug samples assigned to other chemists for analysis.

In the aftermath of Dookhan and Farak, a string of other, less prominent instances of misconduct at other US forensic labs emerged, helping to paint a portrait of a field in disarray. For example, Jonathan Salvador, a forensic scientist working at a Texas public safety lab, was accused of falsifying results in several drug cases in early 2012 and then quickly suspended. Salvador had worked on about 4900 drug cases since 2006, but a review by the Texas Forensic Science Commission found in April 2013 that he was promoted despite having ‘struggled with corrections and an overall understanding of the chemistry.’ A Texas appeals court ruled that Salvador’s work should be deemed invalid, unless proved otherwise by retesting or new evidence.

Multi-state mayhem

In September 2015, Oregon state police announced an investigation of forensic scientist Nika Larsen, who worked at a crime lab in the city of Bend, for allegedly altering drug evidence. She stole more than 700 controlled substances in pill form between January 2013 and August 2015, including morphine and methamphetamine. In December 2016, Larsen received a three-year prison sentence for abusing her post.

Because Larsen tampered with this evidence, over 2500 criminal cases in Oregon underwent review, and all those pending that relied on her analysis were dismissed outright or a motion was sought to retest evidence. In Umatilla County alone, 70 cases were dismissed as of August 2016, and another 40 were expected to be dismissed.

Kamalkant Shah, a lab technician who worked at a New Jersey State Police drug testing unit from 2005 to 2015, was accused in March 2016 of faking drug analyses – what’s known as ‘dry-labbing’ – and was suspended without pay. Since then, evidence involving roughly 1300 defendants in criminal drug cases has been retested due to his involvement. Four cases have been dismissed so far, yet no criminal charges have been filed against Shah.

Overall, about 100 cases, mostly criminal, have reportedly been overturned because of Salvador’s actions. David Weeks, the district attorney from Walker County, Texas, says at least 24 cases were vacated in his jurisdiction alone, yet Salvador was never indicted.

Another example is Derek Thrush, a Montana crime lab chemist charged in February 2018 with three counts of criminal possession of methamphetamine he allegedly stole from work for his own use. Although Thrush pleaded not guilty in April, Missoula County has already dismissed several criminal cases because of his actions, according to James McCubbin, deputy county attorney.

‘In Missoula County, there are 11 cases that I have declined since this came to light in February, directly due to the involvement of Thrush,’ McCubbin explains. So far, he says about 25 criminal drug cases were entirely or partly vacated due to Thrush’s illegal activity. McCubbin expects that he won’t go to trial because a plea agreement will be reached.

One of the most recent US scandals involves Ana Romero, an analyst at a crime lab in El Paso in Texas, who in April 2018 was accused of dry-labbing blood alcohol samples. Romero worked at the lab from 2006 until 2015, and defence attorneys have estimated that 22 people were wrongly convicted of intoxication-related offenses due to her misconduct.

In response, El Paso County district attorney Jaime Esparza has stated clearly that the lab’s scientific techniques are not in doubt. ‘The science of gas chromatography is well-accepted and validated as a means of testing a person’s blood for alcohol content,’ he said. The investigation is in the ‘very early stages,’ according to Romero.

Such forensic fraud cases are not restricted to the US, and similar problems have been uncovered in the UK. Last year, allegations of data manipulation by two employees at the forensic toxicology company Randox Testing Services (RTS) in Manchester affected approximately 10,000 criminal cases that mostly involved drug-driving. In Yorkshire alone, nearly 2000 criminal cases were thrown into doubt. The individuals involved were immediately suspended and later dismissed.

International intrigue

After RTS alerted the UK Forensic Science Regulator (FSR) about the data quality problems in January 2017, it quickly became clear that the lab’s results were questionable and evidence would have to be re-analysed. The FSR, Gillian Tully, estimated that retesting would take two to three years. The instability of some of the drugs combined with the different analytical approaches between labs will probably mean differences between the original results and those obtained on retesting, Tully noted. 

We now have lawyers trained in things like gas chromatography

Josh Lee, criminal defence attorney

‘This is a serious and widespread issue that is being addressed at a national level, and we are doing absolutely everything we can locally to support that national effort,’ North Yorkshire Police’s assistant chief constable Phil Cain declared at the time .

Meanwhile, another major forensics controversy has since erupted in the UK. London’s Metropolitan Police announced in May 2018 that it is ‘urgently’ reviewing allegations that a scientist in its forensic lab mishandled evidence related to 33 separate criminal investigations, 21 of which involve rapes and other sexual assaults.

The scientist in question, who was suspended in March, is specifically accused of ‘not completing the requisite forensic examinations’ and having ‘wrongly informed’ investigators about the progress of forensic examinations. The materials now under re-review were submitted for forensic examination between 2012 and 2017. The Met’s investigation aims to determine whether these actions compromised criminal justice, and to take corrective action if needed.

That these controversies have hit the UK is not entirely surprising. Last year, a study led by University College London examined nearly 1000 cases involving criminal evidence that came before the UK Court of Appeal of England and Wales between 2010 and 2016, and found that evidence had been misinterpreted for 218 of these cases.

‘For 22% of the cases where the appeal was upheld, there were issues with evidence being misinterpreted at the original trial, including all “criminal evidence” of which forensic science was a component,’ explains study co-author Ruth Morgan, who directs the UCL’s centre for the forensic sciences.

Canada has endured its own forensics scandal, with significant implications for its judicial system. At the centre of that storm was the Motherisk drug testing lab at the Hospital for Sick Children in Toronto, which fielded requests from children’s aid societies to test hair samples for drug and alcohol use in child protection cases. Between 2005 and 2015, the facility evaluated more than 24,000 hair samples from over 16,000 people, and more than 9,000 of them tested positive.

Canadian child welfare agencies had paid for this testing because it was supposed to provide scientific proof of substance abuse, but questions arose about the reliability of the lab’s results in late 2014. In response, the Ontario government reviewed the facility and found multiple failures, concluding that the lab’s hair-strand drug and alcohol testing between 2005 and 2015 was ‘inadequate and unreliable’ for use in child protection and criminal proceedings.

The hair scare

Even though this investigation was ongoing, Canada’s Ministry of Child and Youth Services ordered children’s aid societies to discontinue using the Motherisk lab’s results in April 2015. A high-level commission was ultimately established to determine whether inaccurate hair tests performed under this programme significantly affected individual child protection cases. In February 2018, the commission found these results had a ‘substantial impact’ on the outcome of 56 of the 1271 child protection cases it reviewed, breaking up families.

‘The judges we spoke with expressed great concern that the Motherisk hair testing was unreliable and could have wrongly influenced decision making in past child protection cases,’ the commission noted. Its report advised that expert evidence, particularly that involving novel science, should undergo ‘more rigorous scrutiny’.

The science of hair testing has come under scrutiny before. In 2015, the US Federal Bureau of Investigation (FBI) concluded that at least 90% of trial transcripts examined as part of its microscopic hair comparison analysis review contained ‘erroneous’ statements.

Before mitochondrial DNA testing was used to analyse hair in criminal cases, prosecutors throughout the US routinely relied on microscopic hair comparison to link a criminal defendant to a crime. In that practice, analysts microscopically observe various characteristics of hairs, typically comparing samples obtained from a crime scene to those taken from a suspect. This is not DNA analysis, and a major criticism has been that these examiners have often not simply testified in court that the hairs were microscopically similar, but have actually asserted or heavily suggested a definitive match.

Overall, the US government identified nearly 3000 cases worked prior to the year 2000 in which FBI examiners potentially submitted reports or testified in trials using microscopic hair, and the agency had reviewed approximately 500 cases as of March 2015. The FBI examiners provided testimony to incriminate a defendant at trial in 268 of these cases, and the FBI said they made erroneous statements 96% of the time. The FBI investigation remains ongoing, and the agency expects to provide an update in the autumn.Beyond microscopic hair analysis, uncertainty has also surrounded several other nascent forensic techniques like complex DNA mixture analysis, which involves examining DNA samples that have multiple contributors. 

‘DNA mixture interpretation has been my speciality, and I have been concerned about the science behind it since around 2010 – it was obvious if you have different experts analyse the exact same evidence, they would reach very different, or even opposite, conclusions,’ says Greg Hampikian, a biology and criminal justice professor at Boise State University in Idaho whose lab is involved in forensic biotechnology. He and colleagues published a key study in 2011 in which the same DNA mixture evidence was provided to 17 independent North American analysts to observe if their conclusions were inconsistent.

‘Only one analyst agreed with the initial conclusion that came from the Georgia Bureau of Investigation that was used to convict someone in a rape case,’ Hampikian recalls. The study found that just one of the examiners concluded, as did the original DNA examiners who participated in the case, that the suspect could not be excluded.

False positives

Concerns about forensic science have lurked for some time. Major science advisory bodies in the US and UK had warned about deficiencies in the field that require action.

In 2013, the US National Institute of Standards and Technology (Nist) initiated a study, known as Mix13, which involved more than 100 crime labs analysing the same DNA mixtures in five mock cases. The complexity of the mixture increased in each case, and in the final case, which was the most complex, about 70% of those labs falsely included a DNA profile that was not actually in the mixture.

Nist has disseminated these findings to the broader community through oral presentations at conferences and elsewhere. However, the agency still hasn’t officially published the results, despite their significant implications for the legal system.

‘Nist has sat on that study for five years,’ says Hampikian. At the time of writing, Nist was expected to submit the Mix13 study for publication in a peer-reviewed journal shortly.

And this is not the first example. The US President’s Council of Advisors on Science and Technology (Pcast) issued a report in September 2016, during President Obama’s tenure, that identified a need for clarity on scientific standards behind forensic methods. It also called for an evaluation of specific techniques – including those used to compare hair strands, DNA samples, bite marks and firearm markings – to determine their scientific validity and reliability.

The Pcast warning came several years after the US National Research Council (NRC) released a report that identified a lack of ‘rigorous and mandatory’ certification programmes for forensic scientists in the US. The congressionally requested study also identified a need for stronger standards to analyse and report on forensic evidence, as well as a dearth of peer-reviewed published research establishing the scientific bases and reliability of many forensic methods.

In the UK, authorities have also sounded alarm bells. Most recently, Tully cautioned in her annual report in January that cuts to forensic science budgets represent an ‘ongoing risk’ to the British justice system. She said continuing financial pressures have eroded the time available for professional development in the field, and described the failure of forensic firms to implement required quality standards.

Before one can address whether forensic techniques are rendered suspect by the recent scandals, experts like US criminal defence attorney Josh Lee suggest that the so-called ‘forensic sciences’ must be separated into ‘real science’ and ‘police activity being held out as scientific’. The former category includes DNA, chromatography and spectrometry, while the latter includes hair strand analysis, fibre matching, pattern recognition, as well as bite mark and blood pattern evidence, according to Lee, who is the forensic science co-chair for the American Chemical Society’s (ACS) chemistry and the law division.

‘Real science’ versus ‘police activity’

‘The faux-sciences are suspect in and of themselves,’ Lee says. ‘The real sciences are not suspect other than the inherent weakness of the human elements involved.’ Difficulties introduced by human analysts include sloppy work, lousy or non-existent lab procedures, a lack of mandatory regulatory oversight, dishonesty, and the use of imprecise language without proper explanation, he explains.

To root out these problems, Lee suggests that defence attorneys must be equipped with the necessary scientific training. ‘The reason this is getting caught more is that we have a more scientifically intelligent and educated defence bar,’ he states. ‘We now have lawyers trained in things like gas chromatography on the defence side – they are actually talking about the science behind the cases.’

Defence lawyers must know what questions to ask to probe the science being used to implicate their clients. The ACS sponsors three science courses for lawyers, through Axion Analytical Labs in Chicago, that focus on things like the use of gas chromatography with flame ionisation detection on ethyl alcohol samples, and scientific techniques for testing drugs in their pre-consumption form, like Fourier-transform infrared spectroscopy and colorimetrics.

Thus far, almost 300 lawyers have graduated from the ACS forensics chromotography class since it began in 2009, 115 lawyers have completed the forensic drug analysis course from its inception in 2010, and almost 100 have completed the driving under the influence of drugs class since its start in 2014.

Although the state of forensic science in the US looks bleak with all of the negative publicity, it could be the exact opposite. ‘In some ways, the US has been leading the way in terms of identifying where there have been problems with forensic science,’ says Morgan. ‘What is happening in the US is encouraging because it means that we are facing the issues.’

However, US Attorney General Jeff Sessions withstood much criticism when he decided in April 2017 to shut down the National Commission on Forensic Science, which advised the federal government on how to enhance the reliability of forensic science. But since then, Sessions has vowed to conduct a needs assessment of US forensic science labs to examine workload, backlog, personnel and equipment needs. He also quickly appointed a new Senior Forensic Advisor position at the Department of Justice (DOJ) to interface with forensic science stakeholders and advise department leadership.

In addition, on 29 May the DOJ’s National Institute of Justice announced the formation of a new working group, made up of forensic lab directors from across the country, as well as forensic science researchers, to address the technological needs of the forensic science community.

Many of the problems with the less-established forensic science disciplines are amenable to technological solutions, such as new methods of objective measurement and the inclusion of statistics. For example, Nist suggests that the subjectivity of these techniques can be significantly reduced through methods like computer analysis, which can produce ‘a statistically meaningful score of similarity’ when a human examiner testifies in court about a particular piece of evidence ‘matching’ a defendant.

‘We lawyers are starting to ask for data,’ says Lee, who co-founded the ACS courses. ‘This sort of forensic misconduct is not new, the lawyers just now know what to look for.’

Hampikian, who founded and directs the Idaho Innocence Project, agrees. ‘I don’t think things have gotten worse on the whole, but what has happened is that we are finally learning how to question some of the technologies we took for granted,’ he says.