In the homicide case against Michael Robinson, accused of killing two people in Duquesne in 2013, the computer program, TrueAllele, found that DNA from a black bandanna recovered near the crime scene was 5.7 billion times more likely his than coincidence.
Mr. Robinson’s attorneys want to know how it is that the program reached the results it did.
“The witness in this case is a computer,” defense lawyer Ken Haber said. “You can’t cross-examine a computer. The Constitution demands, and justice requires, we be permitted to find out what the computer is doing to come up with its answer.”
But when he and co-counsel Noah Geary sought the source code for the program, they were denied. The judge ruled that it could cause harm to Cybergenetics, the Oakland-based company that created TrueAllele, which uses probabilities and statistics to determine if a suspect’s DNA is in a complex biological mixture.
This and other similar cases bring up a key question as probabilistic DNA programs become more prevalent.
How can courts, attorneys and juries examine and verify the results of extremely sophisticated and complex computer statistical modeling? And should business interests trump the defendant’s right to question the evidence presented?
“TrueAllele is a new, innovative program. Nobody knows what it does. Shazam, there’s an answer,” Mr. Haber said. “We’re basically being asked to accept answers, ‘because I say it’s so,’ and that’s just wrong.”
The requests
Courts in six states, including Pennsylvania, have denied defense requests for TrueAllele source code, but the issue remains unsettled.
One of TrueAllele’s biggest competitors, STRmix, now gives its code to defense attorneys when asked.
And in the federal court in the Southern District of New York in June, a judge ordered that the New York City Office of the Chief Medical Examiner turn over the source code for its DNA mixture program, Forensic Statistical Tool (FST), in a gun case.
“FST is a relatively new tool that has not been extensively examined or tested in federal court, and the results obtained from the use of FST on DNA samples recovered from crime scenes are potentially devastating to a criminal defendant,” U.S. District Judge Valerie Caproni wrote. That “increases the need of the court to be diligent about FST’s reliability prior to admitting FST results into evidence.”
Mr. Robinson’s attorneys hope that a split among jurisdictions will help persuade the Pennsylvania Supreme Court to consider their pending appeal. Mr. Robinson’s trial has now been delayed until January. If convicted, Mr. Robinson could be sentenced to death.
And in King County, Wash., attorneys for Emanuel Fair, a convicted sex offender who is accused of first-degree murder with sexual motivation in the Nov. 1, 2008, death of a 24-year-old software engineer, will have a hearing on the question of the TrueAllele source code next month.
In the Fair case, TrueAllele found, among other pieces of evidence, that DNA found on the victim’s robe, recovered in a nearby dumpster, was 3.89 billion times more probable a match to the defendant than coincidence.
In both cases, defense attorneys question how it is that the computer program reached the results it did.
The fight for secrecy
When an attorney requests Cybergenetics to run TrueAllele on DNA data obtained from a crime scene, the company turns over hundreds of pages of documents.
The case packet, as it’s called, includes background on DNA, tutorial videos, validation studies, national guidelines, and hundreds more pages of data listed in columns and graphs and charts.
Mark Perlin, who created TrueAllele, offers to train attorneys to use the program and even to run their own data through it free of charge on a cloud-based server. But source code is off limits.
With a medical degree and two Ph.D.s in computer science and mathematics, Dr. Perlin started work on TrueAllele 22 years ago, and the computer program has been revised 25 times since. A point of pride for him is that the fully automated process, he believes, takes human bias out of DNA mixture interpretation.
He argues that the source code is unnecessary to understand how the program works, and that TrueAllele is so complex -— written in the computer language MATLAB -— that it could take as much as eight years to read and comprehend it.
“In my opinion, it is wholly unrealistic to expect that reading through TrueAllele source code would yield meaningful information,” he wrote in an affidavit for the Fair case in Washington state.
Dr. Perlin also claims that his source code contains trade secrets that, if revealed, would jeopardize his company.
“Cybergenetics has invested millions of dollars over two decades to develop its TrueAllele system, the company's flagship product,” he wrote. Although the product is patented, “patent protection is not automatic, and litigation can cost millions of dollars.”
Dr. Perlin said he does not disclose the source code to anyone, including his own employees.
As for the idea of the court issuing a protective order so that no one outside of a defense expert can view the source code, Dr. Perlin said it would do little to protect the company if there is a breach.
“Once someone has read through your technology, there’s nothing you can do to put it back,” he said. “There are no effective remedies.”
Amy Jeanguenat, the CEO of Mindgen, a forensic DNA consulting firm, said forcing the release of source code could make development stall.
“It’s about protecting innovation,” she said. “Commercial companies are critical to the field and the justice system.”
As for STRmix officials now making its source code available upon defense request, Dr. Perlin shrugs it off, saying it means the computer programming includes no trade secrets, or that it has no monetary value.
“If you were to ask practicing scientists, would you rather be able to read the source code or test out the program on real data, you’ll get a unanimous answer — they want to test the program,” Dr. Perlin said. “They don’t care about the source code.”
Greg Hampikian, director of the Idaho Innocence Project, said Dr. Perlin is right.
“I look at validation. If we put in known mixtures, does it interpret them correctly?” he said. “For me, the proof is in the pudding, not in the recipe.”
Mr. Hampikian, who is also a professor of biology and criminal justice, has used TrueAllele four or five times with several samples, and he said, he has never had a problem.
Combine that with all of the validation studies that have been done -—both by Dr. Perlin and the labs now using TrueAllele -—and Mr. Hampikian is satisfied.
“Unless they’re all lying, the thing works fine,” Mr. Hampikian said.
Existing precedent
In Mr. Robinson’s case, his defense attorneys thought they were entitled to the source code.
They believed there was already precedent in Allegheny County to require disclosure. In June 2014, President Judge Jeffrey A. Manning of Common Pleas Court issued a memorandum opinion in the case of Martell Chubbs, who was charged in Los Angeles County Superior Court with homicide. Chubbs’ defense attorney had issued a subpoena for Dr. Perlin as a material witness and asked for him to provide the source code for TrueAllele, which she argued was material to the California case. Dr. Perlin’s program linked Chubbs to the crime through biological material that had been recovered at the scene.
Because Dr. Perlin and his company are in Pittsburgh, his efforts to fight the subpoena were argued here.
In his opinion, Judge Manning wrote, “It is beyond cavil that Dr. Perlin is a material witness and that the evidence that is sought to be produced is material.”
He continued, “The evidence that places the defendant at the scene of a crime is without question ‘material.’ The means by which Dr. Perlin arrived at his opinions is likewise material. The argument that Dr. Perlin is not a material witness and/or that the evidence sought to be produced is not material is specious.”
But when Mr. Robinson’s attorneys filed their request for Dr. Perlin’s source code, he and the Allegheny County District Attorney’s Office fought them.
Common Pleas Judge Jill E. Rangos denied the request after a two-day hearing in the fall, finding that the source code was not material to Mr. Robinson’s ability to pursue a defense. She issued a written opinion in February.
“An order requiring Cybergenetics to produce the source code would be unreasonable as release would have the potential to cause great harm to Cybergenetics,” she wrote. “Rather than comply, Dr. Perlin could decline to act as a commonwealth expert, thereby seriously handicapping the commonwealth’s case.”
In the same opinion, Judge Rangos dismissed what Judge Manning had written in the Chubbs case, writing that all her colleague was doing was enforcing a subpoena and ordering Dr. Perlin to provide evidence in California. Ultimately, she noted, the California Superior Court did not require the production of the source code, and while the issue was appealed, Chubbs eventually pleaded guilty, and the issue became moot.
Then in another Allegheny County case decided shortly after Mr. Robinson’s, Common Pleas Judge Edward J. Borkowski denied a source code request by the defense in the case against Allen Wade, who was accused of killing two sisters in East Liberty. Judge Borkowski cited Judge Rangos’ decision.
Judge Manning, too, in a recent gun case, denied the request for source code, again citing the Robinson case.
Mr. Robinson’s attorneys call the varying opinions in Allegheny County “irreconcilable.
“Why are judges letting him get away with ‘Just take my word for it. You don’t need the source code?’ ” Mr. Geary asked. “It is anathema to due process of law.”
The fight for access
Defense attorneys for Mr. Robinson and Fair argue that even though TrueAllele produced a match statistic for their clients, they are unable to understand how the results were generated.
In any other instance where an expert is called to render an opinion in a case, the opposing side must be given all of the materials that were used to reach a conclusion. In the case of TrueAllele, they say that includes the source code.
“We’ve never been provided an expert report. I’m not interested in his promotional materials,” Mr. Geary said. “He’s a businessman marketing a product.”
As for the published validation studies Dr. Perlin cites, the defense contends that nearly all of them have had Dr. Perlin as either a lead or co-author.
“Something may be scientifically reliable, but that does not mean it is without flaws,” Mr. Geary said. “And those flaws may rise to the level of reasonable doubt. Those are grossly different standards.”
In the Fair case, Dan Krane, a professor of biological sciences at Wright State University and chief executive officer for the DNA consulting company Forensic Bioinformatics, wrote that while validation studies are important, it is the source code that serves to implement the underlying concepts of the program.
“Human experts are expected to explain how they arrive at a conclusion … this same expectation can and should apply to a computer program such as TrueAllele,” he wrote. “A careful evaluation of the computational steps taken by TrueAllele would allow it to be determined if the program: 1) reflects what is described by Dr. Perlin, 2) is consistent with the practices of the forensic DNA profiling community, 3) is free from bugs and errors, and 4) if TrueAllele can and does provide sufficient explanations for the observed data in this case.”
David Balding, a professor of statistical genetics at the University of Melbourne in Australia who has created his own mixture interpretation software and posted the code online, agreed, saying it is “absolutely inconceivable” that there aren’t errors.
“The question is how many of those are inconsequential?” he said.
In the case of STRmix, which was created by the New Zealand Crown Research Institute of Environmental Science and Research and Forensic Science South Australia, two errors in source code have been identified. One did not lead to any errors in cases, the organization said in a March 18 statement. The other did lead to errors, but none that resulted in a mistrial.
Itiel Dror, a British cognitive neuroscientist at University College London, appreciates the concept of TrueAllele’s computerized program taking the potential for human bias out of DNA interpretation but says bias can’t be completely eliminated.
“The human is in the software in the source code,” he said. “The source code isn’t given by God or aliens.”
Rebecca Wexler, a lawyer in residence at the Data & Society Research Institute in New York City, advocates releasing source code.
“When a person’s life and liberty are on the line, we care about due process,” she said. “We care about public transparency.”
The concept is especially important in something like probabilistic genotyping, which is still considered to be “frontier” science.
“In the wild, things go wrong in ways developers didn’t expect,” Ms. Wexler said.
She also questioned the idea of putting a company’s intellectual property interest above a defendant’s right to a fair trial.
“That’s so stark, this contrast between life and liberty and financial interest,” she said. “It’s questionable whether a trade secret interest is at stake in criminal court at all.”
But even if it were, she continued, there are protective orders that can be put in place to protect Dr. Perlin’s financial interests from the source code. They are used in civil proceedings all the time.
“We have a standard solution in this industry,” Ms. Wexler said. “Why shouldn’t that solution be available to criminal defendants who also have life and liberty at stake?”
Ms. Wexler believes that the claim that providing the source code will cause economic harm to Cybergenetics is overblown.
“If these programs truly are as reliable as the manufacturers claim, there’s nothing to lose and only benefits to gain from this additional scrutiny,” she said.
Paula Reed Ward: pward@post-gazette.com, 412-263-2620 or on Twitter: @PaulaReedWard.
First Published: August 29, 2016, 4:00 a.m.
Updated: August 29, 2016, 4:25 a.m.
