False Positive: When forensic science fails [Full version]

Video Statistics and Information

Video
Captions Word Cloud
Reddit Comments
Captions
At 6:45 in the morning of November 3, 1984, a man walked into a police station on the north side of Milwaukee to report that he’d found what appeared to be a body, lying in a lot near his home. That’s where police discovered 63-year-old Ione Cychosz, wearing only “a pair of white socks and a left shoe.” Her blue shirt and bra were pulled up behind her head, which was bloody and swollen. She’d been beaten to death and possibly raped. Neighbors described the victim as “a nice woman.” She was known for collecting aluminum cans and playing bingo at a Ukrainian church on Friday nights. That’s what she’d been doing the night she was killed. "And she was let off after midnight by friends who normally dropped her off after the bingo games. They observed her walk in front of her home and those are the last people who saw her alive." One of the real challenging things about this case was you had no eyewitnesses. You of course had no confession. You had no motive. This was before DNA profiling, before home security cameras were common, there were no fingerprints on the knife they found nearby. But the killer did leave one clue: He bit her. “Investigators took 60 photographs of numerous bite marks.” "The bite marks are being analyzed by a forensic dental specialist.” Dr. L. Thomas Johnson, a dentist and professor at Marquette University, examined the bite marks on the victim the day she was found. And he gave the cops a lead. He told the detectives you are looking for a guy with an abnormality in one of his upper teeth. So the cops began knocking on doors. Three days after the murder, the detectives knocked on the door of Robert Lee Stinson, a quiet 20-year-old whose only criminal record was a fine for shoplifting from when he was younger. He lived with his mother and siblings in a house right by the crime scene. Stinson told the police that on the night of the murder, he came home from a party around 11:30, went to bed, and slept until morning. But what caught the attention of the police was that Stinson had “an upper right front tooth missing.” That missing tooth made one of the detectives believe they "got the guy". His partner later told the Milwaukee Sentinel that they went back inside and told Stinson some jokes to get him laughing, to confirm that there were those teeth. So the state had Stinson appear at a secret hearing before a judge — called a “John Doe" investigation in Wisconsin — where they could have Johnson, the dentist, check his teeth. The John Doe was the first time that I met Mr. Stinson, but I'm looking at this kid who is very warm, very affable, very cooperative. Everything in my gut, my experience, six years in the DA's office, the Sensitive Crimes Unit. No way. No way did this guy do this. He thought they would eliminate Stinson as a suspect. But that's not what happened. Johnson looked into Stinson’s mouth for 20 seconds, the same amount of time it takes to read this: And then Johnson said Stinson’s teeth matched a sketch that he and a police artist had made — a sketch of what the suspect’s teeth would look like based on the bite marks. Nobody brought the drawing to the hearing, but Johnson said “it resembles what I see here.” Stinson’s mouth, he said, “certainly is consistent with what I drew. It is remarkable.” And that was enough for the court to order a full dental examination. But if they’d had the sketch in the room, they would have seen that it said the suspect would be missing a lateral incisor. Stinson was missing a central incisor, his right front tooth. But now the dentist was on the record saying that they matched. The field of bite-mark analysis is one of the so-called “pattern matching” disciplines in forensic science. Fingerprints are the best known example, but the group also includes shoe prints, tire treads, handwriting, and bullet casings. We call them “forensic sciences” but they developed independently of what we typically think of as the “science world”: The universities and research hospitals, peer-reviewed journals, grant-funding organizations. Those institutions of basic research have fed some forensic fields, but many others lack that foundation. You go up the street to the University of Wisconsin and you look for the Department of Fingerprints, or the Department of Shoe Impressions. It's not there. That’s because these fields developed mostly in response to crime scenes and the obligation to study whatever bits of information they contained. The examiners look at samples closely, and make a subjective assessment of how similar they are. You know, it's funny. When people watch CSI or think about forensic science, they often think that it's very high-tech and glossy and perfect and often automated. And in the real world the pattern identification sciences do not work like that. It's not that subjectivity makes it illegitimate or necessarily incorrect. But we ought to know something about how accurate it is. For the criminal justice system, this is still a new idea. So at the time of Stinson’s case, nobody knew how accurate, or inaccurate, bitemark analysis was. The man who examined Stinson’s teeth, L.T. Johnson, is a forensic odontologist. And bite mark evidence is just a part of what they do. When we have tragedies, plane crashes or other kinds of terrible events sometimes we use teeth that are found to help identify human remains. And that's an important tool. And, I believe, quite an accurate one. “Rescue workers were there quickly, but they quickly learned there was no one to rescue.” The same year he worked on Stinson’s case, Johnson led a team identifying victims of a plane crash that killed 31 people. He would go on to help identify 11 decomposing victims found in serial killer Jeffrey Dahmer’s apartment. Forensic odontology had been an established field for decades. But it was only in the 1970s that they decided to go beyond identifying victims by their teeth and started identifying suspects by bite marks. It began with a 1975 case, People versus Marx, in California. Edgar Marx was accused of murdering his landlady and leaving a bite mark in the cartilage of her nose. The bitemark evidence was deemed admissible even though the appeals court conceded that there was “no established science” behind it. And it became the germinal case. Because all of the precedent-establishing cases in high courts across the country cited back to Marx. Fast forward a few more years. Ted Bundy goes on trial and there's bite mark evidence in the Ted Bundy case and it made movie stars out of a couple of forensic dentists. And bite mark evidence took off. This norm of precedent, of conforming to past decisions, it exists for a reason. But it happens to be the opposite of how science works. Precedent is really important in the judicial system. We want constancy, predictability, and we have a rule that like cases should be decided the same way. Science, by contrast, is inevitably and unavoidably and always contingent, progressing, growing changing, shifting as new research tells us new things and corrects past misunderstandings. The legal system is not really designed to adapt to that. So a game of judicial telephone convinced the courts that bite mark identification was an accepted area of science, when in fact, it had never been tested. There is no science that we have seen to support that. It's really quite astonishing. It's the one thing that gets me truly agitated, because there's just no science to support it. A month after Johnson collected models and photographs of Stinson’s teeth, he came back with his final conclusion. “The teeth of Robert Lee Stinson would be expected to produce bite patterns identical to those which I examined in this extensive and exhaustive analysis.” He said the evidence was “overwhelming.” The sketch Johnson referenced at the prior hearing? It was never mentioned again. He was showing me, you take this overlay of the pictures of his teeth and you put it on the pictures of the bite wounds on Mrs. Cychosz’s body and you kind of see how they align. And it did the same thing with the models. But I wasn’t completely convinced that I could convince a jury based on what Dr. Johnson did, that Mr. Stinson was guilty of an offense like this. I said we need a second opinion on this case. Johnson recommend Raymond Rawson, who was known as one of the leaders in the field. So the detectives flew to Las Vegas to deliver the evidence. And they reported that Rawson “gave a verbal confirmation of Johnson’s finding.” But he didn’t spend much time on it. According to a memo book that one of the detectives kept and later published, Rawson met them at their hotel room at the Four Queens, “took a look at the x-rays and the molds, and said that was good enough for him.” Five days later, Robert Lee Stinson was arrested at his home. And we decided that, given a brutal horrific crime like this, we now have two witnesses who are saying it's him. And matched against that is what, my gut level feeling? And so we issued the charge. The US has an adversarial judicial system where the judge presides over a trial, but isn't responsible for uncovering the truth. Instead, it's up to the opposing lawyers to present evidence and witnesses that support their version of events. “And ultimately the theory is of course that through this conflict of ideas and interpretations the truth will emerge.” But from the start, Stinson could tell he wasn’t entering a fair fight. He prepared a letter asking for a replacement for his lawyer, who “only took his case two weeks” before trial. “Your Honor, I’m facing life for something I did not commit,” he wrote. The judge denied his request, which came in the middle of the trial and she also denied his lawyer’s motion to exclude the bitemark evidence. She said “there are adequate standards and controls in the area of forensic odontology.” “It is a recognized area of science.” “Frankly at the time it was not a close decision for me. Even though it was unique testimony, I didn't have anything in front of me that indicated that it was not reliable and it certainly was helpful to the jury, I think. And relevant to the issues and so I admitted the evidence.” At the time, Wisconsin only required that forensic testimony be relevant and helpful to the jury. Judges weren’t required to assess its reliability. “You have an older woman who's been raped and murdered. We've got an expert who's saying we know who did it. If the judge had excluded that evidence, he would have gone free." And it would have been a legal stretch for her to do that given the state of the law and the precedent at the time. As the appeals court would point out in a footnote, by the time of Stinson’s trial, bitemark evidence had been accepted in 19 jurisdictions and rejected by none. The court in Stinson’s case was no different, and the bitemark evidence went before the jury. The first thing that happens when forensic experts take the stand is that they’re prompted to list their credentials, to show that they’re qualified. “That makes sense from a certain perspective. But there's also a danger that the jury misunderstands the power of that experience.” “Dr. Johnson seemed to make sense to me when he testified. He certainly was qualified, being a professor at the university in the dental school." "Here you have this very learned dentist who has no reason to lie, who comes in and tells you this is rock solid science. How is the jury to decide he must be wrong? That's extremely compelling.” Dr. L.T. Johnson walked the jury through the evidence. He even brought models of Stinson’s lower teeth and one of the bite marks, so he could show how they matched. He concluded his testimony by saying that the bite marks “would have to have been made by Robert Lee Stinson." The second dentist, Raymond Rawson, testified that “there was no question that there was a match to a reasonable scientific certainty." Stinson’s lawyer did try to get a defense expert of his own, but that expert was never called to counter the prosecution’s dentists in court, because after he examined the evidence he agreed with them. Instead of two sides battling it out before the jury, there was one side with two experts making false statements about the science, and one side with no experts at all. During Stinson’s trial, the prosecution and defense would ask the two bitemark experts a combined 240 questions. But they didn’t ask the most important one. “The real question that we should be asking is what is the evidence that shows us that people in this field reach accurate results?" “It's not rocket science.” Forensic scientists can have protocols and guidelines, use well-accepted tools and technology, but we won’t know if their methods are actually reliable until we have what’s called an “error rate study.” You take a bunch of bite mark examiners, give them samples of bite marks and measure how often they make a correct identification. To this day, nobody has ever conducted a proper error rate study for bite mark analysis. “And there's really no incentive to do any research, because courts admit it. If courts stopped admitting it they would do the research, right, because they want it to be introduced at trial.” Studies that have been done with human cadavers show that there’s a lot of distortion when bite marks are made in skin, even in a laboratory setting. The same set of teeth will make marks of different sizes and shapes depending on the skin type, the amount of force, and the orientation of the bite. Missing teeth can look like they’re there. Teeth that are there can look like they’re missing. One study conducted by forensic dentists found an unsatisfactory level of agreement among examiners on whether a given injury was even a human bitemark. But during Stinson’s trial, Dr. Johnson told the jury that there was “no margin for error” in this case. And then Stinson took the stand. He said he was at a party until about 12:30. Went home, and then later went back out, to go to the store with a friend. He said as they walked behind his house, he heard footsteps and shushing coming from the back of the alley. But when Stinson first spoke to the police three days after the murder, he had only told them that he went home after the party and went to bed. As the prosecutor pointed out at the trial, he had changed his story. “It's my experience that the jury looks a lot at those alibis and if somebody's lying for no other reason than perhaps trying to get out of something, that is pretty heavy evidence." After a three-day trial, and less than two hours of deliberation, the jury found Stinson guilty of first degree murder. And he was sentenced to life in prison. On appeal, Stinson challenged the admissibility of the expert testimony in his trial. But the Wisconsin Court of Appeals affirmed his conviction. They were impressed by the dentist's elaborate methods. Dr. Johnson used an "Acrylic ring” “Special camera” “Three dimensional indentations" “Overlay technique” “75 individual tooth marks” They actually claimed “the reliability of the bite mark evidence in this case was sufficient to exclude to a moral certainty every reasonable hypothesis of innocence." “It's so absurd. Tarot card readers have a very complex, ancient methodology. It doesn't mean that they can tell your fortune.” And with that ruling, Stinson became the precedent-setting case for bite mark evidence in Wisconsin. Seven years later, the US Supreme Court issued a decision -- the Daubert ruling -- which should have changed the entire landscape for forensic science. It said that trial judges must assess whether expert testimony is based on reasoning or methodology that is “scientifically valid.” Daubert talks about reliability, validity, empirical data, error rates, peer review. It has all the good stuff and scientists would look at it and feel their hearts would patter, they'd be cheered, because it says all the right words. But the problem with that is it also says it's a flexible standard. The Daubert ruling gave judges a new “gatekeeping role,” but at the same time, it said that “vigorous cross-examination and presentation of contrary evidence” are still an "appropriate means of attacking shaky but admissible evidence." So even though judges are supposed to evaluate expert testimony, they can still choose to offload that responsibility. “They don't want to be responsible for letting the killer go free because they excluded the evidence." After all, the system has always relied on the attorneys to cross-examine witnesses they disagree with and present contrary evidence. “Public defenders are by and large overwhelmed and often don't have the funding to hire their own independent experts. But they shouldn't have to. We shouldn't have to be defending against unreliable scientific evidence.” And ultimately, it’s up to the jury to decide what they believe. “I mean the instructions are very clear that the jury does not have to give any further weight to an expert than any other witness. What kind of weight they want to give to them is totally up to them.” But the jury doesn’t have basic information about the error rates of forensic methods like bite mark analysis. Those studies don’t exist, in part because judges have never required them. “I don't think there is evil intent anywhere in this circle. It's a collection of institutional participants sometimes taking the easy way out." “Nothing is suitable in our existing system for making the scientific determination that this evidence is questionable and here's what juries and judges ought to know about it. It's not happening.” As Robert Lee Stinson began his life sentence, the evidence in the case was boxed up and locked away. That included the blue pullover shirt the victim was wearing when she was killed. It was a crucial piece of evidence, but nobody knew that yet. And it would be 24 years before the true killer was found. “Think about what you were doing between the years 1985 and 2009. All those years you spent getting your education, building a resume." "Mr. Gorbachev, tear down this wall." "Getting married. Buying a home." "The Hubble Space Telescope is released." "Having children." "Can you explain what internet is?" "Building up retirement." "O.J. Simpson not guilty of the crime." "Watching your children grow up. Funerals of your family members and your loved ones. "iPhone." "It's just an enormous amount of time. You come out and how are you supposed to explain, 'Well I was convicted of murder but I didn't do it.' That baggage stays with you forever.” Robert Lee Stinson refused to admit guilt, so he was never granted parole. By 2003, he’d been in prison for 18 years. That’s when when he wrote to the Innocence Project at the University of Wisconsin. “It was a request for assistance saying that he was innocent and that he was convicted on bite mark evidence.” They took the bite mark evidence out of storage and sent it to four forensic odontologists to review. And they found that the dentists who testified against Stinson, L.T. Johnson and Raymond Rawson, had made a “series of errors that resulted in an incorrect determination.” They said Johnson’s technique of placing a semi-opaque overlay on top of the bitemarks obscured the fact that they didn’t match. And they showed how misleading it could be to hold a dental model up to a bite mark impression, which Johnson had done in front of the jury. It makes it look like Stinson’s teeth fit, but it also looks like these three random sets of teeth fit too. Overall, they concluded that “Stinson can be excluded from” having made the bite marks. “We said look, it's not him. And the prosecutors looked at it and they said we gotta talk to Dr. Johnson and they came back and they said, no we stand by it. It's him. Where? How can you see that? But they did.” Fortunately, the bite marks weren’t the only physical evidence in the case anymore. The same day Stinson was convicted, December 12, 1985, a paper was published in the journal Nature reporting that DNA could be used in forensics. “It is evidence that promises a breakthrough future for law enforcement officials.” “These genetic fingerprints are a powerful tool for solving mysterious crimes.” We know today that DNA is a powerful form of evidence, but it’s worth digging into why that is. In our cells, there’s a small section where a short DNA phrase gets repeated several times. We all have it in the same location, but the number of repeats varies between people. I might have 9. You might have 15. We have instruments that can essentially count the repeats by their size. And because we can count them, we can survey a bunch of people to see how common each number is. So let’s say police find a DNA sample with 9 repeats. Well, if 30% of people have 9 repeats, that’s not very strong evidence. But scientists have identified more than 13 different locations on our DNA where we vary in these kinds of repeats. And we have two numbers for each location, because we inherit one from each parent. So maybe I have 9,9 at one location; 14,16 at another; 12,10 at a third, and so on. The more locations you add, the smaller the number of people who could match. That’s what makes DNA so powerful: Our samples are unique, they distinguish us from each other, but it’s also quantifiable, so we can calculate how much it distinguishes us. It comes with statistics. Compare that to fingerprints. We suspect that they’re quite unique, but we can’t calculate how unique because fingerprints lack the kind of universal structure that the DNA molecule has. Instead of looking at a predetermined location and counting up repeats, fingerprint examiners have to decide what’s notable about each print. “And there are so many different permutations, so that makes it very difficult to say this particular configuration is one in a million or one in a quadrillion or whatever the way you could do with DNA.” Researchers have started figuring out what to measure in a fingerprint to get useful statistics. But the other pattern-matching disciplines are further behind. And with bite marks, it might not even be worth trying. In 2010 the federal government gave L.T. Johnson a $700,000 research grant to make bitemark analysis more scientific, using statistics. His team borrowed live pigs and made bite marks in their skin with dental models. They used software to measure the bite marks for tooth width, angle, and spacing. And when those measurements were used to guess the source teeth, they identified the correct dental model only 2% of the time. Even in these controlled conditions, skin did not reliably record the dimensions of teeth. But that was already becoming painfully clear. "Wrongfully convicted." "Wrongfully convicted." "Wrongfully convicted of rape." "He spent more than 10 years in prison before DNA identified the real killer." "The science in this case was just plain wrong." "Bite mark analysis is now being questioned nationwide." As DNA testing began overturning bite mark convictions around the country, the Wisconsin Crime Lab was analyzing the bra and shirt that the victim in Stinson’s case was wearing when she was killed. The male DNA they found on several areas of her clothing didn’t match Stinson’s DNA. He was excluded as the source again, and again, and again. By 2008, a full DNA profile was developed from the blue v-neck pullover and entered into a DNA databank. Two more years would pass before it yielded a match. But in the meantime, the District Attorney’s office finally agreed that Stinson’s conviction should be vacated. And he was released from prison on January 30, 2009. “I remember the scene of when we walked him out. There was media there.” "It was an emotional day for Robert Stinson and his entire family.” “Well it was very powerful when we saw Robert Lee Stinson walk out of this prison this morning.” “And we stood outside the prison gates and they opened up a sliding chain link gate and it was actually a rather subdued, almost melancholy moment where his sister greeted him with a deep hug.” "Finally justice is done." "But at what cost?" "I can't explain it. I can't explain this. Been a long ride for me. I'm finally out and going to enjoy my life now." “I think the lesson we have and this is, I think, regrettable, is that no matter how careful you are, how cautious you are, how objective you think you are in these cases, you can still be wrong.” “I felt bad. I felt bad. You know, all I can say is, you know, to this case and I think it's the only wrongful conviction that I know of, that I presided over, I'm just grateful we don't have a death penalty.” The year Stinson was exonerated, the National Academy of Sciences released a report that rocked the field of forensic science. “That report was big. It had huge ripple effects throughout the community.” “What it did was really to explode the problem and to say to the world we have a serious issue here.” It found that “no scientific studies” supported bite mark identification. And the problem was bigger than that: It said “a number of forensic science disciplines” were lacking basic research. “It mostly said you need more science in forensic science. Unfortunately it was received somewhat defensively by the forensic science community and I think that slowed down our ability to respond to it, to learn from it, to grow from it.” The report recommended the creation of a new federal agency to direct and fund forensic science research, and serve as a scientific authority for the criminal justice system. “And so when you come into court there would be a bible of this information and the prosecutors would have to work with that, and defense attorneys would have it available to them, and juries would be instructed accordingly.” But Congress never created the new agency. And by 2016, an Obama administration report found few studies that were still few studies that were appropriately designed to assess validity. Only the fingerprints and firearms fields had begun to properly measure how accurate their examiners are. Academic researchers haven’t really taken an interest in running these studies. And even if they did, they’d need access to forensic examiners and lots of test samples. And in the crime labs, they don’t necessarily have the resources or incentives to participate. “You don't have to talk to very many lab directors to find out that they are overburdened and they're undermanned. You don't really have the freedom to say hey I think I'll take a third of my people and conduct some error rate studies.” “Our problem now is just trying to get the studies done.” Without some permanent infrastructure to continually examine the accuracy of the forensic sciences, we can’t be sure that a case like Robert Lee Stinson’s won’t happen again. “The detectives that picked you up yesterday. Did they tell you what this was about? A year after Stinson’s release, the DNA from the victim’s blue v-neck pullover finally solved the crime. “They got a cold hit, that is they weren't looking for anybody in particular they just ran the profile through the database and it hit.” His name was Moses Price Jr., and he wasn’t a suspect in the original police investigation. Around the time that Stinson was arrested, Price pled guilty to two armed robbery charges. He’d also been accused of rape. And in 1991, as Stinson sat in prison, Price beat a man to death and set his house on fire. When the DNA match prompted police to question him, Price confessed. He said he remembered following a woman one night in November 1984. He was drinking and said he blacked out. Price said if he’d known Stinson had been convicted for the murder, he would have stepped up a long time ago.
Info
Channel: Vox
Views: 1,100,705
Rating: undefined out of 5
Keywords: vox.com, vox, explain, false positive, false positive full documentary, false positive full doc, joss fong, joss fong vox, false positive joss fong, joss fong false positive, vox false positive, false positive vox, robert stinson conviction, robert stinson trial, robert lee stinson, forensic odontology, forensics, forensic science, vox forensics, forensics vox, wisconsin, dna evidence, innocence project, ted bundy, dental forensics, true crime, free documentary, documentaries
Id: EO6kYkoCEsA
Channel Id: undefined
Length: 32min 43sec (1963 seconds)
Published: Wed Feb 13 2019
Related Videos
Note
Please note that this website is currently a work in progress! Lots of interesting data and statistics to come.