The Great Engine of Truth Not So Great for Eyewitness Evidence

I recently received a copy of a law review article on the effectiveness of cross-examination for getting at the truth of eyewitness evidence. Prof. Jules Epstein explores in depth the efficacy of the "great engine of truth" as applied to eyewitness testimony, and reports that it falls far short. (Cite: Epstein, Jules, The Great Engine That Couldn't: Science, Mistaken Identifications, and the Limits of Cross-Examination, 36 Stetson L. Rev. 727 (2007)).

Prof. Epstein frames the issue with a quote from the Supreme Court case of Watkins v. Sowders:

In Watkins v. Sowders, the Court found identification testimony no different from other categories of proof and cited Wigmore's dictum in holding that cross examination would suffice to establish or debunk the reliability of the evidence as follows:

[W]hile identification testimony is significant evidence, such testimony is still only evidence, and, unlike the presence of counsel, is not a factor that goes to the very heart - the ‘integrity’ - of the adversary process.

Counsel can both cross-examine the identification witnesses and argue in summation as to factors causing doubts as to the accuracy of the identification - including reference to both any suggestibility in the identification procedure and any countervailing testimony such as alibi.

Epstein points out that "[a]t least two flaws are manifest in the Watkins
analysis." Specifically:

Given the significant incidence of mistaken-identification convictions, it is not clear if one can claim that such testimony does not go to the integrity of the adversarial process. Several decades of scientific study raise the question whether cross-examination can in fact secure reliable verdicts in cases of mistaken identification.

Following a detailed and extensively researched analysis, Prof. Epstein concludes that the highly revered truth-seeking tool of cross-examination, while perhaps effective at rooting out liars, is utterly ineffective at uncovering the truth when faced with a witness who is confident, but honestly mistaken about what he or she remembers -- which accounts for the majority of cases in which mistaken identification has led to wrongful conviction.

In short, cross-examination was developed to counteract perjury, not genuine mistakes.

[T]hese accolades also show the limits of cross-examination and its inutility in confronting the truthful but mistaken witness, or in demonstrating the lessons of the science of perception, memory, and recall. A tool designed from its inception to root out liars is ill-suited for the task of exposing the risk or reality of mistaken identification.

Epstein also goes into some detail about the problems associated with getting the results of scientific studies on eyewitness memory in front of juries, without the aid of expert testimony. For example:

Q: Ma'am, you've told us that being robbed was a highly stressful event, correct?
A: Yes.
Q: And you must know that the higher the stress, the more memory and perception get distorted. In other words, the more stress, the more it is likely that people make mistakes. You know that science has proved this, don't you?
A: Look, I don't know about studies. But I sure know what I saw. That man did it--this event was a powerful one, I'll never forget it.

That, coupled with a prosecutor arguing the "she'll never forget that face" line in closing, and a defense attorney is going to be hard pressed to sway a jury's intuitive belief that the identity of the perpetrator is "burned in the mind" of the victim forever. That is, without an expert to testify to the uncontroverted findings of 30 years of social science research that counteract that widely held -- but false -- intuition.

Prof. Epstein makes a strong case for the admission of expert testimony on eyewitness factors, as well as detailed jury instructions, to counteract jurors' "near-religious faith in the accuracy of eyewitness accounts." He concludes that "ongoing reliance on cross-examination as a great engine will, sadly, contribute to the continued phenomenon of wrongful convictions based on eyewitness testimony."

Read more :: Permalink

Conviction Overturned for Failure to "Seriously Consider" Expert Testimony on Eyewitness Factors

The Illinois Court of Appeals overturned a conviction Friday in People v. Allen (PDF) (update: now on Westlaw at 2007 WL 2821966), for the trial court's "failure to conduct a meaningful inquiry" into proposed expert testimony on eyewitness factors by Dr. Steven Penrod.

Subjects of proffered testimony were to include the effects of stress, weapon focus, cross-race, lack of correlation between witness confidence and accuracy, and police procedures. The appellate court's analysis of those factors is imperfect (for example, the court found that cross-race research would not have applied because the witness was married to an African American, which the research shows to be largely irrelevant).

Nonetheless, the Illinois court was concerned with the fact that the prosecutor had argued effects that conflict with research findings:

The State's comments: "[t]here is no higher degree of attention than someone pointing a gun at you;" "[defendant's] face is burned in her memory forever;" " [tlhere is no doubt she was certain;" and "if she is so certain, there is no reason and no doubt that you should be certain."

That doesn't stop our courts from excluding the same research on the same grounds, but I digress. The court didn't go as far as to say that it was error to exclude the expert testimony, but citing Illinois case law, found that it was error to fail to provide a reasoned basis for its exclusion.

Neither at trial nor in this appeal does the State challenge the reliability of the research cited by Dr. Penrod. Nor did the trial court when it rejected the proposed testimony. The court merely said it did not believe: "experts in this particular case will assist the jury in determining the identification in
this case. I believe it would probably confuse them more and I believe that the
instruction that's provided by the Illinois Pattern Jury Instructions is sufficient."

The appellate court also made some affirmative findings on the question of whether the research is common sense/not beyond the ken, which is helpful in light of the fact that courts most often rely on the claim, however unsupported, that these factors are common sense such that expert testimony would not be helpful. Quoting from another Illinois case:

The research challenges the claim that the jury does not require expert assistance. As the prosecutor understood, reasonable people well might believe an eyewitness will be more accurate when faced with a weapon and when the witness shows
confidence in the accuracy of her identification. The expert testimony "dispels myths or attacks commonsense misconceptions about eyewitness identifications, such as the effects of stress and weapon focus on the accuracy of identifications." Tisdell,
338 Ill. App. 3d at 467. In Tisdell I we said: "Numerous studies in the area of eyewitness psychology indicate there is a significant potential for eyewitness error and that jurors have misconceptions about the abilities of eyewitnesses." Tisdell, 316 Ill. App. 3d at 1157.

In other words, there is something wrong with excluding expert testimony on weapon focus and stress on the grounds that their effects are common sense, but nonetheless allowing prosecutors to argue the opposite of the common sense view, as they do in trial after trial.

Further, the court observed that:

No careful scrutiny took place in this case. Relevance of the different parts of Dr. Penrod's proposed testimony was not seriously considered. Nor their weight. The conclusion that the proposed testimony would confuse the jury had no considered basis. The balancing test requires a weighing of "probative value against its prejudicial effect.".

In other words, this presumably means that hard data will be required in support of future attempts to exclude expert testimony on common sense grounds. Fortunately, all the hard evidence (PDF) is firmly in favor of defendants seeking to get expert testimony in front of jurors.

Finally,

Because of the trial court's failure to conduct a meaningful inquiry into Dr. Penrod's proposed testimony, under the specific circumstances of this case, we reverse the defendant's convictions and remand this cause for a new trial.

The court could have taken it a step further and held that it was error to exclude the testimony, and further that it was error to allow the prosecutor to make arguments that conflict with what the court held to be common sense, but it's a start.

UPDATE: This case is now available on Westlaw: People v. Allen, --- N.E.2d ----, 2007 WL 2821966 (Ill. App. 1 Dist. 2007).

Read more :: Permalink

A Bad ID, and Bad Law, in Louisiana

In a recent Louisiana bench trial, eyewitness ID expert Robert Shomer testified in front of a judge in the 21st Judicial District on a collection of factors tending to undermine the reliability of eyewitness evidence. The local press reports:

Had the robbery case been tried before a jury, rather than a judge alone, the testimony of the specialist in eyewitness identification would not have been allowed. Because the defendant had waived his right to a jury trial, the judge permitted the testimony of Robert Shomer, an experimental psychologist who has taught at Harvard and UCLA.

The ID in this case was a police procedure in which the cops showed the witness a photo lineup including five color photos and one in black and white. It shouldn't be hard to guess which one was the suspect, which one got picked, and which one ended up on trial and found guilty.

Though it didn't persuade the judge in this case, Shomer testified as follows:

For a picture lineup to be fair, the pictures must be presented in the same format and should match the initial description of the perpetrator to the same extent, he said.

And:

Like a scientific study, a photo lineup should be presented in a double blind manner, he said. When the effectiveness of a drug is being studied, the nurse actually administering the drug shouldn’t know which patients are receiving the real drug and which are getting the sugar pills. That way, the nurse can’t, either consciously or accidentally, provide any indications to the patients, Shomer said.

In a photo lineup, the person showing the pictures to the witness shouldn’t know which photo is that of the suspect. Without intending to do so, a police officer might provide the witness with subtle clues as to which picture is that of the suspected person, Shomer said.

Unfortunately in this case, the judge was unable to get past the positive ID, notwithstanding the fact that it resulted from an unacceptably suggestive police procedure clearly calculated to induce the witness to pick their suspect.

There's no question in my mind that this sort of police misconduct should result in an exclusion of any identification evidence, even under the ill-conceived current due process standard of Manson v. Brathwaite. And defense attorneys have to keep pushing that the standard be applied with teeth until Manson is eventually revisited by the Supreme Court.

Until then, expert testimony would seem to be an important remedial measure to educate juries about the fallibility of eyewitness testimony and stave off wrongful convictions. Except in states like Louisiana -- and there are many states like Louisiana in this respect -- where the courts don't allow expert testimony in front of juries because they think the work of battalions of social scientists is common sense. This, despite clear evidence (PDF of poll of DC jurors) that it is far from common sense.

Read more :: Permalink

Federal Court Gets ID Science Terribly Wrong, Denies Expert Testimony

In one of the worst eyewitness ID decisions I've read, the US District Court for the Eastern District of Virginia denied expert testimony in US v. White earlier this month, and in the process revealed a substantially flawed comprehension of eyewitness research. US v. White, Slip Copy, 2007 WL 1768718 (E.D.Va. June 15, 2007).

The case involves a police officer as a witness, and despite a wealth of research demonstrating that cops are no better at remembering faces than anyone else -- even when specifically "trained" to do so -- the court relied substantially on that fact in distinguishing this case from other cases in which expert testimony on eyewitness issues had been admitted previously:

First, unlike Lester and other cases admitting expert testimony on eyewitness identification, the eyewitness in the instant case is a trained police officer. Officer Catterton served as a military police officer for the United States Army for six years. He has been trained to be constantly observant of his surroundings, especially the people he encounters on the job. He has been specifically trained to observe and remember the faces he comes across so that they may be later identified. This is in sharp contrast to the participants in Dr. Cutler's cited studies, the majority of whom were college students.

In fact, the specific question of cops vs. college students as eyewitnesses has been studies, and all the evidence shows that trained police are no better at remembering perpetrator identities than random college students. See, for example, a 1998 study by Christianson & Karlsson: Police personnel as witnesses to a violent crime, 3 L. & Crim. Psychol. 59 (1998) (cops are no better at identifying perpetrators than civilian college students). And yet judges continue to rely on this false notion in rendering critical decisions, and in turn juries end up relying on the same false notion in their deliberations, because actual experts are prohibited from testifying to the actual scientific findings that clearly clash with the widely held common sense view.

The police-are-better-witnesses misconception was only the first of the court's errors in rendering this decision.

Confidence and Accuracy

The court also misinterpreted the research on the relationship between the confidence of an eyewitness, and that witness's accuracy. The court characterized the research as showing that "an extremely confident eyewitness is more likely to be accurate than a less confident eyewitness, but only by a modest amount." In fact, the research shows that there is a modest correlation between confidence and accuracy only under very narrow circumstances, more resembling a vacuum than the real world, which rarely occur in actual practice -- namely, if the confidence of a witness is recorded immediately following the identification procedure, and only when no other factors are present to corrupt the witness's perception, such as routine suggestive police practices. In most real-world contexts, there is no correlation between confidence and accuracy, and in some cases even a negative correlation has been observed.

This case was no vacuum. By the time the officer made the identification in this case, he had already been exposed to the strong suggestion of Mr. White's guilt -- he was told that Mr. White had been arrested previously on a similar charge, was suspected of the current charge, and on those grounds the officer requested that his arrest photo be sent over. By the time the officer-witness encountered Mr. White in person, there was little doubt that he would make an identification, regardless of whether or not Mr. White was the actual culprit. There is little question that expert testimony on the lack of correlation between confidence and accuracy would have helped dispel a common misconception in the minds of jurors, and thus enhancing the accuracy of their deliberations.

Cross-Racial IDs, Mugshot Exposure, and Show-Ups

The court went on to deny expert testimony on the cross-race impairment, namely that members of one race are more likely to mis-identify members of another race, despite the fact that the phenomenon is well-documented in the scientific literature and has been shown to be misunderstood by juries.

The court made similar errors with respect to the mugshot commitment effect, and the suggestivity of show-up procedures.

As the exonerations continue to show, juries are going to keep getting it wrong until judges allow science into the courtroom to undo the widely held errors of common sense on the reliability of eyewitness evidence. In the meantime, at least there's Tennessee.

Read more :: Permalink

Tennessee High Court Overturns Death Penalty Conviction For Failure to Admit ID Expert

I'm on the road so this is brief and nearly old news by now, but with that out of the way:

This great opinion (PDF) came down from the Tennessee Supreme Court this week in State v. Copeland, overturning its long-standing precedent excluding eyewitness testimony as common sense, citing (among other things) a poll (PDF) conducted by the Public Defender Service for DC, showing that jurors tend to overestimate the reliability of cross-racial IDs. The defendant in the case had been sentenced to death, but the Tennessee Supreme Court overturned the conviction for the trial court's failure to admit the expert testimony of Jack Brigham on the cross-race effect.

One point that was raised that never receives enough attention is the disparity in admissibility requirements for government vs. defense expert witnesses. So-called "narcotics experts" (i.e., cops) are routinely allowed to testify as experts on the subtleties of drug transactions, and yet eyewitness ID experts (i.e, PhD psychologists) are routinely precluded from testifying:

Another author has observed that while experts are often not permitted to testify regarding eyewitness testimony, police officers and other law enforcement officials are regularly permitted to testify “concerning the general way criminal
schemes and enterprises operate and the usual meaning of criminal slang and code words.” D. Michael Risinger, Navigating Expert Reliability: Are Criminal Standards of Certainty Being Left on the Dock?, 64 Alb. L. Rev. 99, 132 (2000). The author contrasts the “technical” knowledge of the law enforcement officials with the educational and scientific credentials of experts on eyewitness identification. Id. at 131-35.

It is the educational training of the experts and empirical science behind the reliability of eyewitness testimony that persuades us to depart from the Coley rule. Times have changed. Today, many scholarly articles detail the extensive amount of behavioral science research in this area. See generally Gary L. Wells et al., Eyewitness Evidence: Improving Its Probative Value, 7 Psychol. Sci. Pub. Int. 45, 47-49 (2006) (for a brief history of psychological research of eyewitness testimony).
There are literally hundreds of articles in scholarly, legal, and scientific journals on the subject of eyewitness testimony.

And on how cross-examination is rarely enough to undermine faulty eyewitness testimony in the minds of the jury:

Further, the research also indicates that neither cross-examination nor jury instructions on the issue are sufficient to educate the jury on the problems with eyewitness identification, contrary to the conclusion reached by the majority in Coley. See, e.g., id. (“[E]ven when presented with an eyewitness who was quite thoroughly discredited by counsel, a full 68% still voted to convict.”) (citing Elizabeth Loftus, Reconstructing Memory: The Incredible Eyewitness, 15 Jurimetrics J. 188, 189-90 (1975)). “Considered as a whole, the studies of juror knowledge and decision making indicate that expert psychological testimony can serve as a safeguard against mistaken identification.” Steven D. Penrod & Brian L. Cutler, Preventing Mistaken Identification in Eyewitness Identification Trials, Psychology & Law: The State of the Discipline 89, 114 (1999).

More in the opinion (PDF), which is well worth a read.

Read more :: Permalink

Onward from LeGrand

Decisions like LeGrand (PDF), and the Third Circuit's superb decision last summer in United States v. Brownlee (PDF), 454 F.3d 131 (3d Cir. 2006), suggest that courts are starting to warm up to the idea that the social science research can help limit the role mistaken eyewitness testimony has played in creating wrongful convictions. I think it's likely that these cases will hasten the day when expert testimony of this sort is routine in cases that rely primarily on eyewitness testimony. I also think this is important: For too long, the judicial system has virtually ignored this important research. Courts ignore science at their peril.

But it is also not enough. There are not enough experts, there never will be enough experts, and expert testimony will often be limited even in cases where it does occur. And even if expert testimony perfectly assesses the identification and its reliability (or unreliability), that will still not always be enough to counteract a witness who genuinely but mistakenly looks across the courtroom and says: "That's him! I'll never forget that face as long as I live."

So where else do we go from here? Quite apart from expert testimony, we need to improve ways of selecting, educating and instructing jurors throughout the trial. The legal system also can reform itself in other ways. Gary Wells, for example, has suggested that the legal system should prevent "lineup jeopardy" (the risk of mistakenly being accused of a crime simply by virtue of being placed in a lineup) by limiting the ability of police to place someone in a lineup or photo spread -- through a requirement that police independently make some showing of individualized suspicion probable cause (PDF) before an identification procedure occurs.

But probably the most important piece of the reform puzzle will occur outside the courtroom, since it involves preventing eyewitness evidence from becoming unnecessarily "tainted" BEFORE it makes its way into court. That is, by adopting police procedures that prevent misidentifications from occurring at all, rather than trying to blunt or diminish their effect afterwards.

The legal system can do a lot to shape the procedures police use to handle eyewitness evidence, and we now know a lot about which police identification procedures work and which do not. A lot of what we know has even been compiled into guidelines (PDF) by the Department of Justice. Some police departments will choose to follow them on their own, and will provide sufficient training so that officers can professionally handle eyewitness evidence in every case (see, e.g., Hennepin County, Minnesota, and Northampton, Massachusetts (PDF)). The more police that adopt this sort of reform on their own the better because other solutions are more complicated and will take more time.

But inevitably some jurisdictions will lag behind and resist. In those jurisdictions, some form of mandate will be required. Some states will go the legislative route, as West Virginia, Maryland, Virginia and others have done. Others will go the executive branch route, as New Jersey has done. And still other states will require the judicial route, with courts forcing police to change by excluding eyewitness evidence infected by shoddy and outdated police practices.

Some have said that reforms like these cannot be legislated or directed by courts; police need to want to change their conduct before reforms can take hold. Many reform-minded police officers would say otherwise. A good example has arisen in the debate involving recorded interrogations. Many police departments and officers vigorously opposed recording interrogations, usually on the ground that it would deter statements. Our police in Washington D.C. said the same thing when they fought recording legislation. But now that the results are in, police I have spoken with (including police here) love recording interrogations because it produces evidence they can be confident in -- confident in their own minds that any confession is reliable and confident that the evidence will hold up to any challenge in court when scrutinized by judges, lawyers and jurors.

The same thing will eventually happen with eyewitness procedures -- once the bias in favor the status quo is broken, police will see how reformed eyewitness procedures are just another example of how simple, smart police reform can make the system tougher, smarter and fairer for everyone involved.

That's where we're headed. It's just a matter of time.

Read more :: Permalink

LeGrand reversed! (NY Murder Conviction Overturned for Denial of ID Expert)

As previously reported, last month the New York Court of Appeals heard oral arguments in People v. LeGrand, a murder case in which the trial judge last year refused to admit expert testimony (PDF) on the reliability of eyewitness identifications, and further held that such evidence is unreliable under the "general acceptance" standard set forth in Frye v. United States.

Today, New York's high court reversed (PDF) the conviction, on grounds that it was an abuse of discretion for the trial court to exclude expert testimony on the reliability of of eyewitness identifications. With respect to the lack of correlation between confidence and accuracy and "the effect of postevent information on accuracy and confidence malleability," it is error to exclude expert testimony in a case in which the identification is central and there is a lack of corroborating evidence.

In short, the law on expert testimony on eyewitness ID in New York now appears to be:

It is an abuse of discretion to exclude an ID expert when:
- The case turns on an uncorroborated eyewitness ID; and
- The subject matter of the expert's testimony is generally accepted by experts in the field and beyond the ken of the average juror.

And the following factors affecting eyewitness reliability are both generally accepted and beyond the ken:
- Lack of correlation between confidence and accuracy;
- The adverse effect of confirming feedback on eyewitness accuracy; and
- The malleability of eyewitness confidence.

On the above topics, the court at least implied that Frye hearings would no longer be necessary to establish general acceptance prior to admission of expert testimony. The court did not find the "weapon focus effect," however, to be generally accepted among the relevant scientific community, despite substantial research on the subject.

As previously reported, the case was expertly briefed by the Center for Appellate Litigation in New York, with amicus support (PDF) from the Legal Aid Society, Neighborhood Defenders of Harlem, and the New York State Criminal Defense Lawyers Association.

[Cross-posted at Daily Kos.]

Read more :: Permalink

Eyewitness ID Research: Generally Accepted Science? (People v. LeGrande, NY Court of Appeals)

On Wednesday, the New York Court of Appeals heard oral arguments in People v. LeGrand, a case in which the trial judge last year refused to admit expert testimony (PDF) on the reliability of eyewitness identifications, and further held that such evidence is unreliable under the "general acceptance" standard set forth in Frye v. United States.

The case stems from a murder that occurred in upper Manhattan in 1991, and the first lead on a potential culprit emerged a full seven years later, when investigating officers showed a photo array including Mr. LeGrand to witnesses. Based on a seven year old memory of the events, one witness claimed that Mr. LeGrand was the killer; other witnesses admitted that they saw "similarities," but stopped short of making a positive ID -- and others could not make an ID at all. The first trial resulted in a hung jury, but the second ended with a conviction and a 25-year-to-life sentence for Mr. LeGrand.

On appeal, the core issue is whether or not the trial court should have admitted expert testimony on the reliability of eyewitness identifications, particularly in a case where the only ID occurred seven years after the incident, there was a weapon involved, and other issues tending to reduce the reliability of eyewitness identifications were central to the sole evidence purporting to connect Mr. LeGrand to the murder.

During oral arguments yesterday, at issue was Saul Kassin's survey on the "general acceptance" of various issues relating to eyewitness evidence, including the "weapon focus effect" -- an important issue in this case (in addition to confidence-accuracy correlation, the effect of post-event information on eyewitness memory, and confidence malleability). Kassin's survey revealed that 87% of experts polled found the "weapon focus effect" to be reliable and supported by the scientific research, yet the trial court judge refused to allow testimony on the subject, and further found that it is neither reliable nor generally accepted in the relevant scientific community.

This case was expertly briefed by the Center for Appellate Litigation in New York, with amicus support from the Legal Aid Society, Neighborhood Defenders of Harlem, and the New York State Criminal Defense Lawyers Association. Definitely a case to watch, and thanks to Kate for the heads up.

Read more :: Permalink