NIJ Acknowledges "Marred" Results of Illinois Pilot Project

The NIJ Journal put out an article this week called "Police Lineups: Making Eyewitness Identification More Reliable," acknowledging the unreliability of eyewitness memory and the role the police procedures play in making it more or less reliable. The article gives some detail on the simultaneous vs. sequential debate, including Nancy Steblay's continuing great work testing the efficacy of double-blind sequential lineups in the field.

Significantly, the NIJ acknowledges that the Illinois pilot project (the infamous "Mecklenburg Report") was fatally flawed. ("The stunning implications of the Illinois Pilot Program have since been marred ... by questions about the methodology used.") Presumably, then, the NIJ will no longer include the results of that junk study in future serious discussions on lineup reform, such as the project being developed for Dallas County. Taxpayers, not to mention the wrongfully accused, can't afford another colossal waste of public resources on an agenda-driven charade like we saw in Illinois.

Read more :: Permalink

Lipstick on a Pig, Part Deux: Scientists vs. Prosecutors

We reported recently that the so-called "Illinois Report" on lineup procedures was found to be unscientific, unreliable, and generally adds no value to the dialogue on the most accurate and reliable police lineup procedures. Today, the Chicago Tribune is running a story, in the wake of the definitive debunking (PDF) of that police-sponsored report from Chicago, suggesting that the best lineup format has yet to be identified.

Two problems with this characterization strike me with equal force. The first problem is that the "disagreement" is one of conflicting agendas, not unresolved questions of science. Scientists, with their well-known bias in favor of facts, are in general agreement that sequential lineup procedures are less likely to put innocent people in prison. Every peer reviewed study on the topic bears out this finding. A meta-analysis (PDF) was conducted by leading psychologists to extrapolate the comparative accuracy rates of the two types of procedures, and the clear finding was that sequential lineups are far less likely to result in an innocent person being identified. When "moderator variables" are considered, the two methods are also largely equivalent in their likelihood of bringing about a correct selection of the actual perpetrator, when he is present in the lineup. The general consensus among scientists is that "sequential lineups are superior."

To the extent that there is disagreement on this point, it is not a disagreement among scientists or any sort of conflict in empirical findings. Rather, it is an objection by prosecutors at the audacity of scientists who dare to suggest that science should inform criminal justice policy. After all, the Chicago Police Department has been doing lineups for decades. Who are these scientists to tell them their procedures are flawed? It's like a bunch of M.D.s telling cigarette manufacturers that smoking causes cancer.

So the Illinois Report manages yet again to avert attention from the uncontroverted findings of social scientists, namely that sequential lineups are less likely to result in wrongful convictions. To characterize the status of the debate as "not yet resolved" is to fall prey to the prosecutorial agenda to resist reform at any cost; it's tantamount to concluding that the smoking/cancer link has yet to be established because Philip Morris executives find that fact to be in conflict with their profit motive. Except that it's a little less clear what interest is being protected by the prosecutors who resist lineup reforms, since presumably they share the interest in prosecuting actual criminals, while protecting the innocent.

The other problem with the Chicago Tribune's framing of the debate on lineup reform is the myopia with respect to the simultaneous-vs.-sequential question. The fact is, the recommendation that lineups be presented one person at a time rather than all at once is a single bullet point amongst a long list of recommendations to make lineup procedures more reliable. To borrow some words from eyewitness researcher Gary Wells:

[L]et's not become myopic because of the sequential lineup. Let’s not forget that we need to institute a broad range of improvements to lineup procedures that make the terms "biased procedure" and "motions to suppress based on suggestive procedures" mere memories from the past. I continue to be amazed as to why prosecutors are not at the forefront of this effort to jettison suggestive procedures. Do prosecutors like to have these "motion to suppress identification" hearings? Of course not. Would not prosecutors prefer to have cleaner identifications to carry forward to trial? Of course they would. Are prosecutors at the forefront of efforts to: (1) improve instructions to witnesses, (2) select better fillers for lineups, (3) implement procedures that eliminate influence from the lineup administrator, (4) assess eyewitness certainty in an unbiased manner, (5) make sure that full records are kept? The answers are (1) no, (2) no, (3) no, (4) no, (5) no. I apologize to my prosecutor friends in Clinton, Iowa, in New Jersey, and in other places who are clear exceptions to this characterization of prosecutors not assuming leadership roles on identification issues. Still, the failure of law enforcement to significantly improve their lineup procedures is primarily because prosecutors have not pressed for these improvements.

With 27 wrongful convictions revealed by the Innocence Project in Illinois alone -- 19 of which resulting from faulty eyewitness evidence -- you'd think Illinois prosecutors would be eager to put themselves at the forefront of the reform effort, rather than fighting tooth-and-nail to preserve status quo procedures with a long track record of failure.

Read more :: Permalink

NO WAY TO PUT LIPSTICK ON THIS PIG: PANEL OF SCIENTIFIC EXPERTS FINDS THAT HIGHLY-PUBLICIZED CHICAGO POLICE STUDY OF ID PROCEDURES IS JUNK SCIENCE

Approximately a year ago, a highly-publicized study out of Illinois claimed to have raised questions about some of the procedural reforms being adopted around the country to improve the handling of eyewitness evidence. The study, taking the form of a pilot project spearheaded by Chicago police across three counties, purported to reveal that current identification procedures protected against mistaken identifications better than reforms (PDF) proposed by respected social scientists, based on extensive research on eyewitness memory. In other words, the Chicago police were happy to report that, notwithstanding the 19 wrongful convictions recorded in Illinois that resulted from faulty eyewitness evidence, everything was just fine and no pesky (scientific) reforms were needed.

Despite the fact that the "report" trumpeting the study was never subjected to peer review, and despite the fact that it was authored by a lawyer for by the very same Chicago police department that had fought reforms in Illinois for many years, these "findings" were trumpeted on the front page of the New York Times and have since served as serious impediments to reform in Legislatures around the country.

But something seemed rotten in Illinois, since the study's results were at odds with the recommendations of nearly every scientist who had studied the issue and the methodology underlying the pilot project appeared something short of scientific. According to one article on the topic, "the design of the project contained so many fundamental flaws that it is fair to wonder whether its sole purpose was to inject confusion into the debate about the efficacy of sequential double-blind procedures and to thereby prevent adoption of the reforms." Leading eyewitness researcher Gary Wells described the study's methodology as "extremely problematic."

Bearing out these suspicions, a new analysis was released yesterday, in which a "Blue Ribbon" panel of social science luminaries concluded that the study was "crippled by a design flaw that made the study's conclusions a dangerous basis for shaping public policy." This scholarly article does a fine job of showing that the Illinois pilot project on eyewitness lineup procedures was a sham, and a scandalous waste of taxpayer dollars. On that point, six out of six respected scholars agree.

The panel now weighing in includes Nobel Laureate Daniel Kahneman of Princeton and Harvard Professor and author Daniel Schachter, and other eminent scholars across the social sciences. In their own words, (PDF) "the design [of the Illinois study] guaranteed that most outcomes would be difficult or impossible to interpret," and the study's fundamental flaw has "devastating consequences" with respect to its scientific merits.

The story of the Chicago ID report is, in other words, one that is sadly familiar these days. Facts are cooked to suit the theory. The cooked facts are then trumpeted to an uncritical media, and the public understandably gives its officials the benefit of the doubt. But sooner or later the truth comes out, as it did here when the Illinois "study" received the sort of critical scientific analysis that should have occurred all along. In fact, this was just the sort of scientific rigor that the Legislature ordered when it appropriated tax-payer funds to conduct the study in the first place.

We now know that the failure to heed the Legislature's guidance resulted in the waste of many taxpayer dollars and, at least as importantly, the waste of a critical opportunity to determine how to fix a system that is currently relying too heavily on demonstrably mistaken eyewitness testimony. As the scientists suggest, the failure of Illinois now means that we need to await the results of properly-conducted studies in other jurisdictions to obtain this information.

Read more :: Permalink

Exonerations Spur Reform Nationwide

Law.com has a nice current snapshot of the nationwide movement to reform eyewitness ID procedures, which continues to gain traction in response to the mounting number of DNA exonerations.

One common theme throughout the proposed reform measures is a requirement for blind lineup administrators, or an alternative method that has the same effect of preventing the person conducting the lineup from influencing the witness's choice. Many of the proposed bills would also require cautionary instructions prior to any lineup procedure, in which witnesses are alerted that the perpetrator may or may not be present in the lineup. This instruction alone has been shown to reduce false IDs by as much as 41.6%, in lineups where right guy isn't present in the lineup.

Some of the other proposed reforms are less targeted at specific practices, and instead just require that law enforcement agencies develop written policies governing eyewitness procedures, in keeping with research on factors that are known to make lineups more or less reliable.

According to Scott Ehlers of the National Association of Criminal Defense Lawyers (NACDL):

Bills seem to have been going further this year than in the past. Exonerations and the wrongful convictions: I think that is really pushing the issue.

Ehlers mentions the West Virginia eyewitness reform bill which is now a law, as well as eyewitness procedure best-practices bills being proposed in California and Georgia, which Ehlers says have a good chance of passing as well. Overall, 16 bills proposing eyewitness ID reform were submitted across 10 different states this legislative session.

At the same time that reforms have been spreading nationwide, controversy over a study conducted in Illinois continues to draw attention to the same issues. A lawsuit filed by NACDL seeks to expose what many have suggested were serious flaws in the study's methodology, which its authors suggest support the status quo -- the same status quo which we know has resulted in wrongful conviction after wrongful conviction. According to Ezekiel Edwards of the Innocence Project, the questionable study from Illinois has only served to stimulate debate, and in conjunction with a continuing wave of exonerations, has helped push reform efforts forward.

Read more :: Permalink