Law Stack Exchange is a question and answer site for legal professionals, students, and others with experience or interest in law. It only takes a minute to sign up.
Sign up to join this community
To my knowledge most countries use one of two systems for determining the guilt of the accused in criminal cases: “jury of your peers” (who don’t have to justify their decision) vs “judg(es), possibly with lay jurors” (who have to justify their decision). Also some countries allow the striking of deals pre-trial / instead of trial.
Is there solid research on which system performs better (as in how many erroneous convictions, how many erroneous dismissals) in which circumstances?
It is hard to measure accuracy in absolute terms because figuring out the truth is the problem that trials seek to solve in the first place.
This is especially true of cases that go to trial. Cases that are almost sure to be resolved one way or the other by any tier of fact generally plea bargain (in criminal cases) or settle prior to trial (in civil cases). The plea bargaining rate is 90%-95% in criminal cases and less than 5% of civil cases go to trial.
For better or for worse, civil jury trial have become much less common in the last half century, while criminal jury trial rates have been comparatively much more stable (criminal jury trials have become increasingly less common in federal trial courts, however, where the rate at which criminal cases go to trial is about half that of state courts).
The good news is that this suggests that the accuracy of the entire process acting in the shadow of trial in most cases, is much greater than the accuracy of trials themselves, which tend to involve the hardest and closest cases. The bad news is that to the extent that trial outcomes are unpredictable, even those who are innocent or not at fault in a civil case, will sometimes reach an unfair agreed plea bargain or settlement.
As a 1992 law review article comparing outcomes in civil cases between judges and juries notes, "The empirical results prove resistant to simple explanations."
Some key studies summarized here concluded that:
The judge and jury in the Kalven-Zeisel survey of 3,500 criminal cases agreed in 78% of the cases on whether or not to convict. When they disagreed, the judge would have convicted when the jury acquitted in 19% of the cases, and the jury convicted when the judge would have acquitted in 3% of the cases—a net leniency rate of 16%. Disagreement rates were no higher when the judge characterized the evidence as difficult than when the judge characterized it as easy, suggesting that the disagreements were not produced by the jury’s inability to understand the evidence. Disagreement rates did rise when the judge characterized the evidence as close rather than clear, indicating that disagreement cases were, at least in the judge’s view, more likely to be those cases that were susceptible to more than one defensible verdict. Primary explanations offered for the overall differences were differences in judgments about the credibility of witnesses and a different threshold of reasonable doubt. Two smaller, more recent studies using the Kalven-Zeisel method have shown remarkably similar patterns in criminal cases, obtaining 74% to 75% agreement, with a greater leniency of 13% to 20% from the jury. Studies outside the United States have shown similarly high levels of agreement between professionals and juries or lay judges in criminal cases.
For the 4,000 civil trials in their judicial survey, Kalven and Zeisel obtained the same agreement rate of 78% on liability, but disagreement was almost equally divided, so that in 12% of the cases, the jury found for the plaintiff, while the judge favored the defense and in 10% of the cases, the jury found for the defense, while the judge would have made an award. Awards by juries were 20% higher on average than awards by judges. Several smaller, recent studies of civil jury cases in several locations have indicated agreement rates on liability between 63% and 77%, but it is unclear whether any overall change has occurred over time because no national study comparable with the Kalven and Zeisel study has been conducted. Because punitive damages are awarded so rarely (in roughly 3% of contract and tort cases), researchers conducting case-specific judicial surveys have not been able to compare judge and jury decisions on punitive damages.
While the discrepancy between judges and juries in criminal cases (one in six cases) is quite large, this suggests that no more than 1-1.5% of criminal defendants charged with felonies in the system as a whole are inappropriately acquitted by juries, and that no more than about 0.16% of criminal defendants charged with felonies are wrongfully convicted by judges (since only about 10% of criminal trials in the U.S. are bench trials). Moreover, as discussed below, the "inappropriate acquittal" by juries rate as a share of system-wide felony cases is probably lower because there are good reasons to think that the higher conviction rate in bench trials than in jury trials has more to do with the biases of judges than with more accurate fact finding by judges.
A 2010 empirical study looking at the impact of the racial composition of juries on the likelihood of convictions in felony cases in a Florida county found that statistically, a similar number of felony cases (82%) that go to trial are "easy" cases in which any finder of fact would reach the same result, but that the acquittal rate in other cases varied dramatically based upon the racial composition of the jury. This study provides further support for the hypothesis that close cases are more strongly influenced by the implicit biases of the fact finders than by their capacity to assess the facts accurately.
Incidentally, this higher conviction rate is not particular to the United States, it has also been true historically in Japan during periods when it had a jury trial system, even though its legal codes use German legal codes as a starting point.
I have personally had discussions with several South Korean and Japanese judges (while I was in law school and they were pursuing graduate law degrees in the U.S. while on sabbatical from their judicial positions) who say that they have a policy of alway believing the prosecutors in criminal cases and assuming as a starting point, at least, that any defense testimony that contradicts the prosecution's case is a result of lies by the defense witnesses, which doesn't speak highly for the accuracy of bench trials in criminal cases in those countries.
An experienced judge argued in a law review article in the late 1980s based upon his lengthy career presiding over jury trials, that juries were more accurate. Probably the strongest argument he makes is that judges are prone to bias, in part, though what many people would now call the process of regulatory capture. Other experienced trial judges echo this positive assessment of juries as noted in a 2012 article, for example, noting that juries are often better grounded in the reality of everyday life and in community standards than judges.
A 2007 study (full text here) reached a conclusion on the accuracy of jury trials in criminal cases based upon the a priori assumption that judges are at least as accurate as juries, which isn't helpful at resolving your question because it assumes the answer rather than proving it, but does shed some light on the minimum extent to which one or the other triers of fact would have reached the wrong conclusion:
A new Northwestern University study shows that juries in criminal cases are reaching incorrect verdicts. The study, which looked at 271 cases in four areas of Illinois, found that as many as one in eight juries is making the wrong decision – by convicting an innocent person or acquitting a guilty one.
In each case, while the jury deliberated, the judge filled out a questionnaire detailing what his or her verdict would have been had it been a bench trial. The verdicts only matched in 77 percent of cases. The study assumed that judges are at least as likely as a jury to make a correct verdict, leading to the conclusion that juries are only correct 87 percent of the time or less.
The study was conducted by Bruce Spencer, a Northwestern statistics professor, and will be published in the July issue of the Journal of Empirical Legal Studies. Spencer said in a statement that it would take a much larger study to truly predict the accuracy of jury verdicts nationwide in all cases.
The same 2007 study is also critically analyzed here, and notes that judges again are more likely to convict than juries in cases where defendants opt for jury trials (which they do about 90% of the time in criminal cases in state courts).
A different 2007 study used evidence mostly derived from DNA based exonerations in capital rape-murder cases from the 1980s to conclude that the wrongful conviction rate in those cases following jury trials is 3.3% to 5%. Given that judges are much less likely to acquit than juries, this strongly suggests that the wrongful conviction rate for bench trials in serious criminal cases would be much higher than it is in jury trials. The methodology of the study doesn't make it possible to estimate the wrongful acquittal rate in jury trials which is further muddied by the question of the extent to which jury nullification, in the form of an acquittal of someone who is actually guilty beyond a reasonable doubt, is a legitimate exercise of mercy or is problematic - something also influenced by perceptions of where jury nullification is most common (e.g. this may disproportionately benefit police tried for crimes allegedly committed while carrying out their duties).
It is worth noting, however, that juries in capital cases (i.e. those where the death penalty is a potential sentence) are systemically less pro-defendant than other juries because they are "death qualified" which is to say that people opposed to the death penalty are systemically removed for cause from the jury pool, which makes juries in capital cases more likely to convict than juries in other cases. This means that it is also probably likely that wrongful conviction rates are higher in capital cases than in other cases. Indeed, one reason that prosecutors choose to bring death penalty charges in a case is to increase the odds that a defendant will plea bargain to avoid the risk of the death penalty, and to increase the odds that the defendant will be convicted at trial, in a criminal prosecution where the evidence is factually weak compared to a typical felony case that prosecutors would bring.
Yet another 2007 study concluded that:
Drawing on a sample of 798 Ohio criminal justice professionals (police, prosecutors, defense attorneys, judges), the authors examine respondents’ perceptions regarding the frequency of system errors (i.e., professional error and misconduct suggested by previous research to be associated with wrongful conviction), and wrongful felony conviction. Results indicate that respondents perceive system errors to occur more than infrequently but less than moderately frequent. Respondents also perceive that wrongful felony conviction occurs in their own jurisdictions in .5% to 1% of all felony cases, and in the United States in 1% to 3% of all felony cases. Respondents, however, specify an acceptable rate of wrongful conviction to be less than .5%.
A replication of this study with a survey of comparable Michigan officials published in 2008 gave rise to almost identical results.
The review of literature in the 2007 paper surveying Ohio officials also notes the notable fact that 7% of convicted felons whose convictions are reversed on appeal and who are then retried before a new jury are acquitted (I don't have data breaking out the retrial results for capital convictions that are reversed on appeal and retried easily at hand right now), which provides another order of magnitude estimate of the reliability of jury trials (although, of course, cases reversed on appeal are by definition cases whose verdicts are not reliable for some specified reason, relative to cases in which convictions are not appealed or are affirmed on appeal).
About 40% of criminal convictions following a trial are appealed (the appeal rate is very nearly 100% in death penalty cases). This is much higher than the rate of appeals of civil jury verdicts, in part, because convicted criminal defendants who cannot afford a lawyer are entitled to a direct appeal at state expense (although it can be pointless to do so if you already have a felony criminal record, you are likely to be released before the appeal is ruled upon after considering your sentence and time served, and your prospects of success on appeal are poor, which is the case in many minor felonies). About 8% of cases appealed in criminal cases result in a reversal of at least part of a trial court decision (although the figure is about 19% in death penalty cases). So, the odds of a criminal conviction following a trial being appealed, reversed on appeal, and then retried resulting in a conviction is roughly 0.2% (one in five hundred).
There is also considerable evidence to support the hypothesis that certain facts patterns are much more prone to producing wrongful convictions than others. For example, excessive faith in eye witness identification of defendants (especially in cases of white witnesses identifying black defendants who are strangers to the witness) and scientifically unsound forms of expert testimony are particularly common causes of wrongful convictions.
A 2010 review article examining 100 years of research on the question of wrongful conviction notes that "traditional sources of error (eyewitness misidentification, false confessions, perjured testimony, forensic error, tunnel vision, prosecutorial misconduct, ineffective assistance of counsel, etc.) are contributing sources, not exclusive causes, of wrongful convictions."
A 1998 law review article looks at a small sample of convictions allegedly obtained with false confessions and also examines the tricky question of what "innocence" and a "wrongful conviction" really mean. For example, suppose that someone falsely confesses to facts that would make him guilty of first degree murder when in fact he was actually an accessory to manslaughter. He isn't truly "innocent" of any crime related to the death in question, but is likely to be convicted of a crime much more serious than the crime of which he is actually guilty.
A 2006 law review article examining a single case in detail in the context of broader research finings comes to similar conclusions about high risk fact patterns, but throws in another - clusters of mass wrongful convictions driven by one or many deeply corrupt police officers in a single department who routinely lie and falsify evidence to obtain large numbers of wrongful convictions.
A Wall Street Journal blog post from July 16, 2017 cites a Journal of Empirical Legal Research article in support of the proposition that juries get about one in six criminal cases wrong and that judges do only slightly better.
Some studies have looked at indications that judges are not better than juries at accurately reaching correct outcomes that don't look at the actual outcomes themselves:
In study after study, researchers have found more similarities than differences (see Robbennolt, 2006; Rowland, et al., 2010). For example:
Retired judges were no better than college sophomores, teachers and other novices at spotting witness mendacity.
Retired judges were no better than mock jurors at ignoring inadmissible evidence.
Retired judges evince no greater understanding of the scientific principles underlying Daubert standards than do mock jurors.
Moreover, social science research makes it very clear that many decisions by trial judges, federal and state, are affected by extra-legal political influences. For example:
The decisions of Federal district judges, usually thought to be better trained and less susceptible to extra-legal influences than are their state counterparts, can for many case types (e.g., abortion) be predicted at a high rate of accuracy simply by knowing the judge’s appointing president.
Elected state judges’ sentencing decisions change in tandem with the proximity of elections
A comprehensive review of prior studies in a 2005 article reiterates almost all of the conclusions from various kinds of data reached above - i.e. juries are more lenient but their skills at assessing facts are similar to those of judges.
Arguments For And Against The Jury System Unrelated To Their Accuracy
A 2011 law review article reiterates that judges are far more likely to convict that juries, on average, which it argues is good even if it comes at the expense of lower accuracy. This article does not squarely address the real money question of whether judges or juries are more accurate in their decisions to convict or acquit defendants. The author argues that certainty of punishment is more important to deterring bad conduct than sentence length or the accuracy of the process.
A 1981 article by John Langbein, one of the leading scholars in the field of comparing the European legal system (where serious criminal trials often involve "mixed courts" with a judge and lay judges serving together in a capacity that isn't quite a true American style jury) to the American legal system concluded that:
The conclusion is that the mixed court serves the jury policies well, though not fully; and that it is a superior alternative to the indigenous nontrial devices-plea bargaining and bench trial-that have displaced the jury from routine American practice.
It is also worth noting that one of the important justifications for using jury trials rather than bench trials or mixed trials, is that it enhances the institutional legitimacy of the judiciary, regardless of its accuracy, in a system where both triers of fact sometimes make mistakes.
This is because in a jury trial system, an outcome that is perceived as incorrect can be blamed on the diffuse and effectively anonymous jury who cannot be held accountable, deflecting blame from judges who could be held accountable or whose bad decisions would reflect poorly on the judiciary, or the judge in question at least. It is easier to support the legitimacy of the judicial system if judges are rarely blamed for reaching the hard verdict in high stakes cases.
Also, since juries deliver only a verdict, and not a detailed justification for their ruling as a judge in a bench trial must, the jury verdict, while less transparent, does allow people who might potentially agree with the result even if they wouldn't have agreed with the means by which that particular jury actually reached its decision, to give the jury the benefit of the doubt, again increasing popular acceptance of the result.
