Is there "evidence-based" practice in law?

Question

"Evidence-based practice" (sometimes colloquially called "what works") is a buzzword floating around many professions nowadays, especially medicine and education. The idea is to use scientific research to inform and continuously improve day-to-day professional practice.

For example, a schoolteacher following non evidence-based practices might teach square roots the same way he has for decades and the same way he was taught them as long as nothing obviously is going wrong (e.g. a flood of parental complaints, excessive number of students flunking out, etc.). By contrast, an evidence-based teacher would keep tabs on the latest scientific journal publications in mathematics pedagogy and alter his lectures, homework assignments, tests, etc. to conform to what was shown to work best in such studies. He may even do his own studies that he then publishes. In theory, this is supposed to create a virtuous cycle of continuous improvement where everyone is sharing their research and optimizing each others' practices.

Research in evidence-based practice tends to be quantitative, for example, "Doing this instead of that in the classroom resulted in a 20% gain on standardized literacy test scores, a 10% rise in cumulative GPA, and 50% fewer behavioral referrals for student misconduct."

Does evidence-based practice exist in the practice of law? If so, what does it consist of? Is it quantitative or qualitative?

I have read legal research journals, and have found most publications to be deep-dive studies of this or that specific legal issue or area (e.g. intestate probate in California when the decedent was legally incompetent at the time of death and had a living spouse, not resident in California, who refuses to attend the probate hearing) rather than quantitative studies on whether doing X or Y will get me more wins at trial, get my clients more favorable settlements, or give me an edge getting "questionable" evidence not ruled inadmissible. The difference could be compared to a physician reading an essay on "What is non-XYZ skin cancer?" versus reading a study showing that increasing chemotherapy doses by 5% in patients with non-XYZ melanoma resulted in 10% fewer deaths among skin cancer patients at X hospital with caveats A and B and confidence interval C.

In a nutshell:

Non-evidence-based:

• Deep dive into the history of the Chewbacca Defense in criminal law from its origins in the mid-1300's to the present day, with specific notes to its successful use in a landmark Mopery case in Podunk Superior Court in 1955 and an Aggravated Creeping with Dishonest Intent case in the Supreme Court of Ruritania in 1987.

Evidence-based:

• In a sample of Mopery cases in Podunk courts (N=235) from 1973 to 1982, defendants raising the Chewbacca Defense obtained 30% more acquittals than those using the Twinkie Defense. Among the small number of defendants who were convicted after raising the Chewbacca Defense, they, on average, received 20% lighter sentences than those who had lost after raising the Twinkie Defense. Conclusion: the Chewbacca Defense is more effective against Mopery charges than the Twinkie Defense with caveats A and B and confidence C. See Tables A and B for supporting data.

Answer 1

The primary field of research where the kind of studies you are thinking about are done is often called "empirical legal research."

There is a fairly rich literature, that a substantial minority of practitioners in the relevant areas are guided by, in the area of issues related to jury trials: how jury composition influences outcomes, how juries make decisions, the difference in outcomes between jury trials and bench trials, the fact patterns that are prone to wrongful convictions or mistakes about the facts in civil cases, techniques that influence jury decision making outcomes, and the kinds of awards juries tend to make of non-economic damages in particular kinds of cases.

Another area where there is a fairly rich literature that a smaller minority of practitioners reference concerns the facts that influence likelihood of success on appeal, which in turn influences the decision to appeal or not appeal a case, or to bring collateral attacks on a trial court decision (e.g. in habeas corpus review of a criminal conviction).

A third area where empirical studies guide a minority of practitioners is the more general literature on negotiation which is used to influence how both transaction negotiations and settlement discussion negotiations, are conducted.

In most other areas, empirical literature is used mostly to evaluate risk, either to determine its materiality in connection with worrying about it in planning a transaction, or in the early stages of a lawsuit. In these areas this is incorporated holistically into the general career experience and life experience of the attorneys involved.

For example, there is an empirical literature that demonstrates that small business bankruptcies rarely successfully reorganize a business and usually result in tax creditors receiving most of the available assets, while big business bankruptcies usually do successfully reorganize (with some industries more successful than others) and tend to pay different kinds of creditors different proportions of their claims. This literature influences the decisions of individual creditors regarding what kind of cases to pursue more or less aggressively, and also influences how lawyers familiar with this literature advise clients about the risks involved in particular investments. Likewise, there is a literature on the likelihood of small business success over particular time frames with additional other factors considered (e.g. franchise v. non-franchise).