6
  1. I realise that every scenario is unique, but are there are guiding principles forensics investigators use to estimate the prior probability of guilt of a person (P(G)) when using Bayes' Theorem to estimate the probability of guilt after taking evidence into account?

  2. Are there any widely accepted resources that list the hit rate (P(E|G)) and false alarm rate (P(E|I)) for different types of evidence?

4
  • 1
    One question per question.
    – Trish
    Feb 8, 2022 at 18:20
  • 5
    I think multi-part questions are fine when they're related like this.
    – bdb484
    Feb 8, 2022 at 20:28
  • 3
    I believe there is some weirdness were many legal systems will allow a frequentist statement to the effect that "It can be shown that given 100 cases of a murdered woman with a history of being abused by her partner, in 99 cases the partner was the murderer." but the equivalent bayesien statement; "It can be shown that in the case of a murdered woman with a history of being abused by her partner, there is a 99% probability that the partner was the murderer" is forbidden. blog.richmond.edu/physicsbunn/2013/02/28/… the reasoning behind that is beyond me.
    – Clumsy cat
    Feb 9, 2022 at 13:37
  • 1
    @Clumsycat: Theoretically the statements are equivalent, but I can well imagine that the probabilist version is more liable to misunderstanding by non-experts. If jury members may hear “in X type of case, there is a 99% chance the partner is guilty”, and reason “this is X type of case, so this expert is telling us there’s a 99% chance of guilt”. The frequentist phrasing prompts people to think about a variety of cases, and so gives more like the correct Bayesian intuition: you must combine the general statistical information with your other knowledge of this specific case. Feb 9, 2022 at 23:59

3 Answers 3

9

First of all, Bayesian reasoning, as other answers have noted, is generally inadmissible in U.S. courts as evidence at a jury trial or bench trial on the merits of a criminal case, although it may be admissible in a pretrial hearing before a judge regarding the admissibility at trial of certain kinds of expert testimony using certain forensic methods.

Investigators who deal with forensic evidence are generally not in the business of predicting the probability of guilt of a particular suspect. Typically, they do a lot of their work before a suspect is identified at all.

  1. Are there any widely accepted resources that list the hit rate (P(E|G)) and false alarm rate (P(E|I)) for different types of evidence?

There is a literature on this. Groups like the Innocence Project, have developed profiles of types of evidence that are particularly unreliable, and use this to identify which claims of innocence to investigate further. One of their top line charts setting forth these causes (in addition to government misconduct and bad lawyering which are not evidentiary in nature) is as follows:

enter image description here

Right up at the top of the list are eye witness identifications of suspects who are strangers (especially interracial eye witness identifications), which are probably the single greatest cause of wrongful convictions, to the point that some courts will allow expert testimony from a frequentist perspective on the unreliability of this kind of evidence at trial.

Likewise, there is an academic literature (mostly by criminologists, rather than in legal scholarship) on the probability of a forensic sample of DNA or a fingerprint being accurate that is often admissible with expert testimony at trial (overwhelmingly, these methods are quite accurate, although there are circumstances when this is not the case, such as very fragmentary samples when there are multiple suspects who are closely related to each other, or when there are issues with how the DNA evidence was collected and what it really shows). A report below finds that false positive rate for fingerprints to be as follows:

enter image description here

There is also a small cottage industry is making claims that certain kinds of forensic evidence is reliable when, in fact, it is not, and another small cottage industry of refuting the scientific validity of such evidence. This is typically done prior to trial in an evidentiary hearing regarding the admissibility of expert testimony at trial on the grounds that the scientific method of the witness is reliable. In U.S. federal courts, this is called a Daubert hearing.

For example, gunshot analysis, footprint analysis, hair and bite mark comparison are unreliable in accurately identifying criminals. This is set forth in a 2016 report (“the Report”) to the US President titled, Forensic Science in Criminal Courts: Ensuring Scientific Validity of Feature-Comparison Methods. The Report, written by the US President’s Science and Technology advisors (PCAST), concludes that DNA analysis is the only forensic technique that is absolutely reliable.

Outside the U.S., both Canada and the U.K. have both considered their own approaches to keeping "junk science" out of courtrooms, but I am not very familiar with them and don't address them at length in this answer.

An abstract analysis also has strong limitations.

For example, while footprint analysis is not very useful in general in identifying a particular suspect, if other evidence such as DNA evidence, fingerprint analysis, and a lack of alibis puts two possible suspects at the scene and they both have motive and opportunity, one of whom was a woman who wears a size 2 shoe, and the other of which is a man who wears a size 14 shoe, footprints could have highly relevant and probative evidence at ruling out one suspect or other other (or maybe even both, if the footprint is neither tiny nor huge).

2
  • "Investigators who deal with forensic evidence are generally not in the business of predicting the probability of guilt of a particular suspect." Someone needed to remind the writers of shows like "CSI" and "Quincy" about this.
    – Barmar
    Feb 9, 2022 at 14:37
  • 1
    @Barmar The sliver of cases that get CSI/Quincy type treatment is very thin. You are basically looking at a modest percentage of murders where likely suspects are identified in the first 24-48 hours and an even smaller percentage of violent stranger rape cases, and then, only in big cities that have the resources to do it. (And maybe one burglary every couple years when the police chief or Mayor is hit.) Those are a tiny percentage of all serious crimes. Also, the success rates in clearing those cases with everything science can throw at it at all is pretty low.
    – ohwilleke
    Feb 9, 2022 at 16:39
11

You have to deconstruct the question on a country by country basis. In the US legal system, the question presupposes something incorrect, that there are "investigators" who play a rule in legal system, and their job is to determine guilt vs. innocence. That is something determined by the "finder of fact", typically the jury (it could be a judge, when you have a bench trial). The finder of fact may take into consideration testimony provided by a witness, and it is possible that a witness is an expert in some relevant area, who might attest to the significance of a claimed piece of evidence. The expert cannot testify on the question of guilt.

For example, a DNA sample might be admitted as evidence that the defendant was present in certain circumstances, which could reasonably lead the finders of fact to conclude that the defendant did strike the victim. A DNA sample cannot testify, therefore some expert must testify about the significance of the sample, specifically whether the sample "might have" come from the defendant, or "definitely came from" the defendant. Others will testify as to the circumstances surrounding the collection of the sample. It is then up to the finder of fact to evaluate all of the evidence plus understand the instructions regarding the criteria for finding the defendant "guilty".

Theoretically, Bayes Law could enter into an expert witness's (scientific) testimony, and likewise an opposing witness could contradict the putative relevance of BL in terms of reaching a scientific conclusion. BL could enter into the discussion at the level of the bottom line "definitely/possibly does come from the defendent". The only practical way that I can see BL entering into the courtroom at least non-gratuitously would be if there is a dispute over the reliability of a certain test, thus should the testimony be allowed in the first place (it might be excluded as being unreliable). There is a genre of research (for example this) that addresses the problem of sketchy statistic inferences, pointing to a role for BL. I would say that the prospects are dim, not bright.

0
6

using Bayes' Theorem to estimate the probability of guilt after taking evidence into account?

In jurisdictions of the US this approach would contravene the rules of evidence, particularly the inadmissibility of prior act evidence.

For instance, Michigan Rule of Evidence 404(b)(1) prohibits the use of "[e]vidence of other crimes, wrongs, or acts [...] to prove the character of a person in order to show action in conformity therewith". The application of Bayesian techniques would be even worse in that the matter would involve the use of data about individuals who are totally unrelated to the defendant and to the charges against him.

MRE 406 admits "[e]vidence of the habit of a person", but (1) that is for proving "that the conduct of the person [...] on a particular occasion was in conformity with the habit" rather than to prove guiltiness, and (2) it still refers to the same person rather than others. Being realistic, Bayesian techniques will involve data about others.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .