Position paperReframing the debate: A question of probability, not of likelihood ratio☆
Introduction
The title of this special issue of Science & Justice raises the topic of precision in relation to likelihood ratios. The precision of the likelihood ratio is discussed in the context of its two components, the probability of the evidence if the prosecution proposition were true and the probability of the evidence if the defence proposition were true, with particular reference to probability as a subjective measure of belief. This interpretation of probability [8], [16] provides clear answers to questions about the nature and the properties of probability assertions. Most importantly, this interpretation is not imprecise, nor does it have the difficulties of interpretation that are associated with other forms of inference such as ones based on long-term repetitions of experiments under identical conditions. We also note that the laws of probability are precise and should not be confused with other measures of uncertainty that lack an ability to express uncertainty numerically. As a further starting point we emphasise that there is no way of validating an expert's single probability assessment [17], beyond agreement with the rules of probability.
This paper is structured as follows. In 2 Which or whose likelihood ratio?, 3 Not ‘the’ probability, but the scientist's probability, we emphasise that both likelihood ratios and probability assignments relate to the scientists' personal viewpoints. Section 4 clarifies the conceptual consequence of an inability to determine a single numerical value for a probability, an inability that can arise from a procedure known as incomplete probability elicitation. This section also recalls the distinction between the normative and descriptive view on probability, with an emphasis on the former, whereas Section 5 discusses the difference between precision and resolution. 6 Your probability is about your uncertainty, but you are not uncertain about your probability, 7 Analysing reality, not merely describing it emphasise, respectively, the completeness of probability as a description of a given state of knowledge, at a given time, and the natural process of updating one's probability upon receipt of new data. Conclusions are presented in Section 8.
Section snippets
Which or whose likelihood ratio?
Advocates of the process in which there is imprecision in the measure of uncertainty argue that there are forensic systems that output likelihood ratios with differing values. It is argued that, as there is variation in the training data from which the relevant probabilistic models are derived, this should be represented in the value, or values, reported in court. This may well be the case in areas involving highly technical traces (e.g., audio recordings), involving subtle measuring
Not ‘the’ probability, but the scientist's probability
In the absence of data, many scientists struggle with probability assignment and retreat to statements such as ‘I do not know what the probability is’. Such a statement illustrates a lack of understanding of probability because probability is not something that can be known or not known.2
Incomplete probability elicitation: being clear what it means and what it does not mean
The above comments are disputed by those scientists who contend that they cannot state their probability or, for whatever reason, refrain from assuming the responsibility of assigning probabilities in a given case. The difficulty experienced in practise of expressing uncertainty in terms of a single probability value is sometimes turned into a critique of the concept, but this is unsound: any difficulty in comprehension of the concept does not make the concept wrong or deficient in any way. We
A question of resolution, not precision
In the evaluation of evidence there is no concept of a long-term repetition of an experiment for the measurement of a characteristic from which an estimate of the natural variation, and hence precision, of the measurement may be obtained. There may be discussion about whether it is feasible and desirable to determine if a single probability should be given to, for example, four significant figures and hence make a distinction between 0.0101 and 0.0102. This is a question of the level of
Your probability is about your uncertainty, but you are not uncertain about your probability
When a scientist assigns their probability for a proposition about which they are uncertain, it is not necessary to assign a precision to the probability thus provided. A probability has been assigned as a measure of uncertainty in the belief of a proposition. There is no need to assign a measure of uncertainty to the measure of uncertainty.
At times, assigning one's probability may be hard in the sense that one may hesitate to accept or stick to a particular value. The reasons for this may be
Analysing reality, not merely describing it
‘What is my probability for an unknown person (or object) drawn from this population to have the analytical feature F?’ Such a question8 is commonly answered on the basis of the proportion θ of the population that has feature F, that is Pr(F | θ). To learn about the population proportion θ, scientists may investigate measurements x = {x1, … , xm} on a sample of m individuals
Conclusions: what should forensic scientists do (next)?
The expression of the probative strength of forensic results in terms of likelihood ratios has never been discussed as widely in forensic science in the past as now. The development has perhaps taken place too rapidly and some fundamental topics need further examination. Indeed, experience demonstrates that while many contributors have no difficulty in explaining that a likelihood ratio involves two conditional probabilities,9
Acknowledgements
The authors acknowledge the support of the Swiss National Science Foundation through grant No. BSSGI0_155809 and the University of Lausanne.
References (25)
- et al.
Assessing transfer probabilities in a Bayesian interpretation of forensic glass evidence
Sci. Justice
(1998) Evaluation and professionalism
Sci. Justice
(2009)- et al.
De Finetti's subjectivism, the assessment of probabilities and the evaluation of evidence: a commentary for forensic scientists
Sci. Justice
(2001) - et al.
Fundamentals of probability and statistical evidence in criminal proceedings (Practitioner Guide No. 1)
Thinking and Deciding
(2008)- et al.
Bayesian Theory
(2000) The role of the subjectivist position in the probabilization of forensic science
J. Forensic Sci. Med.
(2015)- et al.
Liberties and constraints of the normative approach to evaluation and decision in forensic science: a discussion towards overcoming some common misconceptions
Law Probab. Risk
(2014) The Logic of Modern Physics
(1927)Theory of Probability, a Critical Introductory Treatment
(1974)
Upper and lower probabilities induced by a multivalued mapping
Ann. Math. Stat.
ENFSI Guideline for Evaluative Reporting in Forensic Science
Cited by (31)
A practical treatment of sensitivity analyses in activity level evaluations
2024, Forensic Science InternationalBayesian multivariate models for case assessment in dynamic signature cases
2021, Forensic Science InternationalCitation Excerpt :Researchers have realized that in addition to data collection, there is an urgent need for a framework for evidence evaluation. Probabilistic procedures [27–41] for evidence evaluation have been suggested as solutions. These data-driven methods provide the means for thorough validation, as well as implement a practical model to assess the value of observations made by the expert in the legal context.
Are low LRs reliable?
2020, Forensic Science International: GeneticsHow can courts take into account the uncertainty in a likelihood ratio?
2020, Forensic Science International: GeneticsProbabilistic reporting in criminal cases in the United States: A baseline study
2020, Science and JusticeFormal description of the probabilistic models
2020, Forensic Practitioner's Guide to the Interpretation of Complex DNA Profiles
- ☆
This paper is part of the Virtual Special Issue entitled: Measuring and Reporting the Precision of Forensic Likelihood Ratios, [http://www.sciencedirect.com/science/journal/13550306/vsi], Guest Edited by G. S. Morrison.