Elsevier

Science & Justice

Volume 56, Issue 5, September 2016, Pages 392-396
Science & Justice

Position paper
Reframing the debate: A question of probability, not of likelihood ratio

https://doi.org/10.1016/j.scijus.2016.05.008Get rights and content

Abstract

Evidential value is measured by a likelihood ratio. This ratio has two components, the probability, or probability density, of the evidence if the prosecution proposition is true and the probability (density) of the evidence if the defence proposition is true. It takes the form of a single value, even if these probabilities are subjective measures of belief of the reporting forensic scientist.

Introduction

The title of this special issue of Science & Justice raises the topic of precision in relation to likelihood ratios. The precision of the likelihood ratio is discussed in the context of its two components, the probability of the evidence if the prosecution proposition were true and the probability of the evidence if the defence proposition were true, with particular reference to probability as a subjective measure of belief. This interpretation of probability [8], [16] provides clear answers to questions about the nature and the properties of probability assertions. Most importantly, this interpretation is not imprecise, nor does it have the difficulties of interpretation that are associated with other forms of inference such as ones based on long-term repetitions of experiments under identical conditions. We also note that the laws of probability are precise and should not be confused with other measures of uncertainty that lack an ability to express uncertainty numerically. As a further starting point we emphasise that there is no way of validating an expert's single probability assessment [17], beyond agreement with the rules of probability.

This paper is structured as follows. In 2 Which or whose likelihood ratio?, 3 Not ‘the’ probability, but the scientist's probability, we emphasise that both likelihood ratios and probability assignments relate to the scientists' personal viewpoints. Section 4 clarifies the conceptual consequence of an inability to determine a single numerical value for a probability, an inability that can arise from a procedure known as incomplete probability elicitation. This section also recalls the distinction between the normative and descriptive view on probability, with an emphasis on the former, whereas Section 5 discusses the difference between precision and resolution. 6 Your probability is about your uncertainty, but you are not uncertain about your probability, 7 Analysing reality, not merely describing it emphasise, respectively, the completeness of probability as a description of a given state of knowledge, at a given time, and the natural process of updating one's probability upon receipt of new data. Conclusions are presented in Section 8.

Section snippets

Which or whose likelihood ratio?

Advocates of the process in which there is imprecision in the measure of uncertainty argue that there are forensic systems that output likelihood ratios with differing values. It is argued that, as there is variation in the training data from which the relevant probabilistic models are derived, this should be represented in the value, or values, reported in court. This may well be the case in areas involving highly technical traces (e.g., audio recordings), involving subtle measuring

Not ‘the’ probability, but the scientist's probability

In the absence of data, many scientists struggle with probability assignment and retreat to statements such as ‘I do not know what the probability is’. Such a statement illustrates a lack of understanding of probability because probability is not something that can be known or not known.2

Incomplete probability elicitation: being clear what it means and what it does not mean

The above comments are disputed by those scientists who contend that they cannot state their probability or, for whatever reason, refrain from assuming the responsibility of assigning probabilities in a given case. The difficulty experienced in practise of expressing uncertainty in terms of a single probability value is sometimes turned into a critique of the concept, but this is unsound: any difficulty in comprehension of the concept does not make the concept wrong or deficient in any way. We

A question of resolution, not precision

In the evaluation of evidence there is no concept of a long-term repetition of an experiment for the measurement of a characteristic from which an estimate of the natural variation, and hence precision, of the measurement may be obtained. There may be discussion about whether it is feasible and desirable to determine if a single probability should be given to, for example, four significant figures and hence make a distinction between 0.0101 and 0.0102. This is a question of the level of

Your probability is about your uncertainty, but you are not uncertain about your probability

When a scientist assigns their probability for a proposition about which they are uncertain, it is not necessary to assign a precision to the probability thus provided. A probability has been assigned as a measure of uncertainty in the belief of a proposition. There is no need to assign a measure of uncertainty to the measure of uncertainty.

At times, assigning one's probability may be hard in the sense that one may hesitate to accept or stick to a particular value. The reasons for this may be

Analysing reality, not merely describing it

‘What is my probability for an unknown person (or object) drawn from this population to have the analytical feature F?’ Such a question8 is commonly answered on the basis of the proportion θ of the population that has feature F, that is Pr(F | θ). To learn about the population proportion θ, scientists may investigate measurements x = {x1,  , xm} on a sample of m individuals

Conclusions: what should forensic scientists do (next)?

The expression of the probative strength of forensic results in terms of likelihood ratios has never been discussed as widely in forensic science in the past as now. The development has perhaps taken place too rapidly and some fundamental topics need further examination. Indeed, experience demonstrates that while many contributors have no difficulty in explaining that a likelihood ratio involves two conditional probabilities,9

Acknowledgements

The authors acknowledge the support of the Swiss National Science Foundation through grant No. BSSGI0_155809 and the University of Lausanne.

References (25)

  • A.P. Dempster

    Upper and lower probabilities induced by a multivalued mapping

    Ann. Math. Stat.

    (1967)
  • ENFSI

    ENFSI Guideline for Evaluative Reporting in Forensic Science

  • Cited by (31)

    • Bayesian multivariate models for case assessment in dynamic signature cases

      2021, Forensic Science International
      Citation Excerpt :

      Researchers have realized that in addition to data collection, there is an urgent need for a framework for evidence evaluation. Probabilistic procedures [27–41] for evidence evaluation have been suggested as solutions. These data-driven methods provide the means for thorough validation, as well as implement a practical model to assess the value of observations made by the expert in the legal context.

    • Are low LRs reliable?

      2020, Forensic Science International: Genetics
    • Formal description of the probabilistic models

      2020, Forensic Practitioner's Guide to the Interpretation of Complex DNA Profiles
    View all citing articles on Scopus

    This paper is part of the Virtual Special Issue entitled: Measuring and Reporting the Precision of Forensic Likelihood Ratios, [http://www.sciencedirect.com/science/journal/13550306/vsi], Guest Edited by G. S. Morrison.

    View full text