4  Interpretations of probability

🥅 Goals
  1. Understand that there are different ways to interpret probability values.

This chapter covers some different ways in which we can interpret probabilities in the real world. The axioms in Chapter 1 are helpful to determine a framework for mathematical probability, but they leave us lots of room to choose a model within that framework.

We have already discussed one approach in Chapter 2: the “classical” approach, in which each outcome in the sample space is assigned the same probability. This interpretation has some obvious limitations in practice. Often we cannot find a set of outcomes that it is reasonable to think of as a priori equally likely. Therefore, it is essential to have more widely applicable models to deal with uncertainty.

In this chapter, we discuss two more approaches to determining probabilities in real-world applications: the relative frequencies approach and the subjective probability (or betting) approach. Either can be used, depending on the context, to help us to assign probabilities to events.

The goal of this chapter is to help you to develop more intuition and probabilistic thinking. For the rest of the course, we will assume that we “know” the probability of each event, without worrying too much about how it was determined.

4.1 Relative frequency interpretation

This interpretation applies to trials giving chance outcomes of an experiment that can be repeated indefinitely under essentially unchanged conditions and which exhibits long term regularity.

Suppose that we run \(n\) trials of an experiment with a known list of possible outcomes and the number of trials on which event \(A\) occurs is \(n_A\) (\(A\) is again a set of possible outcomes). The relative frequency of occurrence of \(A\) is \(n_A/n\).

For example, if we toss a coin 1000 times and observe 490 heads, then the relative frequency of heads is 490/1000.

For some experiments, it may be reasonable to suppose that relative frequencies are stable for very large \(n\).

If we toss a fair coin one billion times, we might expect that the relative frequency of heads after the first few thousand throws would remain very close to 1/2.

As a mathematical idealization, we suppose that there is a unique, empirical limiting value for \(n_A/n\), as \(n\) tends to infinity, which we call the relative frequency probability of \(A\).

For our coin, the statement \(\pr{\text{heads}} = 1/2\) means ‘if we tossed the coin an extremely large number of times, then the proportion of heads would be arbitrarily close to 1/2’.

This interpretation is widely used, especially in physics, where experiments are designed for repeatability and we can expect future trials to behave like those in the past. In this view, probability is a property of the experimental setup and may be “objectively’’ discovered by sufficient repetitions of the experiment.

Amongst the problems with this interpretation are:

  • it is often impossible to decide what “essentially unchanged conditions’’ are;
  • we often have no way of knowing when limiting frequencies become stable (how many trials should we do to test this?);
  • we can only use it in situations that are repeatable.

4.2 Betting interpretation

A very different way of interpreting probability goes by considering probability as a quantification of someone’s (yours, mine, your neighbour, ) belief that an event will occur. There are various different ways in which we can measure this belief numerically. Here is one of the simplest.

Your subjective probability that \(A\) will occur is measured by the amount \(p_A\) that you would consider to be a fair price for the following gamble:

  • if \(A\) occurs, you receive 1;
  • if \(A\) does not occur, you receive nothing.

In this interpretation, there are no “true’’ probabilities. Different individuals will have different information relevant to a problem and so may validly make different probability assessments.

For instance, if you say your probability that ‘Your Team’ wins its next match is 1/2 this means that you view 1/2 as a fair price for the gamble winning you 1 if Your Team wins but otherwise nothing. Others may disagree with you.

Subjective probability ideas are often used by decision makers who have to consider problems concerning unique, non-repeatable events, based on their informed but subjective judgements. The advantages of this interpretation are that probability measures the belief of a subject, and is no longer seen as a property of the experimental setup. Potential issues are that the highest ‘buying price’ may differ from lowest ‘selling price’; a subject may have reason to misrepresent their fair price; placing the bet itself might affect the experiment.

4.3 Interpretation and the axioms

We claimed that the axioms of probability are the same regardless of the interpretation of the probabilities that we are using. A1 and A2 are clearly very sensible in any interpretation. The justification of A3 (and, by extension, A4) needs some more thought.

A3 feels intuitive for the classical model of probability by its relation to counting: in the classical model if \(A\) contains \(m_A\) outcomes and \(B\) contains \(m_B\) outcomes, with none in common with \(A\), then \(m_{A\cup B} = m_A + m_B\). The argument is very similar for the relative frequency model and only slightly more subtle for the betting model.

📖 Textbook references

For more information on these ideas, check out: