8.1 Probability Rules

In theoretical probability, we need to define a few terms and set some rules (known as axioms).

The sample space, \(S\), is the set of all possible outcomes of a random process.

  • Example: If you flip two coins (one side Heads and one side Tails), then the sample space contains four possible outcomes: Heads and Heads (HH), Heads and Tails (HT), Tails and Heads (TH), and Tails and Tails (TT), \(S = \{HH,HT,TH,TT\}\).

A subset of outcomes is called an event, \(A\).

  • Example: If you flip two coins, an event \(A\) could be that exactly one of the coins land Heads, \(A = \{HT,TH\}\).

For the rules of probability, we can define them with set notation as well as words. If you aren’t familiar with set notation,

  • \(\cup\) means union (inclusive OR)
  • \(\cap\) means intersection (AND)
  • \(A^C\) means complement (NOT)

For events \(A\) and \(B\) and sample space \(S\), the probability of an event \(A\), notated as \(P(A)\), follows the rules below:

  • Rule 1: \(0\leq P(A)\leq 1\) (probability has to be between 0 and 1)
  • Rule 2: \(P(S) = 1\) (one of the outcomes has to happen)
  • Rule 3: \(P(A^c) = P(\text{not }A) = 1 - P(A)\) (if we know the chance of something happening, we also know that chance it doesn’t happen)
  • Rule 4: \(P(A\cup B) = P(A\text{ or }B) = P(A) + P(B)\) if \(A\) and \(B\) are disjoint events.
    • \(A\) and \(B\) are disjoint/mutually exclusive if \(A\) occuring prevents \(B\) from occurring (they both can’t happen at the same time).
  • Rule 4*: \(P(A\text{ or }B) = P(A\cup B) = P(A) + P(B) - P(A\cap B)\)
  • Rule 5: \(P(A\cap B) = P(A\text{ and }B) = P(A)\times P(B)\) if \(A\) and \(B\) are independent.
    • \(A\) and \(B\) are independent if \(B\) occurring doesn’t change the probability of \(A\) occurring.
  • Rule 5*: \(P(A\text{ and }B) = P(A\cap B) = P(A~|~B)P(B) = P(B~|~A)P(A)\).
    • The conditional probability of A given that event B occurs, \(P(A~|~B)\), is equal to the probability of the joint event (A and B) divided by the probability of B. \[P(A ~| ~B) = \frac{P(A \text{ and } B)}{P(B)} = \frac{P(A \cap B)}{P(B)}\]
    • Intuition: Given that \(B\) happened, we focus on the subset of outcomes in \(S\) in which \(B\) occurs and then figure out what the chance of \(A\) happening within that subset.

8.1.0.1 Example: Blood Types

The American Red Cross estimates that 45% of U.S. population has Type O blood, 40% are Type A, 11% Type B, and 4% AB blood.

Imagine that we have a blood drive in St. Paul. The next donor’s blood type can be thought of as a random process. The sample space for this random process includes the 4 blood types: \(S= \{O,A,B,AB\}\) (it includes all possible outcomes). Assume the people who donate blood have the same distribution of blood types as the U.S. and that St. Paul has the same distribution as the entire U.S.

Think about how you’d justify your answer to the following questions:

  1. What is the probability that the next donor is Type O blood?

  2. What is the probability that the next donor is Type A or Type B or Type AB blood?

  3. What is the probability the next three donors are all Type O blood?

  4. What is the probability the next donor is Type O or Type A or Type B or Type AB?

If the possible outcomes were equally likely, we could calculate probabilities \[P(A) = \frac{\text{Number of outcomes in }A}{\text{Number of possible outcomes}}\]

But the chances of Type O, A, B, and AB blood are all different because they occur with different frequency in the population.

Let’s change the sample space to make it easier. Let our sample space, \(S\), be the set of 100 equally likely outcomes (45 are O, 40 are A, 11 are B, and 4 are AB). Now, you can calculate probabilities based on this framework of equally likely outcomes (after we changed the sample space).

  1. P(Type O) = 45/100 assuming equally likely outcomes

  2. P(Type A or B or AB) = 1 - P(Type O) = 1 - 45/100 by Rule 3

  3. P(Type O and then Type O and then Type O) = \((45/100)^3\) by Rule 5 assuming donors are independent, in that the probability of Type O blood stays the same

  4. P(Type A or B or AB or O) = P(S) = 1 by Rule 2

8.1.0.2 Example: 52 Cards

Let’s consider a perfectly shuffled deck of playing cards. Each card has an icon and a number (or A, J, Q, K) on it. The icon is either a red heart, red diamond, black spade (leaf), or black club (3 leaf clover). The numbers range for 2 to 10 and A is for Ace, J is for Jack, Q is for Queen, K is for King.

The sample space is below.

  1. What is the probability of drawing a card with a heart icon on it?

  2. What is the probability of drawing a card with a heart or Ace (A) on it?

  3. What is the probability of dealing a card with a heart on the table and then another heart card?

Focus on how we’d justify the answer, not just the number.

  1. P(heart) = 13/52 by equally likely outcomes

  2. P(heart or ace) = 13/52 + 4/52 - 1/52 = 16/52 by Rule 4*

  3. P(heart and then heart) = 13/52*12/51 by Rule 5* (draws are not independent here since the probability of hearts changes after you remove a card)

8.1.1 Disjoint/Mutually Exclusive

Think back to the Blood Type example.

Let’s say we were interested in the next two donors.

  • P(First Type A or Second Type A) = ?

Think about all the ways this could happen.

We will always use an inclusive OR, which means that we care about one or the other or both happening. We just need to make sure we don’t double count, which is why we subtract the chance of both.

  • P(First Type A or Second Type A) = P(First Type A) + P(Second Type A) - P(both Type A)

So,

  • P(First Type A or Second Type A) = 0.40 + 0.40 - 0.40*0.40 = 0.64

Which is the same as if we were to consider the three disjoint options (A: Type A, N: Not Type A),

  • P(AN or NA or AA) = 0.4*0.6 + 0.6*0.4 + 0.4*0.4 = 0.64

8.1.2 Independence

Let’s stay with the Blood Type example for one moment more.

What if there were only 50 donors in St. Paul? Say 30 of them Type 0 and the other 20 were A or B.

  • Would the second donor be independent of the first donor? In other words, would the probability of getting a Type O donors change between donors?

No, they wouldn’t be independent. In that case, let’s calculate the probability that the first two donors are Type O.

  • P(Type O and then Type O) = P(Type O)P(2nd Type O | 1st Type O) = (30/50) * (29/49) = 0.355