Chapter 1: Foundations of Probability
This chapter will help you understand how probabilities are calculated using rules and logic grounded in the structure of all possible outcomes. We'll go from basics like outcome spaces to the famous Bayes' Rule.
1.1 The Outcome Space:
Every probability problem starts with the outcome space, also known as the sample space. This is the complete list of all possible outcomes of a random experiment.
Example 1: Rolling a Die
If you roll a standard six-sided die, the possible outcomes are:
\[ \Omega = \{1, 2, 3, 4, 5, 6\} \]
This outcome space tells us everything that could happen in this experiment.
Example 2: Tossing a Fair Coin Twice
Let’s say we flip a fair coin two times. The outcomes we could get are:
- Heads then Heads (HH)
- Heads then Tails (HT)
- Tails then Heads (TH)
- Tails then Tails (TT)
So the outcome space is:
\[ \Omega = \{HH, HT, TH, TT\} \]
This structured way of listing outcomes helps us calculate probabilities. Suppose we want the probability of getting exactly one head and one tail. That happens in two of the four outcomes: HT and TH. So:
\[ P(\text{1 head and 1 tail}) = \frac{2}{4} = \frac{1}{2} \]
The outcome space is the foundation for all of probability. It sets the stage for every calculation by showing us what is possible.
1.2 The Addition Rule and Mutually Exclusive Events:
Sometimes, we want to know the probability of either of two events happening. For that, we use the Addition Rule:
\[ P(A \cup B) = P(A) + P(B) - P(A \cap B) \]
This accounts for overlap between events. If the events cannot both happen—in other words, if they are mutually exclusive, then the formula simplifies to:
\[ P(A \cup B) = P(A) + P(B) \]
Definition: Mutually Exclusive Events
Two events are mutually exclusive if they cannot occur at the same time. For example, rolling a 2 and rolling a 5 on a single die roll are mutually exclusive.
Venn Diagram Explanation:
Use a Venn diagram to show overlap (non-exclusive) and separation (exclusive). For exclusive events, the circles don’t touch.
Example: Drawing a Card
You draw a single card from a deck. Let \( A \) be the event "draw a red card" and \( B \) be "draw a black card". Since these events can’t happen together:
\[ P(A \cup B) = P(A) + P(B) = \frac{26}{52} + \frac{26}{52} = 1 \]
1.3 The Subtraction Rule and Complements
Another way to find probabilities is to subtract from the total. The Subtraction Rule says:
\[ P(A^c) = 1 - P(A) \]
Where \( A^c \) is the complement of event \( A \): all outcomes in the space that are not in \( A \).
Example:
If the chance of rain tomorrow is 40%, then the chance it does not rain is:
\[ P(\text{No rain}) = 1 - 0.4 = 0.6 \]
Use complements when it's easier to calculate what you don’t want and subtract from 1.
1.4 The Multiplication Rule and Conditional Probability
Now let’s move into cases where events can happen together. If two events \( A \) and \( B \) are independent (i.e., one happening does not affect the other), then:
\[ P(A \cap B) = P(A) \cdot P(B) \]
But if they’re not independent, we must use:
\[ P(A \cap B) = P(A) \cdot P(B \mid A) \]
This is the Multiplication Rule in its general form.
Conditional Probability
Conditional probability measures the likelihood of \( B \) happening given that \( A \) has already occurred:
\[ P(B \mid A) = \frac{P(A \cap B)}{P(A)} \]
This is also called the Division Rule, since it "undoes" the multiplication rule to isolate \( P(B \mid A) \).
Example: Drawing from a Bag
A bag has 3 red and 2 blue balls. If you draw one ball and don’t replace it, the chance that the second ball is blue depends on the first draw. This is a conditional situation.
1.5 Independence and Dependence
Events \( A \) and \( B \) are independent if knowing \( A \) tells us nothing about \( B \):
\[ P(A \cap B) = P(A) \cdot P(B) \]
Otherwise, they are dependent, and we must consider conditional probabilities.
Example: Rolling Two Dice
Rolling a 2 on die 1 and a 5 on die 2 are independent events:
\[ P(\text{2 on die 1 and 5 on die 2}) = \frac{1}{6} \cdot \frac{1}{6} = \frac{1}{36} \]
1.6 Tree Diagrams and Updating Probabilities
Tree diagrams help visualize sequences of events, especially when outcomes depend on earlier events.
Each "branch" shows a conditional path, and multiplying along the branches gives the joint probability.
1.7 Bayes Rule
Bayes Rule lets us reverse conditional probabilities. It’s used when we know \( P(B \mid A) \) but want to find \( P(A \mid B) \):
\[ P(A \mid B) = \frac{P(B \mid A) \cdot P(A)}{P(B)} \]
Example: Medical Testing
- 1% of people have a disease
- Test is 99% accurate (true positive and true negative)
If someone tests positive, what is the chance they actually have the disease?
You’ll see through Bayes' Rule that even with a “positive test,” the chance of having the disease is surprisingly low due to the rarity of the disease.