Chapter 2: Random Variables

2.1 What is a Random Variable?

We now start off explaining what a random variable is: a random variable is a way to assign numbers to outcomes of a random process. For example, a random variable is the number of heads that occur when we flip a coin 5 times. These random variables are typically denoted by X, Y, or Z. If we are working with one random variable we denote it with X, and if there is a second random variable we denote it with Y, and so on and so forth.

2.2 Coin Toss Example

Now we will begin with an example: Let \( X \) be a random variable which accounts for the number of heads in 3 tosses of a fair coin. The possible outcomes for X are HHH, HHT, HTT, HTH, THH, TTH, TTT, and THT. This means that \( X \in \{0, 1, 2, 3\} \).

Each outcome has equal probability of occurring. Since there are 8 outcomes, each outcome has the probability of \( \frac{1}{8} \). There is only one outcome where \( X = 3 \), so \( P(X = 3) = \frac{1}{8} \).

There is 1 outcome where \( X = 0 \), giving \( P(X = 0) = \frac{1}{8} \). There are 3 outcomes each for \( X = 1 \) and \( X = 2 \), which gives each a probability of \( \frac{3}{8} \).

\( X \)	Outcomes	\( P(X = x) \)
0	TTT	\( \frac{1}{8} \)
1	TTH, THT, HTT	\( \frac{3}{8} \)
2	THH, HTH, HHT	\( \frac{3}{8} \)
3	HHH	\( \frac{1}{8} \)

We will introduce the probability distribution through a histogram of the events for this example. Then we’ll add another example asking for the probability of \( X \geq 1 \). Using the complement rule, \( P(X \geq 1) = \frac{7}{8} \). We’ll highlight this on the histogram.

2.3 Equality vs. Equality in Distribution

Now we will start talking about equality. Consider the same 3 tosses of the coin. Let NH represent the number of heads, and NT the number of tails. For outcome HHH, NH = 3 and NT = 0, so they are not equal. However, from the previous example, we see that they are equal in distribution since NH and NT have the same probability distribution.

Value	\( P(NH = x) \)	\( P(NT = x) \)
0	\( \frac{1}{8} \)	\( \frac{1}{8} \)
1	\( \frac{3}{8} \)	\( \frac{3}{8} \)
2	\( \frac{3}{8} \)	\( \frac{3}{8} \)
3	\( \frac{1}{8} \)	\( \frac{1}{8} \)

Then we will explain straight-up equality where \( X = Y \) and wrap it up by saying that this is equality.

2.4 Bernoulli Distribution

Now we begin talking about distributions. We first bring up the Bernoulli distribution. It is essentially a binary indicator and describes the distribution of something that can occur as either 0 or 1. It has the values \( p \) and \( 1-p \) and highlights success and failure.

We will talk about how a coin flip is the perfect example of this, where \( p = \frac{1}{2} \) and \( 1-p = \frac{1}{2} \).

2.5 Binomial Distribution

Now we bring up the binomial distribution which takes into account the summation of multiple \( \text{Bernoulli}(p) \) random variables. Let \( X_1, X_2, \dots, X_n \) be i.i.d. (independent identically distributed) \( \text{Bernoulli}(p) \) random variables. Let \( S_n \) be the summation of the total number of successes for all of these variables.

Our main goal is to find the distribution of \( S_n \). We will now use the example of a dice roll: roll a die 10 times and find the probability that the number 4 is rolled exactly 4 times.

In this example, we want \( S_{10} = 4 \). The probability of rolling a 4 is \( \frac{1}{6} \) and we want it to occur 4 times, so we get \( \left(\frac{1}{6}\right)^4 \). For the 6 failures, we get \( \left(\frac{5}{6}\right)^6 \).

Any sequence of four S’s and six F’s has the same probability. By the addition rule:

\[ P(S = 4) = \binom{10}{4} \cdot \left(\frac{1}{6}\right)^4 \cdot \left(\frac{5}{6}\right)^6 \]

\( \binom{10}{4} \) counts the number of ways you can choose 4 places out of 10 in which to put the symbol S, with the remaining 6 places being filled with F. Mathematically this is:

\[ \frac{n!}{k!(n-k)!} \]

The general formula for any binomial distribution problem is:

\[ P(S_n = k) = \binom{n}{k} \cdot p^k \cdot (1-p)^{n-k}, \quad k = 0, 1, 2, \dots, n \]

To use the binomial formula, you first have to recognize that it can be used. Check the conditions:

• a known number of independent, repeated, success/failure trials
• you are counting the number of successes

Steps:

• Identify the two parameters n and p
• Identify each k for which the event occurs
• Add up the \( \text{Binomial}(n, p) \) probabilities for all k in the set (or subtract the complement if it’s easier)

2.6 Multinomial Distribution

What if we don't have just two outcomes? This is where we bring in the multinomial distribution. It’s very similar to the binomial, but it allows us to work with multiple outcomes.

Let’s say we have a box with 3 blue tickets, 5 red tickets, and 2 green tickets — 10 tickets in total. If we draw 15 times with replacement, what is the probability of getting 6 blue, 5 red, and 4 green?

Let:

• \( B = 0.3^6 \)
• \( R = 0.5^5 \)
• \( G = 0.2^4 \)

There are \( \binom{15}{6} \) ways of choosing the number of blues, and \( \binom{9}{4} \) ways of choosing the remaining outcomes.

The full probability expression becomes:

\[ P = \binom{15}{6} \cdot \binom{9}{4} \cdot (0.3)^6 \cdot (0.5)^5 \cdot (0.2)^4 \]

This is simply an extension of the binomial with a third added category.

2.7 Hypergeometric Distribution

Now let’s talk about the hypergeometric distribution. Suppose we have a population of fixed \( N \) individuals, made up of \( G \) “good” elements and \( B \) “bad” elements. So \( N = G + B \).

If we take a simple random sample of \( n \) elements, and let \( X \) be the number of good elements in our sample, then for a possible value \( g \), the distribution is:

\[ P(X = g) = \frac{\binom{G}{g} \cdot \binom{B}{n-g}}{\binom{N}{n}} \]

This is called the hypergeometric distribution.

2.8 Poisson Distribution

Finally, what if we have a rare event? This is where the Poisson distribution comes in.

We know that when \( n \) is large and \( p \) is small, the chance of \( k \) successes in \( n \) i.i.d. \( \text{Bernoulli}(p) \) trials is roughly:

\[ P(X = k) \approx \frac{\lambda^k e^{-\lambda}}{k!} \]

This is known as the Poisson approximation to the Binomial distribution.

Practice Problems

Problem 1: Binomial Distribution

A basketball player has a free throw success rate of 80% (i.e. \( p = 0.8 \)). Suppose she takes 10 free throws in a game.

• What is the probability that she makes exactly 8 shots?
• What is the probability that she makes at least 8 shots?

▶ Click to show solution

Let \( X \sim \text{Binomial}(n = 10, p = 0.8) \)

• \( P(X = 8) = \binom{10}{8} (0.8)^8 (0.2)^2 = 45 \cdot (0.16777) \cdot (0.04) \approx 0.302 \)
• \( P(X \geq 8) = P(X = 8) + P(X = 9) + P(X = 10) \approx 0.302 + 0.268 + 0.107 = 0.677 \)

Problem 2: Multinomial Distribution

A bag contains 5 red marbles, 3 blue marbles, and 2 green marbles. You draw 10 marbles with replacement.

• What is the probability that you draw 4 red, 4 blue, and 2 green marbles?
• What is the most likely outcome combination of red, blue, and green counts?

▶ Click to show solution

Let \( X_1 = \text{# red}, X_2 = \text{# blue}, X_3 = \text{# green} \)

\( p_r = 0.5,\ p_b = 0.3,\ p_g = 0.2 \)

\[ P = \frac{10!}{4!4!2!} (0.5)^4 (0.3)^4 (0.2)^2 \approx 0.1003 \]

The most likely outcome is the one closest to expected values:
\( (5, 3, 2) \)

Problem 3: Hypergeometric Distribution

In a class of 30 students, 12 are wearing glasses and 18 are not. You randomly select 5 students to answer a question.

• What is the probability that exactly 2 of them are wearing glasses?

▶ Click to show solution

\( N = 30, G = 12, B = 18, n = 5, g = 2 \)

\[ P(X = 2) = \frac{\binom{12}{2} \cdot \binom{18}{3}}{\binom{30}{5}} = \frac{66 \cdot 816}{142506} \approx 0.378 \]

Problem 4: Poisson Distribution

A website receives an average of 3 spam messages per hour. Assume the number of spam messages follows a Poisson distribution.

• What is the probability the site receives exactly 5 spam messages in one hour?
• What is the probability it receives no spam messages in one hour?

▶ Click to show solution

\( \lambda = 3 \)

• \( P(X = 5) = \frac{3^5 e^{-3}}{5!} = \frac{243 \cdot e^{-3}}{120} \approx 0.1008 \)
• \( P(X = 0) = \frac{3^0 e^{-3}}{0!} = e^{-3} \approx 0.0498 \)