Module 1: Random Variables & Expectation

Interactive Review & Geometric Demonstrations

1. Probability Basics

Axioms of Probability

  • Non-negativity: \( 0 \le P(E) \le 1 \)
  • Normalization: \( P(S) = 1 \)
  • Additivity: For disjoint events, \( P(\cup E_i) = \sum P(E_i) \)

Inclusion-Exclusion

For non-disjoint events:

\( P(A \cup B) = P(A) + P(B) - P(A \cap B) \)

Interactive Geometric Demo: Inclusion-Exclusion

Drag circles to move • Use sliders to resize

Radius A 80
Radius B 80
Area(A) 0
Area(B) 0
Intersection (A ∩ B) 0
Union (A ∪ B) 0

Formula Check:

2. Conditional Probability & Independence

Conditional Probability

\( P(A|B) = \frac{P(A \cap B)}{P(B)} \)

Bayes' Theorem

\( P(A|B) = \frac{P(B|A)P(A)}{P(B)} \)

Interactive Geometric Demo: Conditional Probability

Visualize the sample space shrinking! The large rectangle is the Universe. Green is Event B. Blue is Event A. The Darker Region is \( A \cap B \).
Notice how \( P(A|B) \) is just the fraction of the Green box that is covered by Blue.

Size of Event B (Condition) 50%
Overlap (Intersection) 20%

Conditional Probability

P(A|B) = 0.40

\( \frac{P(A \cap B)}{P(B)} = \frac{\text{Area}(Intersection)}{\text{Area}(B)} \)

3. Random Variables

Feature Discrete RV Continuous RV
Values Countable (e.g., integers) Uncountable (e.g., reals)
Function PMF \( p_X(x) = P(X=x) \) PDF \( f_X(x) \)
Probability Sum: \( \sum_{x \in A} p_X(x) \) Integral: \( \int_A f_X(x) dx \)
Point Prob \( P(X=x) \ge 0 \) \( P(X=x) = 0 \)

4. Expectation & Variance

Linearity of Expectation

Holds for ANY RVs X, Y (even dependent)!

\( E[aX + bY] = aE[X] + bE[Y] \)

Variance Formula

\( Var(X) = E[X^2] - (E[X])^2 \)

Variance Properties

  • \( Var(aX + b) = a^2 Var(X) \)
  • \( Var(X + Y) = Var(X) + Var(Y) \) Only if Independent

Example: Linearity with Coin Tosses

Problem: A fair coin is tossed \( n \) times. Let \( X \) be the number of heads. Find \( E[X] \).

Solution Steps:

  1. Define indicator variable \( X_i \) for the \( i \)-th toss: 1 if Heads, 0 if Tails.
  2. Express \( X \) as sum: \( X = X_1 + X_2 + \dots + X_n \).
  3. Calculate individual expectation: \( E[X_i] = 1 \cdot (1/2) + 0 \cdot (1/2) = 1/2 \).
  4. Apply Linearity: \( E[X] = \sum E[X_i] = n \cdot (1/2) = n/2 \).

Test Your Knowledge

Select True or False for each statement.