Download CS 312: Algorithm Analysis

This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License.
CS 312: Algorithm Design &
Analysis
Lecture #11: Review of Basic
Probability Theory
Slides by: Eric Ringger
Announcements
 Due now: HW #28
 Project #7: TSP
 Early: next Wednesday
 Due: next Friday
 Competition (“The Bake-Off”)
 Big 3 problems in the Competition set survey
 All 3 problems in the One-Time Competition set
survey
Objectives
 Learn important ideas from probability
theory
 Prepare for the proof of the average case
analysis of Quicksort
Average Case Analysis
 We know:
 Quicksort takes 𝑂(𝑛2 ) in the worst case
 and 𝑂(𝑛 log 𝑛) in the best case.
 What’s the average case?
 How would you prove it?
Average Case Analysis
 How would you approach an average case
analysis?
Basic Probability Theory
We need the following ideas:
 Sample spaces
 Samples / Outcomes
 Events
 Probability measures
 Probability spaces
 Random variables
 Values of random variables
 Probability mass functions
 Expected value of random variable
Samples
Samples
Samples
Events
Sigma Field
Probability Measure
Probability Space
Example: One Fair Die
Example: One Fair Die
Probability in 3-D
Random Variables
Random Variables
Example: Two Rolls of a Die
 Ω = {1: 1, 1: 2, 1: 3, … 2: 1, 2: 2, 2: 3, …
6: 1, 6: 2, … 6: 6}
 Let 𝑋 be a random variable representing
the sum of the two rolls:
𝑆 = 2, 3, 4, … , 12
Values and Distributions
 Define a probability mass function for 𝑋:
 We write: 𝑝 𝑋 = 𝑥 = 𝑝𝑋 𝑥 = 𝑝 𝑥
 We mean: 𝑝 𝐴𝑥 where 𝐴𝑥 = 𝜔 ∈ Ω 𝑋 𝜔 = 𝑥}
 Lurking behind every value of a RV is an event!
 The pre-image of every value 𝑥 is an event 𝐴𝑥
 We speak of values of 𝑥 (in the range) as “events”, just as we did for
subsets 𝐴𝑥 of the domain.
 We say: 𝑋 ~ 𝑝 𝑥
Back to the Example
Lurking behind every value of a RV is an
event!
Expectation
Example
Example #2
Basic Probability Theory
We now have the following ideas that build nicely
from one to the other:
 Sample spaces
 Samples / Outcomes
 Events
 Probability measures
 Probability spaces
 Random variables
 Values of random variables
 Probability mass functions
 Expected value of random variable
Example #3
Assignment
 HW #8