Download Lecture 1: “Introduction - The Method of Positive Probability”

The Probabilistic Method - Probabilistic Techniques
Lecture 1: “Introduction - The Method of
Positive Probability”
Sotiris Nikoletseas
Associate Professor
Computer Engineering and Informatics Department
2014 - 2015
Sotiris Nikoletseas, Associate Professor
The Probabilistic Method
1 / 28
Άδειες Χρησης
Το παρον εκπαιδευτικο υλικο υποκειται σε αδειες χρησης
Creative Commons.
Για εκπαιδευτικο υλικο, οπως εικονες, που υποκειται σε αλλου
τυπου αδεια χρησης, η αδεια χρησης αναφερεται ρητως.
Sotiris Nikoletseas, Associate Professor
The Probabilistic Method
2 / 28
Χρηματοδοτηση
Το παρον εκπαιδευτικο υλικο εχει αναπτυχθει στα πλαισια του
εκπαιδευτικου εργου του διδασκοντα.
Το εργο “Ανοικτα Ακαδημαϊκα Μαθηματα για το Πανεπιστημιο
Πατρων” εχει χρηματοδοτησει μονο την αναδιαμορφωση του
εκπαιδευτικου υλικου.
Το εργο υλοποιειται στα πλαισια του επιχειρισιακου
προγραμματος “Εκπαιδευση και Δια Βιου Μαθηση” και
συγχρηματοδοτειται απο την Ευρωπαϊκη Ένωση (Ευρωπαϊκο
Κοινοτικο Ταμειο) και απο εθνικους πορους.
Sotiris Nikoletseas, Associate Professor
The Probabilistic Method
3 / 28
The Probabilistic Method - major applications
a powerful tool used in many applications in different topics:
study of random graph models (Gn,p , Gn,R , Gn,k etc) which are
typical instances for average case analysis of graph
algorithms.
design and analysis of randomized algorithms:
evolution based on random choices
solutions provided a) either are always correct but their
running time is a random variable (Las Vegas algorithms) b) or
may be erroneous but are correct w.h.p. (Monte Carlo
algorithms)
trade-off performance (faster, simpler) with very small,
controlled error probability.
Sotiris Nikoletseas, Associate Professor
The Probabilistic Method
4 / 28
The core of the method
The Probabilistic Method
uses simple techniques
the Basic Method
Linearity of Expectation
as well as complex ones
the Local Lemma
Martingales
Markov Chains
but there is a common, underlying concept:
The core of the method
Non-constructive (μη-κατασκευαστικη) proof of existence of
combinatorial structures that have certain desired properties.
Sotiris Nikoletseas, Associate Professor
The Probabilistic Method
5 / 28
The Basic Method (method of “positive probability”)
Construct (by using abstract random experiments) an
appropriate probability sample space of combinatorial
structures (thus, the sample points correspond to the
combinatorial structures whose existence we try to prove).
Prove that the probability of the desired property in this
space is positive (i.e. non-zero).
⇓
There is at least one point in the space with the desired
property.
⇓
There is at least one combinatorial structure with the desired
property.
Sotiris Nikoletseas, Associate Professor
The Probabilistic Method
6 / 28
Characteristics of the P.M.
comprehensible, pretty short proofs
simple (basic knowledge of Probabilistic Theory, Graph
Theory, Combinatorics suf ices)
elegant
qualitative ideas, subtle notions
not lengthy, mechanical operations
still very powerful (use to resolve extremely dif icult
problems)
Sotiris Nikoletseas, Associate Professor
The Probabilistic Method
7 / 28
Examples in this lecture
(i) Monochromatic arithmetic progressions (Van der Waerden
property)
(ii) Ramsey Numbers
(iii) Coloring Hypergraphs
(iv) Tournaments with property Sk
Sotiris Nikoletseas, Associate Professor
The Probabilistic Method
8 / 28
(I) Van der Waerden property
De inition 1
W(k) is the smallest natural number n, such that for any
two-coloring of the numbers 1, 2, ..., n there is a monochromatic
arithmetic progression of k terms.
Theorem 1
k
W(k) > 2 2
Sotiris Nikoletseas, Associate Professor
The Probabilistic Method
9 / 28
Proof of Theorem 1 (1/3)
We construct a probability space by two-coloring the
numbers 1, 2, ..., n at random, equiprobably for the two
colors and independently for every number. Clearly, the
sample points of this space are random two-colorings of the n
numbers.
Let S be any ixed arithmetic progression of k terms.
De ine the event MS := {S is monochromatic}.
i.e, all terms of S must have the same color.
Compute the probability Pr[MS ].
every term is colored red (or blue) with probability 1/2
all k terms are red-colored (or blue-colored) with probability
( 12 )k
( )k ( )k
1
1
+
= 21−k
Pr[MS ] =
2
2
Sotiris Nikoletseas, Associate Professor
The Probabilistic Method
10 / 28
Proof of Theorem 1 (2/3)
De ine the event M := {∃ at least one monochromatic
∪
arithmetic progression of k terms } ⇒ M = |S|=k MS .
An arithmetic progression of k terms is (de) ined uniquely by
n
its two irst terms ⇒ There are at(most
2 arithmetic
)
progressions ⇒ #(S : |S| = k) ≤ n2
Using Boole’s inequality we can compute Pr[M]


( )
∪
 ∑
n 1−k
Pr[MS ] ≤
Pr[M] = Pr
MS ≤
2


2
|S|=k
Sotiris Nikoletseas, Associate Professor
|S|=k
The Probabilistic Method
11 / 28
Proof of Theorem 1 (3/3)
We easily get:
Pr[M] <
n2 1−k n2
2
= k
2
2
k
If n < 2 2 then Pr[M] < 1 ⇒ Pr[M] > 0.
Hence, there is a two-coloring without a monochromatic
k
arithmetic progression of k terms when n < 2 2 .
k
Thus, W(k) > 2 2 .
Sotiris Nikoletseas, Associate Professor
The Probabilistic Method
12 / 28
(II) Ramsey Numbers
De inition 2
The Ramsey number R(k, l) is the smallest integer n such that in any
two-coloring of the edges of the complete graph on n vertices Kn by
red and blue colors, either there is a red Kk or there is a blue Kl .
Dif iculty of computation:
Ramsey (1930) proved that R(k, l) is inite
Greenwood and Gleason (1955) computed R(3, 3) = 6 and
R(4, 4) = 18
since then there is no notable progress - R(4, 5) is still
unknown
Erdö s suggested that R(6, 6) is too dif icult to be computed
Sotiris Nikoletseas, Associate Professor
The Probabilistic Method
13 / 28
Ramsey Numbers
R(k, k): diagonal Ramsey number (a monochromatic Kk is
required).
Theorem 2 (Erdö s, 1947)
If
(n)
k
k
21−(2) < 1 then R(k, k) > n.
Sotiris Nikoletseas, Associate Professor
The Probabilistic Method
14 / 28
Proof of Theorem 2 (1/3)
Construct a probability sample space by two-coloring at
random, equiprobably (for the two colors) and
independently (for the edges) every edge of Kn .
Let S be any ixed set of k vertices and consider the edges
induced.
De ine the event MS := {S is monochromatic}.
i.e. all
(k)
2
edges in S have the same color.
Compute the probability Pr[MS ].
every edge is colored red (or blue) with 1/2 probability
( )( k ) ( )( k )
k
1 2
1 2
Pr[MS ] =
+
= 21−(2)
2
2
Sotiris Nikoletseas, Associate Professor
The Probabilistic Method
15 / 28
Proof of Theorem 2 (2/3)
De ine the event M := {∃ at least one monochromatic set of k
vertices}.
∪
Hence, M = |S|=k MS .
Using Boole’s inequality we can compute the Pr[M]
( )
∑
n 1−( k )
Pr[M] ≤
Pr[MS ] =
2 2
k
|S|=k
Sotiris Nikoletseas, Associate Professor
The Probabilistic Method
16 / 28
Proof of Theorem 2 (3/3)
If Pr[M] < 1 ⇒ Pr[M] > 0
()
k
⇒ if nk 21−(2) < 1 then there is a point in the sample space
without M ⇒ there is a monochromatic Kk .
Hence, it must be R(k, k) > n.
Sotiris Nikoletseas, Associate Professor
The Probabilistic Method
17 / 28
Lower Bound of Ramsey Numbers
()
k
We proved that if nk 21−(2) < 1 then R(k, k) > n
()
k
If nk 21−(2) ∼ 1 then we can ind the best possible lower
bound for R(k, k) (with this derivation).
By using Stirling’s formula and binomial approximation we
obtain:
k2
nk 1−( k )
nk
·2 2 ∼ √
· 2− 2 ∼ 1
(
)
k
k!
2πk ke
( )k
√
k2
k
k
⇒ n ∼ 2πk ·
·22
e
k k
⇒ R(k, k) > n ∼ √ 2 2
e 2
Sotiris Nikoletseas, Associate Professor
The Probabilistic Method
18 / 28
(III) Coloring Hypergraphs
De inition 3
A Hypergraph H = (V, E) consists of:
V : a inite set of vertices
E : a set of subsets of V (the “edges”)
De inition 4
A Hypergraph H = (V, E) is called n-uniform iff all edges contain
exactly n vertices.
Sotiris Nikoletseas, Associate Professor
The Probabilistic Method
19 / 28
Property B
De inition 5
A Hypergraph H = (V, E) has property B (it is two-colorable) iff ∃ a
two-coloring of V such that no edge is monochromatic.
De inition 6
m(n) is the minimum number of edges on a n-uniform hypergraph
that does not have property B.
Theorem 3 (Erdö s, 1963)
m(n) ≥ 2n−1
Sotiris Nikoletseas, Associate Professor
The Probabilistic Method
20 / 28
Proof of Theorem 3 (1/2)
Construct a probability sample space by two-coloring the
vertices of H at random, equiprobably for the two colors and
independently for every vertex.
Let e be any ixed edge.
De ine the event Me := {e is monochromatic}.
i.e. all vertices of edge e must have the same color.
Compute the probability Pr[Me ].
Pr[Me ] = 2 ·
1 1 1
1
· · · · · = 21−n
2}
|2 2 {z2
n times
Sotiris Nikoletseas, Associate Professor
The Probabilistic Method
21 / 28
Proof of Theorem 3 (2/2)
De ine the event M := {∃ at least one monochromatic edge}.
∪
Hence, M = e Me
Using Boole’s inequality we can compute Pr[M]
∑
Pr[M] ≤
Pr[Me ] = |e|21−n
e
If |e| · 21−n < 1 (i.e., |e| < 2n−1 ) then Pr[M] < 1 ⇒ Pr[M] > 0.
Hence, there is a two-coloring without a monochromatic
edge when m(n) < 2n−1 ⇒ property B.
Hence, m(n) ≥ 2n−1 is necessary for avoiding property B.
Sotiris Nikoletseas, Associate Professor
The Probabilistic Method
22 / 28
(IV) Tournaments
De inition 7
A tournament Tn is a complete directed graph on n vertices i.e., for
every pair (i, j), there is either an edge from i to j or from j to i, but
not both.
Why do we call these graphs tournaments?
Each vertex corresponds to a team playing at some
tournament.
The directed edge (i, j) means that team i wins team j.
all teams play against each other.
Sotiris Nikoletseas, Associate Professor
The Probabilistic Method
23 / 28
The Sk Property
De inition 8
A tournament Tn is said to have property Sk if for any set of k
vertices in the tournament, there is some vertex that has a directed
edge to each of those k vertices.
Theorem 4 (Erdö s, 1963)
∀k, ∃ a tournament Tn that has the property Sk .
Sotiris Nikoletseas, Associate Professor
The Probabilistic Method
24 / 28
Proof of Theorem 4 (1/2)
Construct a probability sample space with points random
tournaments by choosing the direction of each edge at
random, equiprobably for the two directions and
independently for every edge.
Let S be any ixed set of k teams and de ine the event
MS := {∄ a team that wins all teams in S}.
For any team, the probability to win all teams in S is ( 12 )k .
Hence, the probability of not winning at least one of them is
1 − ( 12 )k .
The probability that this is happening for all n − k teams that
don’t belong in S is:
(
( )k )n−k
1
Pr[MS ] = 1 −
2
Sotiris Nikoletseas, Associate Professor
The Probabilistic Method
25 / 28
Proof of Theorem 4 (2/2)
De ine the event M := {∃ a set S of k teams such that ∄ a team
u : u ̸∈ S that wins all teams in S}.
∪
M = S MS
Using Boole’s inequality we can compute Pr[M]
Pr[M] ≤
∑
S,|S|=k
If
(n) (
k
1−
( )(
( )k )n−k
n
1
Pr[MS ] =
1−
k
2
( 1 )k )n−k
2
< 1 then Pr[M] < 1 ⇒ Pr[M] > 0.
Hence, there is a tournament with property Sk .
Sotiris Nikoletseas, Associate Professor
The Probabilistic Method
26 / 28
Bibliography
Σ. Νικολετσεας και Π. Σπυρακης, “Στοιχεια της Πιθανοτικης
Μεθοδου”, Gutenberg, 1996.
N. Alon and J. Spencer, “The Probabilistic Method”, John Wiley
& Sons, 1992.
B. Bollobá s, “Random Graphs”, Academic Press, 1985.
R. Motwani and P. Raghavan, “Randomized Algorithms”,
Cambridge University Press, 1995.
Sotiris Nikoletseas, Associate Professor
The Probabilistic Method
27 / 28
Sotiris Nikoletseas, Associate Professor
The Probabilistic Method
28 / 28