Download STA 348 Introduction to Stochastic Processes Lecture 5

STA 348
Introduction to
Stochastic Processes
Lecture 5
1
Example

Starting with $i, you play a game of coin flips:
At each flip you either win or lose $1 w.p. ½.
What is the probability of reaching $n before
you lose all of your initial $i (where 0<i<n)?
2
Example
3
Conditioning & Difference
Equations

Want to find some probability/expectation, call
it un, by conditioning. When applying Law of
Total Prob. / Exp. to the problem, often end up
with an equation of the form
un  fn  un1, un2,


where fn is some function, and un−1, un−2,… are
additional related prob’s/exp’s
Called Difference (or Recurrence) Equation

Discrete analogue of differential equations
4
Difference Equations
How do you solve Difference Equations?

Some cases can be solved by standard
methods (e.g. substitution, linear algebra)

General solutions for any function fi can be
difficult / impossible to find analytically


May require numerical solution using a computer
For specific types of function f, there are
closed form expressions for the solution
5
Examples

Solve the difference equation un=n·un−1, with
initial condition u0=c
6
Classification of Difference
Equations

Focus on equations with non-changing fn


Difference Equation classification:




i.e. fn(x) = f(x) for all n
Linear equation: f is a linear function
Order of equation: highest “lag” in u’s subscript
Homogeneous: no constant term in linear function
E.g. 2nd order linear difference equations:
un  aun 1  bun  2
(homogeneous)
un  aun 1  bun  2  c
(inhomogeneous)
7
1st Order Homogeneous
Linear Difference Equation

For un=aun−1 with initial condition u0=c, the
solution is un=anu0=anc

Proof:
8
1st Order Inhomogeneous
Linear Difference Equation

For un=aun−1+b with a≠1, initial condition u0=c
n
n
1 a
1 a
n
solution is un  a u0  b
 a cb
1

a
1

a
 Proof:
n
9
2nd Order Homogeneous
Linear Difference Equation


Consider the equation un=aun−1+bun−2
Let’s try solution of the form un=Arn, for A,r∈ℝ

Plugging it back to the difference equation, we get
Ar n  aAr n 1  bAr n  2  r 2  ar  b  r 2  ar  b  0
2
 r  ar  b  0 called the characteristic equation
2
r
 ar  b  0
 Let r1 ≠ r2 be distinct roots of


2
a

a
 4b
Roots of quadratic: r 
2
2
, for a  4b  0
Any linear combination of r1, r2 is also a
solution to the original difference equation
10
2nd Order Homogeneous
Linear Difference Equation

For the difference equation un=aun−1+bun−2
the solution is of the form un=A1r1n+A2r2n
where r1 ≠ r2 are distinct roots of the
characteristic equation r2−ar − b=0

If characteristic equation has double root r1=r2
then solution is of the form un=(A1+A2 · n ) r1n

The values of the constants A1, A2 are given by
solving for two initial conditions u0, u1
11
Initial Conditions

The initial conditions of a difference equation
are not necessarily the first values


E.g. for 2nd order equation, the initial conditions
can be: {u0, u1}, or {u0, uN} for a particular index N,
or {uN−1, uN}, or any other 2 values
In general, we need as many initial conditions
as the order of the difference equation

E.g. for 1st order equation the initial condition can
be u0, or uN, or any other value
12
Example

The Fibonacci numbers are given by:
Fn  Fn 1  Fn  2 , with F0  0 & F1  1

Find a closed form expression for Fn
F0  0
F1  1
F2  1
F3  2
F4  3
F5  5
F6  8
F7  13

13
2nd Order Inhomogeneous
Linear Difference Equation

For the difference equation un=aun−1+bun−2+c
the solution is of the form un=g(n)+h(n) , where


g(n) is solution of the associated homogeneous
equation un=aun−1+bun−2 , i.e. g(n)=A1r1n+A2r2n
h(n) is particular solution of un=aun−1+bun−2+c


●
h(n) must satisfy h(n)=ah(n−1)+bh(n−2)+c
h(n) can only be one of three possible functions:
● h(n) = k OR h(n) = kn OR h(n) = kn2
● Try out all three to find the one which works
Finally, solve for 2 initial conditions to find A1, A2
14
Example

Solve un−6un−1+5un−2=3, with u0=u1=3
15
Example
16
Example (Gambler’s Ruin Problem)

Starting with $i, you play a sequence of
gambles: At each gamble you either win $1
w.p. p≠1/2, or lose $1 w.p. q=1−p. What is the
probability of reaching $n before you lose all
of your initial $i (where 0<i<n)?
17
Example (Gambler’s Ruin Problem)
18
Example

Gamble $10 in Roulette (betting on
red / black) till you double or loose it


P(winning individual bet) = 18/(36+2) = .4737
Find the probability of doubling your money if:
A.
Bet $1 at a time
B.
Bet $2 at a time
19