Download MC Costs

IENG 362
Markov Chains
Cost Functions in M.C.
•Suppose C(Xt) = cost of being in state Xt. In our
inventory example, suppose it costs $2 / unit for
each time period inventory is carried over. Then
•
C( X t ) =
{
0
2
4
6
,
,
,
,
Xt = 0
Xt =1
Xt = 2
Xt = 3
Cost Functions
•In general, the expected cost over all states
and n time periods then is given by
n
1
E [  C( X t ) ]
n t =1
Cost Functions
•Consider the first 4 time periods, with some
arbitrary path. For the path given,
3
Cost = C ( X 0 = 3) + C ( X 1 = 2)
+ C ( X 2 = 0) + C ( X 3 = 2)
2
= 6+ 4+ 0+ 4
1
0
t=0
t=1
t=2
t=3
Cost Functions
•But the probability of incurring this cost is
the probability associated with this path; i.e.
P{Cost = 14} = P32 C( X 0 = 3)
+ P20C( X 1 = 2)
+ P02C( X 2 = 0)
+ P2 j C( X 3 = 2)
3
2
1
0
t=0
t=1
t=2
t=3
Cost Functions
•The expected cost is the cost associated with
each path x the probability associated with each
path.
M
E [ Cost ] =  P{ X 1 = k | X 0 = i }C ( X 0 = i )
k =0
3
2
1
0
t=0
t=1
t=2
t=3
Cost Functions
•The expected cost is the cost associated with
each path x the probability associated with each
path.
M
E [ Cost ] =  P{ X 1 = k | X 0 = i }C ( X 0 = i )
k =0
3
M
+  P{ X 2 = i | X 1 = k }C ( X 1 = k )
i =0
2
M
+  P{ X 3 = k | X 2 = i }C ( X 2 = i )
k =0
1
M
+  P{ X 4 = i | X 0 = k }C ( X 3 = k ) 0
i =0
t=0
t=1
t=2
t=3
Cost Functions
•But we know how to consider all possible
paths to a state. Consider
( 2)
Cost x Prob = Pi 2 C ( X 2 = 2)
3
2
3
E [ Cost t = 2 ] =  Pij
j =0
(2)
C( X 2 = j) 1
0
t=0
t=1
t=2
t=3
Cost Functions
•But we know how to consider all possible
paths to a state. Consider
3
1 n
1 n
E[  C ( X t ) ] =  E[ C ( X t ) ]
n t =1
n t =1
2
1 n M (t )
=   Pij C( X t = j ) 1
n t =1 j = 0
0
1 M n (t )
=   Pij C ( X t )
n j = 0 t =1
t=0
t=1
t=2
t=3
Cost Functions
•The long run average cost then may be given
by
1 n
E [Cost ] = lim E [
n
M
=
(
j =0
M
C( X

n
1 n (t )
Pij

n t =1
=  p j C( j )
j =0
j =1
t
= j) ]
) C( X
t
= j)
Example; Inventory
•Suppose C(Xt) = cost of being in state Xt. In
our inventory example, suppose it costs $2 /
unit for each time period inventory is carried
over. Then
0 , Xt = 0
2 , Xt =1
• ( Xt ) =
C
4 , Xt = 2
6 , Xt = 3
{
t=1
t =
Example; Inventory
•Suppose C(Xt) = cost of being in state Xt. In
our inventory example, suppose it costs $2 /
unit for each time period inventory is carried
M
6
over.
E [Cost ]Then
=  p j C( j )
j =0
4
•
2
0
t=1
t =
Complex Costs
• Inventory;
•
•
Suppose an order for x units is made at a cost of $25 +
10x. For unsatisfied demand, cost of a lost sale is $50.
If (s,S) = (0,3) inventory policy is used,
{
C ( X t -1 , Dt ) =
25 + 10( 3)
, X t -1 < 1 , Dt  3
25 + 10( 3) + 50( Dt - 3) , X t -1 < 1 , Dt > 3
0
, X t -1  1 , Dt  Xt-1
50( Dt -Xt-1 )
, X t -1  1 , Dt > Xt-1
Complex Costs
E [ Costs ] = lim
n 
1 n (t )
E [  Pij C ( X t -1 , Dt ) ]
n t =1
Complex Costs
E [ Costs ] = lim
n 
1 n (t )
E [  Pij C ( X t -1 , Dt ) ]
n t =1
a miracle occurs
M
=  p j k ( j)
j =0
Example; Inventory
M
E [Cost ] =  p j k ( j )
j =0
6 p3 = .166
k (0) = E [C (0, Dt ) ]
= 55 + 50 P{Dt = 4}
4
+ 100 P{Dt = 5} + ...
= 56
p2 = .264
2 p1 = .285
0 p0 = .285
t=1
t =
Example; Inventory
M
E [Cost ] =  p j k ( j )
j =0
k (1) = E [C (1, Dt ) ]
6 p3 = .166
= 50 P{Dt = 2} + 100 P{Dt = 3}
4
+ 150 P{Dt = 4} + ...
-1
-1
2 p1 = .285
-1
e
e
e
+ 100
+ 150
+ ...
2!
3!
4!
= 18.4
t=1
= 50
p2 = .264
56
0 p0 = .285
t =
Example; Inventory
M
E [Cost ] =  p j k ( j )
j =0
k (2) = E [C (2, Dt ) ]
6 p3 = .166
= 50 P{Dt = 3} + 100 P{Dt = 4}
+ 150 P{Dt = 5} + ...
-
-
-
e1
e1
e1
= 50
+ 100
+ 150
+ ...
3!
4!
5!
= 5.2.
t=1
4
p2 = .264
18.4 2 p1 = .285
56
0 p0 = .285
t =
Example; Inventory
M
E [Cost ] =  p j k ( j )
j =0
k (3) = E [C (3, Dt ) ]
6 p3 = .166
= 50 P{Dt = 4} + 100 P{Dt = 5}
+ 150 P{Dt = 6} + ...
-
-
-
e1
e1
e1
= 50
+ 100
+ 150
+ ...
4!
5!
6!
= 1.2.
t=1
5.2 4
p2 = .264
18.4 2 p1 = .285
56
0 p0 = .285
t =
Example; Inventory
M
E [Cost ] =  p j k ( j )
j =0
1.2 6 p3 = .166
= 56(0.285) + 18.4(0.285)
+ 5.2(0.264) + 12
. (0166
. )
5.2 4
p2 = .264
18.4 2 p1 = .285
= $22.70
56
t=1
0 p0 = .285
t =