Download Math 626 Problem Set I (1) The problem of investing money over a

Math 626 Problem Set I
(1) The problem of investing money over a period of N years can be represented as a multi stage decision process where each year $x is subdivided in $u which can be invested and $(x − u) which are spent. The
process is assumed to have a dynamic equation x(i + 1) = au(i), where
the integer i counts the years and the growth rate a is constant and
greater than unity. The satisfaction from money spent in any year is
specified by a function H, H(i) = H(x(i) − u(i)) and the objective is
to maximise the total satisfaction I over N years:
I=
N
X
H(i) .
i=1
Define an optimal return function and write the dynamic programming
functional recurrence equation for the process. Hence find the optimal
investment policy for the two cases:
(a) diminishing returns, H(x − u) =
√
x − u,
(b) linear returns, H(x − u) = x − u.
(2) Find an extremal control for the optimization problem with equation
of motion
dx
=x+u ,
dt
cost function,
Z 1
I=
[x(t)2 + u(t)2 ]dt ,
0
and initial condition x(0) = 1.
(3) A continuous time controlled process has differential equation,
dx
= −x3 + u ,
dt
1
and cost function,
Z t2
I=
(x2 + u2 )dt .
t1
Write down the Hamiltonian and show that the differential equation
for the state and adjoint variables combine to give
d2 x
− x − 3x5 = 0 .
dt2
If the initial value x(t1 ) is given and the final value x(t2 ) is free, what
are the boundary conditions on x in this differential equation?
(4) A continuous time controlled process has differential equation,
dx
=u,
dt
and cost function,
I = x2 (T ) +
Z T
u2 dt .
0
Find the optimal control law for the process.
(5) A continuous time controlled process has differential equation,
dx
=u,
dt
cost function,
Z T
I=
u2 dt ,
0
and the initial and final states are x(0) = X, x(T ) = 0. Show that
the optimal control law results in a trajectory, x(t) = X[1 − t/T ].
2