method has to be applied.1 This involves a dis-
cretisation scheme.
The Kushner [9] approach is an efficient discreti-
sation scheme, in which the state-space and time
steps remain related. An implementation of this
approach to infinite horizon decision problems has
been successful even in the case of stochastic dif-
ferential games [3].
In [8], [7] and [6] a simple approach was intro-
duced that produced numerical solutions to a
few finite-horizon stochastic optimal control prob-
lems. Instead of looking for a solution to the HJB
equation, as in the Kushner approach, a Markov
decision chain, discrete in time and space, was
solved. This is a more elementary exercise: instead
of looking for a numerical solution to a second-
order partial differential equation (HJB), a first
order difference equation (Bellman’s) needs to be
solved.
In this paper, the original continuous optimal con-
trol problem is discretised to produce a Markov
decision chain. A method of approximating the
continuous noise by a discretely valued noise is ap-
plied. Value iteration is used to solve the Bellman
equation for the Markov decision chain thus ob-
tained. This “simple” Markovian approximation
method was directly applied in [8] and [7] to esti-
mate the discounted profit of stochastic resource
utilisation. Encouraging results were reported; in
particular, a good level of agreement of numerical
solutions with the existing solutions (see [11] ) was
achieved and a sensible degree of computational
complexity of the method was observed. In [6],
the same method was used for the solution to the
classical portfolio management problem (see [2]).
While the utility measures of the approximating
and original problems were similar, there were
some discrepancies in the policy shapes. These are
overcome in this paper through a scaling in the
policy space and a weakly consistent discretisation
scheme of the Ito diffusion process.
The emphasis of the paper is on the solution
method. However, a few financial engineering
problems, difficult to solve analytically, will be
solved numerically in this paper. In particular,
rules will be computed for non stationary2 and
1 Computational methods have been used for financial
optimisation for quite some time, see for example [11] and,
for a review, [15].
2 Analytical optimal portfolio rules are known for the
HARA (Hyperbolic Absolute Risk-Aversion) utility func-
Constrained3 portfolio problems. A bond pricing
problem will be solved through a repetitive solu-
tion of the Markov decision chain.
The rest of this paper is organised as follows. In
Section 2 the Markovian approximation method
from [8] is modified through use of weakly con-
sistent (Euler) 2-value noise discretisation, rather
than an intuitively motivated 3-value noise dis-
cretisation scheme used in [8], [7] and [6]. The
method is applied, in Sections 4-5, to a classical
optimal portfolio selection problem from [2]. The
portfolio problem is defined and solved analyti-
cally in Section 3. Numerical solutions of varying
degree of computational effort are calculated in
Section 4 and compared to the analytical solution.
In Section 5, a few specific problems of financial
engineering are solved. Concluding remarks close
the paper.
2. A SIMPLE MARKOVIAN
APPROXIMATION
2.1 Optimal Stochastic Control
Consider the stochastic system to be controlled
dX(t) = f(X(t),u(t),t)dt
+b(X(t),u(t),t)dW(t) (1)
where
X = {X(f) ∈ X C IR”, t > O, X(O) = Xq — given}
is the state process, u(t) ∈ U C IR™ is the control,
W(f) is a Wiener process, f(X(t),u(t),t) is a
drift, and b(X(t),u(t),t)dW(t) is diffusion. For
the formal treatment of the optimally controlled
diffusion process refer to [2]. The optimal control
rule μ that determines the control и is Markovian
u(t) = μ(t,X(t)) (2)
and chosen so as to maximise a functional J
max J(0,xo',u) (3)
U
where
fions include isoelastic, exponential and quadratic utility
functions, family of utility functions, see [10]. However, the
explicit solutions to some “practical” problems that would
allow for time dependent model parameters are usually
beyond the simple quadratures.
3 This is another class of analytically intractable yet sen-
sible portfolio problems. In principle, constrained policies
could be obtained through the Kuhn-Tucker conditions. In
practice, their closed forms are unobtainable.