A Appendices
A.1 Finding the value function by iteration
In Sections A.2 and A.3 of this appendix, we will use an iterative method to find the value
functions that attain the suprema of the two DOPs considered in Sections 3 and 4. We
introduce some basic concepts of stochastic dynamic programming from the textbook by
Stokey and Lucas (1989) and finally formulate Theorem 9.12. This verification theorem
states that under certain conditions a solution to the Bellman equation is necessary and
sufficient even in the stochastic unbounded returns case.
Let (X, X) and (Z, Z) be any measurable spaces, and let (S, S) := (X × Z, X × Z) be
the product space. The set X is the set of possible values for the vector of endogenous
state variables, Z is the set of possible values for the exogenous shock, and S is the set
of possible states of the system. The evolution of the stochastic shocks is described by a
stationary transition function Q on (Z, Z).
In each period t, the decision-maker chooses the vector of endogenous states in the
subsequent period. The constraints on this choice are described by a correspondence
Γ : X × Z → X; that is, Γ(x, z) is the set of feasible values for next period’s state
variables if the current state is (x, z). Let A be the graph of Γ:
A = {(x, y, z) ∈ X × X × Z : y ∈ Γ(x, z)} .
Let F : A → R be the per-period return function. Hence F (x, y, z) gives us the current
period return if the current state is (x, z) and y ∈ Γ(x, z) is chosen as next period’s vector
of endogenous state variables. The constant one-period discount factor is denoted by β
and we assume β ∈ (0, 1). The givens for the problem at hand are (X, X ), (Z, Z), Q, Γ,
F , and β .
In period 0, with the current state (x0, z0) known, the decision maker chooses a value
for x1 . In addition, he makes contingency plans for periods t ∈ N. He realizes that the
decision to be carried out in period t depends on the information that will be available
at that time. Thus he chooses a sequence of functions, one for each period t ∈ N. The
t-th function in this sequence specifies a value for xt+1 as a function of the information
that will be available in period t. For t ≥ 1, this information is the sequence of shocks
(z1, z2, .., zt). The decision maker chooses this sequence of functions to maximize the
expected discounted sum of returns, where the expectation is over realizations of shocks.
We define the following product spaces:
(Zt,Zt)= (Z × .. × Z,Z × .. × Z),
4 sz z 4 sz z
t times t times
for all t ∈ N. Furthermore let zt = (z1, .., zt) ∈ Zt denote a partial history of shocks in
periods 1 through t.
Definition 1. A plan is a value π0 ∈ X and a sequence of measurable functions πt :
Zt → X, t ∈ N.
Hence, in period t with the partial history of shocks zt, the function πt (zt) tells us
the value of next period’s states xt+1 .
Definition 2. A plan π is feasible from (x0, z0) ∈ S if
(1a) π0 ∈ Γ(x0, z0),
13