The value of un (π, x0 , z0) is the sum of expected discounted returns in periods 0 through
n from plan π if the initial state is (x0 , z0). Assumption 2 also ensures that for each
(xo, z0) ∈ S we can define u(∙, x0, z0) : Π(xo, z0) → RR to be the limit of the series as the
horizon recedes:
u(π, x0, z0) = lim un (π, x0, z0).
n→∞
Thus u(π, x0 , z0) is the infinite sum of expected discounted returns from the plan π if
the initial state is (x0, z0). Under Assumptions 1 and 2, the function u(∙,x, z) is well
defined on the nonempty set Π(x, z), for each (x, z) ∈ S. In this case we can define the
supremum function v* : S → R by:
v*(x,z) = sup u(π,x,z).
π∈Π(x,z)
That is v* is the unique function satisfying the following two conditions:
v* ≥ u(π, x, z), all π ∈ Π(x, z);
v* = lim u(πk,x, z), for some sequence {πk} inΠ(x, z).
k→∞
In the bounded returns case, a solution v to the functional equation must have the
property that the expected discounted value of the implied policy in the very far future
is equal to zero, that is we exclude for example sustained overinvestment. The difficulty
with the unbounded returns case is that there may be some (x0, z0) ∈ S and π ∈ Π(x0, z0)
for which the condition:
lim βt
t→∞
V v[∏t-i(zt-1 ),zt]μt(zo
Zt
, dzt) = 0,
∀π ∈ Π(x0, z0), ∀ (x0, z0) ∈ S
(37)
does not hold. For each (x0, z0) ∈ S, however, we can define Π(xo, z0) to be the subset
of Π(xo, z0) on which this condition holds. Then define v : S → R by
v(x, z) = sup u(π, x, z)
π∈Π(x,z)
Clearly v ≤ v*. The following theorem provides sufficient conditions for the two functions
to be equal.
Theorem 3. Let (X, X ), (Z, Z), Q, Γ, F , and β satisfy Assumptions 1 and 2. Let Π,
Π, u, v*, and v be as defined above. Suppose v is a measurable function satisfying the
functional equation
[f(x,y,z)+ β J v(y,z')Q(z,dz')j,
v(x, z) = sup
y∈Γ(x,z)
and that the associated policy correspondence G is nonempty and permits a measurable
selection. For each (x, z) ∈ S, let π*(∙,x, z) be a plan generated by G from (x, z). Suppose
in addition that
(a) π*(∙,x, z) ∈ Π(x, z);and
(b) for any (xo, zo) ∈ S and π ∈ Π(xo, zo), there exists π ∈ IΠ(x, z) such that u(π, x, z) ≥
u(π, x, z).
Then v*(x, z) = v(x, z) = v(x, z) = u(π*(∙;x, z), x, z), ∀(x, z) ∈ S.
Proof. See Stokey and Lucas (1989), page 274. □
In the next sections we will apply this theorem to our model.
15