(1b) πt(zt) ∈ Γ[πt-1(zt-1), zt].
Let Π(x0 , z0) denote the set of plans that are feasible from (x0 , z0). This set is
nonempty if the correspondence Γ is nonempty and a certain measurability constraint is
met.
Assumption 1. Γ is nonempty-valued and the graph of Γ is (X × X × Z)-measurable.
In addition, Γ has a measurable selection; that is, there exists a measurable function
h : (X, Z) → X such that h(x, z) ∈ Γ (x, z) for all (x, z) ∈ S.
Under this assumption, the set Π(x0, z0) is nonempty for all (x0, z0) ∈ S.6 A plan π
constructed by using the same measurable selection h from Γ in every period t is said to
be stationary or Markov, since the action it prescribes for each period t depends only on
the state [πt-1(zt-1), zt] in that period. Nonstationary plans can be constructed by using
different measurable selections ht in each period. Let a feasible plan and the transition
function Q on (Z, Z) be given. We want to calculate the total, discounted, expected
returns associated with this plan. Given the initial state (x0 , z0) ∈ S, we define the
following probability measures μt(z0, ∙) : Zt → [0,1]:
μt(zo, Z)
Z1 .. Zt-1 Zt
Q(zt-1, dzt)Q(zt-2, dzt-1)..Q(z0, dz1),
∀t ∈ N.
The domain of the per-period return function F is the set A, the graph of Γ. Then we
can define the set A as:
A = {C ∈X×X×Z : C ⊆ A}.
Under Assumption 1, A is a σ-algebra. Furthermore, if F is A-measurable, then for any
(x0, z0) ∈ S and any π ∈ Π(x0, z0),
F [πt-1(zt-1), πt(zt), zt] is Z t -measurable, ∀ t ∈ N.
This rationalizes our next assumption.
Assumption 2. F : A → R is A-measurable, and either (a) or (b) holds.
(a) F ≥ 0 or F ≤ 0
(b) For each (x0, z0) ∈ S and each plan π ∈ Π,
F[πt-1(zt-1), πt(zt), zt] is μt-integrable, ∀ t ∈ N,
and the limit
F[x0, π0, z0] +
lim
n→∞
n
t=1 Z
βtF[∏t-i(zt-1), ∏t(zt), zt]μt(xo, dzt)
exists (although it may be plus or minus infinity).
Assumption 2 ensures that, for each (x0, z0) ∈ S, we can define the functions un (∙, x0, z0) :
Π(x0,z0) →R,n ∈ N0, by:
u0 (π, x0, z0)
un (π, x0, z0)
F[x0, π0, z0],
F[x0, π0,
n
z0]+
t 1 Zt
βtF [∏t-i(zt-1),∏t (zt),zt] μt(xo,dzt).
6A proof of this result can be found in Lucas and Stokey (1989), page 243.
14