(1b) πt(zt) ∈ Γ[πt-1(zt-1), zt].
Let Π(x0 , z0) denote the set of plans that are feasible from (x0 , z0). This set is
nonempty if the correspondence Γ is nonempty and a certain measurability constraint is
met.
Assumption 1. Γ is nonempty-valued and the graph of Γ is (X × X × Z)-measurable.
In addition, Γ has a measurable selection; that is, there exists a measurable function
h : (X, Z) → X such that h(x, z) ∈ Γ (x, z) for all (x, z) ∈ S.
Under this assumption, the set Π(x0, z0) is nonempty for all (x0, z0) ∈ S.6 A plan π
constructed by using the same measurable selection h from Γ in every period t is said to
be stationary or Markov, since the action it prescribes for each period t depends only on
the state [πt-1(zt-1), zt] in that period. Nonstationary plans can be constructed by using
different measurable selections ht in each period. Let a feasible plan and the transition
function Q on (Z, Z) be given. We want to calculate the total, discounted, expected
returns associated with this plan. Given the initial state (x0 , z0) ∈ S, we define the
following probability measures μt(z0, ∙) : Zt → [0,1]:
μt(zo, Z)
Z1 .. Zt-1 Zt
Q(zt-1, dzt)Q(zt-2, dzt-1)..Q(z0, dz1),
∀t ∈ N.
The domain of the per-period return function F is the set A, the graph of Γ. Then we
can define the set A as:
A = {C ∈X×X×Z : C ⊆ A}.
Under Assumption 1, A is a σ-algebra. Furthermore, if F is A-measurable, then for any
(x0, z0) ∈ S and any π ∈ Π(x0, z0),
F [πt-1(zt-1), πt(zt), zt] is Z t -measurable, ∀ t ∈ N.
This rationalizes our next assumption.
Assumption 2. F : A → R is A-measurable, and either (a) or (b) holds.
(a) F ≥ 0 or F ≤ 0
(b) For each (x0, z0) ∈ S and each plan π ∈ Π,
F[πt-1(zt-1), πt(zt), zt] is μt-integrable, ∀ t ∈ N,
and the limit
F[x0, π0, z0] +
lim
n→∞
n
t=1 Z
βtF[∏t-i(zt-1), ∏t(zt), zt]μt(xo, dzt)
exists (although it may be plus or minus infinity).
Assumption 2 ensures that, for each (x0, z0) ∈ S, we can define the functions un (∙, x0, z0) :
Π(x0,z0) →R,n ∈ N0, by:
u0 (π, x0, z0)
un (π, x0, z0)
F[x0, π0, z0],
F[x0, π0,
n
z0]+
t 1 Zt
βtF [∏t-i(zt-1),∏t (zt),zt] μt(xo,dzt).
6A proof of this result can be found in Lucas and Stokey (1989), page 243.
14
More intriguing information
1. The Functions of Postpartum Depression2. Pupils’ attitudes towards art teaching in primary school: an evaluation tool
3. The name is absent
4. The name is absent
5. Parallel and overlapping Human Immunodeficiency Virus, Hepatitis B and C virus Infections among pregnant women in the Federal Capital Territory, Abuja, Nigeria
6. The name is absent
7. The name is absent
8. Disentangling the Sources of Pro-social Behavior in the Workplace: A Field Experiment
9. The name is absent
10. Short report "About a rare cause of primary hyperparathyroidism"