Proposition 2 V(X) is concave and differentiable with V'(X) = -Bk — 2X
We will briefly outline the idea behind the second proof and give some intuition why the value
function is differentiable even at Xc .
The proof uses a contraction mapping argument. Contraction mappings have a unique fixed
point, which is the value function: continuous application of the contraction mapping will lead to
convergence towards the value function. The first step is to establish that the Bellman equation
constitutes a contraction mapping. In the second step we show that the Bellman equation maps
concave functions into concave functions. We can hence start with an arbitrary concave function,
and after repeatedly applying the contraction mapping, we will converge to the true value function,
which must be concave as well. In the third step we use the result of Benveniste and Scheinkman
[1] that gives conditions under which concave functions are differentiable.
The reason why the value function is differentiable even at Xc lies with the error term vt . Re-
call that vt enters the equation that determines whether the threshold is passed. Start with the case
where σv = 0, so that there is no uncertainty whether the threshold is crossed or not. If we slightly
perturb the loading BXt + b + lt around Xc, the equation of motion has a discrete discontinuous
jump equal to r > 0 and hence the value function would not be continuous either. However, as
long as σv > 0, there is no discrete jump as the system crosses the threshold dependent on whether
BXt+b+lt is less or more than Xc-vt. By design, vt has a continuous probability distribution, so
changing the combined loading shifts this continuous probability distribution of the discontinuous
jump, which ensures that the value function is itself differentiable. Note that σv can be as small as
desired, so long as it is nonzero.
Using the fact that V is differentiable we can now solve for the optimal loading c by maximizing
the right hand side of the Bellman equation.
Proposition 3 The optimal combined loading is given by
c = k [1 - B - r [1 - φ ΓXc ck 1 - 1—φ fX ck [Bkr + 2Xcr + r2 + σ22 - σU]
2 δ σv 2σv σv u2 u1
The derivation is again given in the Appendix.
Several things deserve further explanation. First, the above equation includes the results of
[25] as special cases. A model with no additional input r is equivalent to saying that Xc = ∞,
which implies that φ (Xσ-c) = 0, Φ (Xσ-c) = 1 and hence the expected stock in the next period
is E[Xt+1] = c= 2 [ɪ — B]. On the other hand, if the additional input r is always present,
Xc
∞, which implies that φ (Xσ-s) = 0, Φ (X⅛-s) = 0, and hence the expected stock in