AN ANALYTICAL METHOD TO CALCULATE THE ERGODIC AND DIFFERENCE MATRICES OF THE DISCOUNTED MARKOV DECISION PROCESSES

and

D_j^k_i, k = 1, 2, 3, . . . , N, are constant factors different from zero.

Proof:

From linear algebra results, that the determinant of matrix (I - βP ) always exist,
it is bigger than zero and it is a polynomial of N - degree. Each polynomial of N -
degree has exactly N different real roots. Hence we can show det (I - βP ) in form
(16). Let prove (15) and write this formula in the following form:

[Dji (β )]  =  [⅞]  ₊ jd⅛l ...JDNL

det (I — βP)   (1 — α₁β)   (1 — α₂β)        (1 — αNβ)

This formula shows decomposition of the left side of dependence (17) into sum
of N partial fractions. Such a decomposition is always possible. If we want to cal-
culate values of factors D_j^k_i, k = 1, 2, 3, . . . , N, we should solve (N × N) systems
of equations:

D1      D².           Dⁿ          ___

--j— +--j— +  +--j-—  ij = 1 N
1 — αι β ⁺1 — α2β ^{+   +}1 — αN β ^,j ,

it means

Dji (β)    = Dji ^{(1 — α}2^β) ... ^{(1 — α}N^β) ⁺ Dji ^{(1 — α}1^β) ^{(1 — α}3^β) . . .

det (I — βP)                       det (I — βP)

... (1 — αNβ) + ... + DjN (1 — αιβ) (1 — α2β)... (1 — αN_-ιβ)
det (I — βP)

After rejection of denominators of both sides we compare factors which stand in
front of the same powers β, β⁰, β¹ , . . . , β^N-1 of the left and right side of nume-
rators.

So each element of matrix

[Dji (β)]   = Γ   Dji (β)

det (I — βP) det (I — βP)

<<   <   3   4   5   6   7   8   9   >   >>