AN ANALYTICAL METHOD TO CALCULATE THE ERGODIC AND DIFFERENCE MATRICES OF THE DISCOUNTED MARKOV DECISION PROCESSES

3 Method of calculation of ergodic and differen-
ce matrices

We consider dependence for total discounted rewards given by formula (10) again.

ν∞ (β) = (I - βP)^-1 ∙ q.                         (11)

It is not difficult to notice, that

(I - ^βP) = det (i — βp) (I ^{- βp}) ad,               (¹2)

where (I - βP) ad is an algebraically complement of matrix (I - βP). Next
we can write

(I - βP) ad = [Dji (β)], i,j = 1,N

where Dji (β) = (—1)^j+i ∙ Mji (β), and Mji (β) is a minor of matrix (I — βP)T,
hence

Theorem:

Let determinant of matrix (I — βP) have real and singular roots, then for each
stochastic matrix P and factor β < 1 exist such αk 6= 0, k = 1, 2, . . . , N that true
is the following formula:

JDiiL ₊ [D2i] ₊...₊JDNL
(1 — αιβ) (1 — α2β)         (1 — αNβ),
where

det (I — βP) = (1 — α1β) (1 — α2β) . . . (1 — αNβ) . . . ,           (16)

<<   <   2   3   4   5   6   7   8   >   >>