AN ANALYTICAL METHOD TO CALCULATE THE ERGODIC AND DIFFERENCE MATRICES OF THE DISCOUNTED MARKOV DECISION PROCESSES



3 Method of calculation of ergodic and differen-
ce matrices

We consider dependence for total discounted rewards given by formula (10) again.

ν(β) = (I - βP)-1 ∙ q.                         (11)

It is not difficult to notice, that

(I - βP) = det (i βp) (I - βp) ad,               (12)

where (I - βP) ad is an algebraically complement of matrix (I - βP). Next
we can write

(I - βP) ad = [Dji (β)], i,j = 1,N

(13)


where Dji (β) = (1)j+i Mji (β), and Mji (β) is a minor of matrix (I βP)T,
hence

(I-βP)-1


[Dji (β)]
det (I — βP)


(14)


Theorem:

Let determinant of matrix (I βP) have real and singular roots, then for each
stochastic matrix P and factor β < 1 exist such α
k 6= 0, k = 1, 2, . . . , N that true
is the following formula:

(I βP )-1


JDiiL + [D2i] +...+JDNL
(1 αιβ) (1 α2β)         (1 αNβ),
where

(15)


det (I βP) = (1 α1β) (1 α2β) . . . (1 αNβ) . . . ,           (16)



More intriguing information

1. Land Police in Mozambique: Future Perspectives
2. The Tangible Contribution of R&D Spending Foreign-Owned Plants to a Host Region: a Plant Level Study of the Irish Manufacturing Sector (1980-1996)
3. NATURAL RESOURCE SUPPLY CONSTRAINTS AND REGIONAL ECONOMIC ANALYSIS: A COMPUTABLE GENERAL EQUILIBRIUM APPROACH
4. The name is absent
5. CROSS-COMMODITY PERSPECTIVE ON CONTRACTING: EVIDENCE FROM MISSISSIPPI
6. The Triangular Relationship between the Commission, NRAs and National Courts Revisited
7. ‘I’m so much more myself now, coming back to work’ - working class mothers, paid work and childcare.
8. The name is absent
9. Food Prices and Overweight Patterns in Italy
10. The name is absent