AN ANALYTICAL METHOD TO CALCULATE THE ERGODIC AND DIFFERENCE MATRICES OF THE DISCOUNTED MARKOV DECISION PROCESSES

Hence

After some easy transformation we receive:

(I-βP)=

1 - 0, 5β -0, 5β

-0, 4β 1 - 0, 6β

T 1 - 0, 5β -0, 4β

⁼ -0, 5β 1 - 0, 6β

and

(I - βP)^-¹

1 1 - 0,6β 0,5β

det (I - βP)[ 0, 4β 1 - 0, 5β

11 β²

det(I-βP) = (1 -0,5β) (1 -0,6β) = 1

-₁₀ β + β₀ = (i — β )(1 — 0, ιβ ).

Finally

(I - βP)^-¹

1-0,6β

(¹^-β)(¹^-⁰^,¹^β)
0,4β

(¹^-β)(¹^-⁰^,¹^β)

0,5β

(¹^-β)(¹^-⁰^,¹^β)
1-0,5β

(¹^-β)(¹^-⁰^,¹^β)

Now as an example, we decompose into partial fractions the first element of ma-
trix (1 - βP)^-¹.

We receive:

1 - 0,6β = D1¹1 ₊ D21 = 4 ₊ 5

⁽¹^-^β)⁽¹^-⁰^, 1^β) ⁽¹^-^β) ⁽¹^-⁰^, 1^β) ⁽¹^{- e}^{) (1}^-⁰^, 1^β)

because

1 - 0, 6β = D₁¹₁ (1 - 0, 1β) + (1 - β) D₁²₁.