AN ANALYTICAL METHOD TO CALCULATE THE ERGODIC AND DIFFERENCE MATRICES OF THE DISCOUNTED MARKOV DECISION PROCESSES

det (I - βP) = (1 - 0, 5β) (1 - β²) = (1 - β)(1 + β) (1 - 0,5β)

(I - βP)^-1

ɪ ^β „ 0

1-β 1-β²⁰

^β¹ 0

1-β² 1-β²⁰

0,5β² 0,5β² 1

₁ Γ 0, 5 0, 5 0 '
0,5 0,5 0 +

¹^-^β 0, 5 0, 5 0

₁ Γ 0 0 0 ■

+-------- 0 0 0

1 - 0,5β _2 _ 1 1

- 3 3 ¹ -

(1-β² )(1-0,5β) (1-β²)(1-0,5β) 1-0,5β

₁ Γ 0,5 -0,5 0 '

+ —- -0,5 0,5 0

¹⁺^β¹ -¹ 0

Hence

^ν∞ (^β) = ⁽¹^-^β ∙ ^P) ¹^q =C-j----X [∙] ⁺ 1 I [ [∙ ∙ ∙] ⁺ ^j----+~^ξ+ [∙ ∙ ∙]

y 1 - β 1 + β 1 - 0, 5β

-3

5, 5

2, 5

-2, 5

5
6

_ι 1

⁺ 1 - 0, 5β

Now we can calculate total finite expected rewards for given values β, β₁ = 0, 5
and β₂ = 0, 99. For β₁ = 0, 5 we obtain

ν∞ (0, 5) =

1 - 0, 5

5, 5

_ι 1

⁺ 1 + 0, 5

2, 5
-2,5

5
6

_ι 1

⁺ 1 - 0,5 ∙ 0, 5

3 -

and next

ν₁,∞ (0, 5) = 2 ∙ 5, 5 + 0, 666 ∙ 2, 5 + 1, 333 ∙ 0 = 11 + 1, 666 + 0 = 12, 666,
ν2,∞ (0, 5) = 2 ∙ 5, 5 - 0, 666 + 0 = 9, 334,