AN ANALYTICAL METHOD TO CALCULATE THE ERGODIC AND DIFFERENCE MATRICES OF THE DISCOUNTED MARKOV DECISION PROCESSES



In this same way, we decompose the other three elements of matrix and we rece-
ive:

4          5         5         _ 5

9 I__9                 9      _|__9

(I - βP)-1


(1) + (1-0,1β)   (1) + (1-0,1β)

4         _4        5         4

9                 9            9               9

(1) + (1-0,1β)   (1) + (1-0,1β)

Finally formula (15) becomes the following form:

(I - βP)-1


(1 - β)


0, 1β


We can check that the first matrix with factor 1/ (1 - β) is ergodic matrix of Mar-
kov Process for given stochastic matrix of transition P . The second matrix is so
named difference matrix. The sum of elements is equal zero in rows of this matrix.
Taking formula (19) into consideration, we receive total finite expected reward:

ν(β) =


1

(1 - β)


' 4

9

4

. 9


9  +--------

9      1 - 0,1β


• q.


Now we can find value ν (β) for two different β, β1 = 0, 5 and β2 = 0, 99.

After providing of values and simple calculations we receive:

ν(0, 5) =

5
9
5

9


+ 1, 052


-3


Hence we obtain for the starting state and n → ∞

ν1,∞ (0, 5) = 2 • 1 + 1, 052 • 5 =

7, 260,




More intriguing information

1. Short Term Memory May Be the Depletion of the Readily Releasable Pool of Presynaptic Neurotransmitter Vesicles
2. The name is absent
3. Subduing High Inflation in Romania. How to Better Monetary and Exchange Rate Mechanisms?
4. The name is absent
5. Reversal of Fortune: Macroeconomic Policy, International Finance, and Banking in Japan
6. Ability grouping in the secondary school: attitudes of teachers of practically based subjects
7. The Role of State Trading Enterprises and Their Impact on Agricultural Development and Economic Growth in Developing Countries
8. PACKAGING: A KEY ELEMENT IN ADDED VALUE
9. Innovation and business performance - a provisional multi-regional analysis
10. The name is absent