AN ANALYTICAL METHOD TO CALCULATE THE ERGODIC AND DIFFERENCE MATRICES OF THE DISCOUNTED MARKOV DECISION PROCESSES



was decomposed into N component forms:

. ,Dji      ʌ, k =1, 2, 3,...,N.

det (1 - αkβ)

It can be shown, that |Dij |k 1 and even values express probability and α1 ≡ 1.
Next connecting so received
N elements on condition equal factors (1 - αKβ),
we create
N separate (N × N) matrices. The first matrix is ergodic matrix pro-
ducted by 1
/ (1 - β). For irreducible ergodic Markov Chain, this matrix will be con-
structed from the same rows.

Next N - 1 matrices will be difference matrices, each different and elements will
be divided by factors (
I - αkβ), k = 2, 3, 4, . . . , N.

Now the formula for total rewards can be written in the following form:

ν(β ) = (ʌ D1ii + i-½ hD2ii +... + i—½ [dn ]) ∙ q. (19)

1 - β j 1 - α2β  j         1 - αN β  j

Next we consider and solve two simple examples, which show application of pre-
sented method.

4 Examples

Example 1

Let P be one-step stochastic matrix of transition of irreducible Markov Chain
with finite set of states
N . Let N = 2 and let matrix P have the following form
25]:

0, 5 0, 5

0, 4 0, 6

Matrix of rewards R is also given,

R=


3 -7




More intriguing information

1. Foreign Direct Investment and Unequal Regional Economic Growth in China
2. Concerns for Equity and the Optimal Co-Payments for Publicly Provided Health Care
3. An Efficient Circulant MIMO Equalizer for CDMA Downlink: Algorithm and VLSI Architecture
4. APPLICATIONS OF DUALITY THEORY TO AGRICULTURE
5. An Interview with Thomas J. Sargent
6. Special and Differential Treatment in the WTO Agricultural Negotiations
7. DISCUSSION: ASSESSING STRUCTURAL CHANGE IN THE DEMAND FOR FOOD COMMODITIES
8. ROBUST CLASSIFICATION WITH CONTEXT-SENSITIVE FEATURES
9. THE INTERNATIONAL OUTLOOK FOR U.S. TOBACCO
10. IMMIGRATION POLICY AND THE AGRICULTURAL LABOR MARKET: THE EFFECT ON JOB DURATION