24-06-2013, 03:15 PM
Markov Chains Matrix Analysis
Markov Chains.pdf (Size: 550.22 KB / Downloads: 104)
Agenda
Finite State DTMC can be represented by matrices
Thus, we can analyze techniques to study their
Transient Bahaviour
Stationary Distribution
Steady-State Distribution
Innite DTMC can be analyzed in steady-state by using balance
equations
properties
Pn remains a column stochastic matrix for n 0.
The sum of each column of Pn is exactly 1.
All elements of Pn are real nonnegative numbers.
Nonzero elements in P can increase or decrease in Pn but can never
become zero.
Zero elements in P can increase in {n but can never become negative.
There is a dominant eigenvalue equal to 1 ( = 1)
Thus, as n ! 1 all non-dominant eigenvalues i ! 0
MATLAB Exercise
Dene an initial condition vector s(0)
Compute the distribution vector for a specic n
MATLAB function: Pns(0)
Compute the distribution vectors from 0 to n
Repeated multiplication: MATLAB function state vs time(P; s(0); n)
Plot the distribution vector evolution
MATLAB function: plot state vs time(P; s(0); n)