• No results found

Show that the uniform distribution is a stationary distribution for the associated Markov chain

N/A
N/A
Protected

Academic year: 2021

Share "Show that the uniform distribution is a stationary distribution for the associated Markov chain"

Copied!
2
0
0

Loading.... (view fulltext now)

Full text

(1)

Markov Chains (273023), Exercise session 2, Tue 22 Jan 2013.

Exercise 2.1. Let P be a doubly stochastic matrix, i.e. all the row sums and the column sums equal to one. Show that the uniform distribution is a stationary distribution for the associated Markov chain.

Exercise 2.2. Let (X0, X1, . . .) and (Y0, Y1, . . .) be irreducible and ape- riodic Markov chains with finite state spaces Ω1,Ω2 and transition probability matrices P1, P2. Let (Z0, Z1, . . .) be a process defined on Ω = Ω1 ∪ Ω2 with a transition probability matrix

P = P1 0 0 P2

 .

Let π1 and π2 be the unique stationary distributions of (X0, X1, . . .) and (Y0, Y1, . . .). Show that (Z0, Z1, . . .) is a Markov chain and find its stationary distributions in terms of π1 and π2.

Exercise 2.3 (Levin, Peres, Wilmer: Ex. 1.2 p. 18). A graph G is con- nected when, for any two vertices x and y of G, there exists a sequence of vertices x0, x1, . . . , xk such that x0 = x, xk = y, and xi ∼ xi+1 for all 0 ≤ i ≤ k − 1. Show that the simple random walk on G is irreducible if and only if G is connected.

Exercise 2.4 (Levin, Peres, Wilmer: Ex. 1.11 p. 19). Let P be a transition probability matrix of a finite Markov chain. Let µ be the uniform probability distribution over the state space. Define

νn= 1 n

n−1

X

k=0

µPk

for n = 1, 2, . . . . Show that the sequence νn is bounded and hence there exists a converging subsequence with limit ν. Further show that ν is a stationary distribution.

Exercise 2.5 (Levin, Peres, Wilmer: Ex. 1.15 p. 19). Let A be a subset of a finite state space Ω of a Markov chain. Define

f(x) = E(τA, X0 = x) where

τA= min{t ≥ 0 : Xt ∈ A}.

Show that f (x) = 0 for every x ∈ A and f(x) = 1 +X

y∈Ω

P(x, y)f (y)

for every x ∈ Ω \ A. Conclude that f is uniquely determined by the above equations. This method is called First Step Analysis.

1

(2)

2

Exercise 2.6 (from the final exam of the Spring 2011). Let

P =

.3 .2 0 0 .5 0 0 .6 .4 0 0 0 0 .4 .6 0 0 0 .1 .1 .1 .6 0 .1

0 0 0 0 0 1

0 0 0 0 1 0

on the state space Ω = {1, 2, 3, 4, 5, 6}. Let A ⊂ Ω be the union of the absorbing classes. Assume that X0 = 4. Using the result of the previous exercise, compute E(τA, X0 = 4).

References

Related documents

Att förhöjningen är störst för parvis Gibbs sampler beror på att man på detta sätt inte får lika bra variation mellan de i tiden närliggande vektorerna som när fler termer

In our modelling we assumed the contribution of root decomposition to soil respiration in the trenched plots to be minor during the second growing season after the trenching and

The set of all real-valued polynomials with real coefficients and degree less or equal to n is denoted by

Let A be an arbitrary subset of a vector space E and let [A] be the set of all finite linear combinations in

Vad vyerna i största möjliga mån skall innehålla för att få en förståelse är en initierare, en beskrivning av säkerhetsfunkt- ionen, vilka krav som finns på

Barnets ståndpunkt skall klarläggas och tas hänsyn till, samtal med eller observationer av barnet skall ingå, utredningen ska ge en helhetsbild genom att belysa barnets situation

Like before, the problem with too few data (see chapter 4.3) appears for the third order Markov chain, it appears for the weekly prediction when using window length 4, 5, 7 and

Further research has to be conducted in order to establish a more concrete business model for a pro-active DSO. The target consumer element is something that needs more research