Summary

Stationary distributions describe the limiting behaviour of ergodic Markov chains. However, there are also transient (not ergodic) Markov chains which seem to settle down to an equilibrium long before the inherent instable or evanescent behaviour becomes manifest. This phenomenon is called quasi stationary behaviour of a Markov chain. In this thesis we analyse this quasi stationary behaviour in the setting of time-homogeneous Markov chains with a discrete time parameter and denumerably infinite state space.

A first category of Markov chains which may display quasi stationary behaviour are chains for which the state space consists of an irreducible class plus an absorbing state. If the time to absorption is large, it is possible for the chain to reach an apparent equilibrium over the non-absorbing states after a relatively short time. One way to describe this quasi stationary behaviour is by means of the limit, as time tends to infinity, of the conditional distribution of the process, given that absorption has not yet occurred. If this limit exists and constitutes a proper distribution, it is called a limiting conditional distribution. A related concept is that of a quasi-stationary distribution which is an initial distribution on the non-absorbing states such that the conditional distribution of the process, given that no absorption has occurred, is independent of time. There exists a close connection between these two distributions since every quasi-stationary distribution is a limiting conditional distribution and vice versa. For a Markov chain with a given initial distribution a problem is to determine conditions for the limiting conditional distribution to exist and, if it exists, to identify it with the corresponding quasi-stationary distribution.

A second category of Markov chains which may display quasi stationary behaviour are transient Markov chains on the nonnegative integers for which state 0 is a reflecting boundary. As long as the drift to infinity has not set in, the Markov chain may seem to be in equilibrium. We can gain an understanding of this behaviour by studying the limiting behaviour of the state probabilities conditioned on not having left state 0 for the last time. If these limits constitute a proper distribution, this distribution is once again called a limiting conditional distribution.

In Chapter 1 some real-life examples of quasi stationary behaviour are discussed. We also give an overview of the most important results in the literature on quasistationary and limiting conditional distributions of Markov chains with an absorbing state.

Chapter 2 treats a number of specific features of discrete-time birth-death processes, to which we shall refer throughout as random walks. The analysis is largely based on the spectral representation of the n-step transition probabilities and an important role is played by the decay parameter of the chain. The spectral representation is in terms of a sequence of polynomials (random walk polynomials) which is orthogonal with respect to a positive measure (random walk measure) on the interval [−1, 1]. If the decay parameter is less than 1 the Markov chain is called geometrically ergodic. Besides a description of all these concepts and their relations, we present the Derman - Vere-Jones transformation: a method by which we can analyse a random walk with absorbing state through a random walk with a reflecting boundary.

In Chapter 3 we analyse a random walk on the nonnegative integers for which state -1 is an absorbing state. Under the assumption that absorption is certain, we prove that geometric ergodicity is a necessary and sufficient condition for the existence of a quasi-stationary distribution. When this condition is satisfied, however, there even exists a complete family of quasi-stationary distributions for which we can find an explicit expression in terms of the parameters of the random walk and the corresponding random walk polynomials.

Since every quasi-stationary distribution is a limiting conditional distribution, geometric ergodicity must also be a necessary condition for the existence of the limiting conditional distribution of a random walk with arbitrary initial distribution. However, in Chapter 4 we show that this condition is not sufficient. Before determining necessary and sufficient conditions for the limiting conditional distribution to exist, we first examine the ratio limits of the n-step transition probabilities of a random walk. We give a necessary and sufficient condition for the simultaneous existence of the ratio limits in terms of the random walk measure. It appears that under this same condition the limiting conditional distribution of a geometrically ergodic Markov chain also exists. In this chapter we do not assume absorption to be certain. We can still use the concept of limiting conditional distribution if we condition on the fact that absorption has not yet occurred but will occur eventually.

In Chapter 5 we discuss quasi stationary behaviour of transient Markov chains which are skip-free to the left on the nonnegative integers for which state 0 is a reflecting boundary. We prove that R-positivity of the Markov chain is a sufficient condition for the limiting conditional distribution to exist. We next determine the limiting conditional distribution of a truncated Markov chain. It turns out that this distribution always exists and that, under the assumption of R-positivity, the limiting conditional distribution of the truncated Markov chain converges, as the size of the state space of the truncated chain increases to infinity, to the limiting conditional distribution of the original Markov chain.