Background

Stationarity is a very common assumption in time-series analysis. In the context of Gaussian processes, it asserts that the mean, variance and covariances governing the process remain constant over time. A practical advantage of modelling using a stationary process is that the variance of the forecast distribution does not grow without bound as the forecast horizon increases. This is often a desirable property, especially for applications in the life sciences where dynamic processes can be subject to functional or structural constraints, which limit the plausible values over which the time-series can vary.

Autoregressive models are a widely used class of time-series model and applied in a variety of fields, such as psychology, neuroscience, biology, macroeconomics and finance. An autoregressive model of order \(p\), often abbreviated to an AR(\(p\)) model, expresses the random variable \(y_t\) at time \(t\) as a linear combination of its previous \(p\) values and an error term \(\epsilon_t\), that is, \(y_t = \phi_1 y_{t-1} + \ldots + \phi_p y_{t-p} + \epsilon_t\). Conditional on \(y_{t-1}, \ldots, y_{t-p}\), the random variable \(y_t\) is independent of any observations further back in the history of the time series. In Gaussian models, the error terms \(\epsilon_t\) are modelled as normal random variables, independent of past values \(\epsilon_{t-1}, \epsilon_{t-2}, \ldots\). A sufficient condition for an autoregressive process to be stationary is that the roots of the so-called characteristic polynomial lie outside the unit circle. This restricts the autoregressive coefficients \(\phi_1, \ldots, \phi_p\) to lie in a constrained space called the stationary region, for example \(\phi_1 \in (-1, 1)\) if \(p=1\). When the order of the process is greater than \(p=2\), the stationary region has a complicated shape which presents challenges for Bayesian inference. Fortunately, a simple reparameterisation of the model in terms of the partial autocorrelations at lags 1 through \(p\) eliminates this problem, requiring only that each partial autocorrelation lies in the open interval \((-1, 1)\).

In many applications in time-series analysis, the complexity of the process being studied, demands that we account for features such as time trends, periods of volatility or regime shifts; some examples are illustrated in Figure 1. This can be achieved by using autoregressive processes as building blocks in more sophisticated models, such as Markov-switching autoregressive processes or time-varying parameter autoregressions. Such models often contain a large number of parameters. Fortunately, to avoid issues caused by overparameterisation, a novel strategy is to restrict such models to be locally stationary, that is, constraining the autoregressive coefficients at every time point to lie within the stationary region, by reparameterising the model in terms of the partial autocorrelations.

(a) Response times (log ms) of a single participant on a series of lexical decision tasks, showing switches between two states (slow but accurate responding versus fast guessing) as the reward for accurate responding was varied. (b) Monthly US adult male unemployment rate from 1956 to 1999, showing gradual shifts over time. (c) Daily log-returns of the Swiss Market Index (SMI) from 12th November 1990 to 20th October 2000, showing evidence of time-varying volatility.

Figure 1: (a) Response times (log ms) of a single participant on a series of lexical decision tasks, showing switches between two states (slow but accurate responding versus fast guessing) as the reward for accurate responding was varied. (b) Monthly US adult male unemployment rate from 1956 to 1999, showing gradual shifts over time. (c) Daily log-returns of the Swiss Market Index (SMI) from 12th November 1990 to 20th October 2000, showing evidence of time-varying volatility.

In a Bayesian analysis, the model specification is completed through an appropriate choice of a prior for the parameters of the model. But even for the simplest stationary AR(1) process, the posterior is analytically intractable and so the usual approach is to sample from it using Markov chain Monte Carlo (MCMC) methods. This is challenging because the mapping between autoregressive coefficients and partial autocorrelation parameters is non-linear. But it is exciting because of its novelty and the wide range of potential applications!

Individualising the project

The goal of this project is to introduce and explore Bayesian inference for models based on stationary autoregressions. However, there is substantial flexibility over the precise topic of research. One option would be to focus on specific features of stationary, or near-stationary, autoregressions, such as:

Another option would be to focus on a class of locally stationary autoregressive processes through, for example

Mode of operation and evidence of learning

The project will revolve around learning through reading and programming. Students will demonstrate their understanding by comparing theory to simulation results, writing code to implement core methodology, analysing simulated and real data sets, and clearly communicating the material in both written and oral formats. Implementing core methodology could involve writing bespoke code in R or Python, using a probabilistic programming language like Stan, or both.

Prerequisites and Co-requisites

Prerequisites: Bayesian Computation and Modelling III.

Co-requisites: Spatio-temporal Statistics.

Additional information

If you would like more information about this project, please contact me at

Resources