Project 3 description 2026–27

Stochastic Optimal Control

Supervisor: Andrew Allan

Project research area: Probability

Background

Many problems require one to decide on the input of a system with a randomly evolving output. In other words, they involve a system which evolves randomly in time, and require the user to choose the value of a control variable in order to optimise the expected output of the system. This might be deciding on the production speed to best meet supply demand, or deciding on the value of a financial portfolio to maximise the terminal wealth. A stochastic optimal control problem is one in which one wishes to optimise an expected outcome by determining the corresponding optimal control.

Mathematically, we can model the input and output of a randomly evolving system using a stochastic differential equation (SDE). A stochastic optimal control problem can then be expressed as the task of determining the coefficient(s) of the SDE which result in the solution which best satisfies a specific reward or loss criterion. The combined use of stochastic calculus and the dynamic programming principle allows one to convert this stochastic problem into a particular type of PDE, known as a Hamilton–Jacobi–Bellman (HJB) equation and, at least in principle, by solving this equation one can find the optimal control.

Group project

In the group project you would begin by getting to grips with some of the fundamental principles of SDEs and stochastic calculus, and then explore some of the theory and applications of stochastic optimal control theory.

More specifically, by the end of the group project you will have a solid understanding of some fundamental aspects of stochastic analysis, including
  • stochastic processes and martingales in continuous-time,
  • Brownian motion,
  • stochastic integration and stochastic calculus,
  • and stochastic differential equations.
and you will have learned about some of the theory of stochastic optimal control, in particular
  • the dynamic programming principle,
  • the derivation of the HJB equation,
  • the verification theorem,
  • and linear-quadratic stochastic control,
and you will have implemented an optimal control strategy on simulated data.

Mode of operation and evidence of learning for the group project

The project will revolve around learning through reading with focus on the underlying theory, mathematical rigour, and the development of deep conceptual understanding. Students will demonstrate their understanding by solving relevant problems, exploring examples and theoretical applications of the material, and clearly communicating it in both written and oral formats.

Individual project

In the individual project you would build on the knowledge gained in the group project. There is scope for either a somewhat pure project on some theoretical aspects of stochastic control, such as
  • the Pontryagin maximum principle,
  • backward stochastic differential equations (BSDEs),
  • or optimal stopping problems,
or a slightly more applied project looking at applications to mathematical finance, for instance
  • optimal execution,
  • pairs trading,
  • market making,
  • or stochastic portfolio theory.

Mode of operation and evidence of learning for the individual project

The project will revolve around learning through reading with focus on the underlying theory, mathematical rigour, and the development of deep conceptual understanding. Students will demonstrate their understanding by solving relevant problems, exploring examples and theoretical applications of the material, and clearly communicating it in both written and oral formats.

Prerequisites

Probability II is an essential prerequisite. This project would be suitable for students also intending to take the course Mathematical Finance III. Taking Stochastic Processes III alongside may also be helpful.

Resources

There are a number of books on the subject, such as:

You can also find various sets of lecture notes online, such as: