Description

Many problems require one to decide on the input of a system with a randomly evolving output. In other words, they involve a system which evolves randomly in time, and require the user to choose the value of a control variable in order to optimise the expected output of the system. This might be deciding on the production speed to best meet supply demand, or deciding on the value of a financial portfolio to maximise the terminal wealth. A stochastic optimal control problem is one in which one wishes to optimise an expected outcome by determining the corresponding optimal control.

Mathematically, we can model the input and output of a randomly evolving system using a stochastic differential equation (SDE). A stochastic optimal control problem can then be expressed as the task of determining the coefficient(s) of the SDE which result in the solution which best satisfies a specific reward or loss criterion. The combined use of stochastic calculus and the dynamic programming principle allows one to convert this stochastic problem into a particular type of PDE, known as a Hamilton-Jacobi-Bellman (HJB) equation and, at least in principle, by solving this equation one can find the optimal control.

In this project, you would begin by getting to grips with some of the fundamental principles of SDEs and stochastic calculus, and then explore some of the theory and applications of stochastic optimal control theory. There is scope for either a somewhat pure project on the theoretical aspects of stochastic control, such as the Pontryagin maximum principle and the use of backward SDEs, or a slightly more applied project looking at applications to mathematical finance, for instance problems of optimal consumption or stochastic portfolio theory.

Prerequisites

Probability II is the main prerequisite. This project would be particularly suitable for students also intending to take the course Mathematical Finance III. Taking Analysis III alongside may also be helpful.

Resources

There are a number of books on the subject, such as:

• Jiongmin Yong and Xun Yu Zhou, Stochastic Controls, Hamiltonian Systems and HJB Equations, Springer, 1999.
• Huyên Pham, Continuous-time Stochastic Control and Optimization with Financial Applications, Springer, 2009.

You can also find various sets of lecture notes online, such as:

• Gonçalo dos Reis and David Siska, Stochastic Control and Dynamic Asset Allocation, Lecture notes, 2018.
• H. Mete Soner, Stochastic Optimal Control in Finance, Lecture notes, 2004.