Stochastic Control Optimization
A.Y. 2026/2027
Learning objectives
Basic problems, methods and results in the theory of optimization of stochastic dynamical systems will be presented. Both discrete-time and continuous-time models will be considered, over finite and infinite horizon. Continuous-time models will be mostly described by stochastic differential equations. The main approaches will be dynamic programming, the study of the Hamilton-Jacobi-Bellman equation (including cases of solutions with low regularity), backward stochastic differential equations, the stochastic maximum principle (in the sense of Pontryagin). A brief introduction to other optimization problems will be given, such as optimal stopping or impulse control, and applications to basic models will be presented, for instance optimal investment problems in Mathematical Finance or linear quadratic optimal control.
Expected learning outcomes
Students attending the course will become acquainted with various classes of control and optimization problems for stochastic systems (with discrete time, with continuous time and formulated by stochastic differential equations, on finite and infinite horizon). They will learn the basic methods to solve such problems: dynamic programming and Hamilton-Jacobi-Bellman equations, backward stochastic differential equations, the stochastic maximum principle. They will also see how important models can be analyzed, such as optimal investment problems in Mathematical Finance and linear quadratic problems.
Lesson period: First semester
Assessment methods: Esame
Assessment result: voto verbalizzato in trentesimi
Single course
This course cannot be attended as a single course. Please check our list of single courses to find the ones available for enrolment.
Course syllabus and organization
Single session
Responsible
Lesson period
First semester
Course syllabus
1) Discrete-time stochastic optimal control and reinforcement learning.
Markov decision processes: finite horizon, stochastic horizon, infinite horizon. Value functions and related Bellman equations. State-action value functions. Policy evaluation, policy improvement, policy iteration, value iteration. Introduction to model-free algorithms. Monte Carlo and temporal-difference methods.
2) Continuous-time stochastic optimal control.
Controlled stochastic differential equations, cost or reward functionals over a finite or infinite horizon. Value function and the dynamic programming principle. Hamilton-Jacobi-Bellman (HJB) equation of elliptic or parabolic type. Classical solutions of the HJB equation and verification theorems. Introduction to viscosity solutions of the HJB equation.
3) Backward stochastic differential equations (BSDEs).
Formulation and existence and uniqueness results. Probabilistic representation for solutions of semilinear partial differential equations and for the value function of a stochastic control problem. Stochastic maximum principle in the sense of Pontryagin.
4) Brief overview of other problems and methods.
Markov decision processes: finite horizon, stochastic horizon, infinite horizon. Value functions and related Bellman equations. State-action value functions. Policy evaluation, policy improvement, policy iteration, value iteration. Introduction to model-free algorithms. Monte Carlo and temporal-difference methods.
2) Continuous-time stochastic optimal control.
Controlled stochastic differential equations, cost or reward functionals over a finite or infinite horizon. Value function and the dynamic programming principle. Hamilton-Jacobi-Bellman (HJB) equation of elliptic or parabolic type. Classical solutions of the HJB equation and verification theorems. Introduction to viscosity solutions of the HJB equation.
3) Backward stochastic differential equations (BSDEs).
Formulation and existence and uniqueness results. Probabilistic representation for solutions of semilinear partial differential equations and for the value function of a stochastic control problem. Stochastic maximum principle in the sense of Pontryagin.
4) Brief overview of other problems and methods.
Prerequisites for admission
The course assumes knowledge of the contents of an advanced probability course. Prerequisites also include basic notions of stochastic processes, stochastic integration with respect to Brownian motion, the associated stochastic calculus, and stochastic differential equations driven by Brownian motion.
Teaching methods
In-person lectures. Attendance is not mandatory, but highly recommended.
Teaching Resources
Textbooks:
H. Pham. Continuous-time Stochastic Control and Optimization with Financial Applications. Springer, 2009.
R. S. Sutton, A. G. Barto. Reinforcement Learning: An Introduction. MIT Press, 2018, 2nd ed.
Lecture notes of the teacher will be freely available on the website of the course.
H. Pham. Continuous-time Stochastic Control and Optimization with Financial Applications. Springer, 2009.
R. S. Sutton, A. G. Barto. Reinforcement Learning: An Introduction. MIT Press, 2018, 2nd ed.
Lecture notes of the teacher will be freely available on the website of the course.
Assessment methods and Criteria
The exam consists of an oral examination. During the exam, students will be asked to discuss specific topics from the course syllabus in order to assess their knowledge and understanding of the material covered, as well as their ability to apply it.
MATH-03/B - Probability and Mathematical Statistics - University credits: 6
Exercises: 12 hours
Lessons: 35 hours
Lessons: 35 hours
Professor:
Cosso Andrea
Professor(s)
Reception:
By appointment via email
Department of Mathematics, Via Saldini 50, or via Microsoft Teams