Advanced Probability
A.Y. 2026/2027
Learning objectives
The course provides advanced training on two fundamental classes of discrete-time stochastic processes, martingales and Markov chains, and introduces some basic tools for the passage to continuous time. The main theoretical results will be developed together with selected applications in analysis, statistical inference, data science, optimization and stochastic algorithms.
Expected learning outcomes
At the end of the course, students will know and be able to use the main results of the theory of martingales and Markov chains, understanding their relevance from both a theoretical and an applied perspective. They will also acquire an initial familiarity with the passage to continuous time through the notion of weak convergence on path space.
Lesson period: First semester
Assessment methods: Esame
Assessment result: voto verbalizzato in trentesimi
Single course
This course cannot be attended as a single course. Please check our list of single courses to find the ones available for enrolment.
Course syllabus and organization
Single session
Responsible
Lesson period
First semester
Course syllabus
1. Conditional expectation and discrete-time martingales
· Conditional expectation
· Martingales: fundamental examples, stopping times and stopped processes, Doob decomposition, predictable transforms, optional stopping theorems, Doob maximal inequalities, upcrossing inequality, almost sure and L^1, L^2 convergence
2. Applications of martingales, chosen from the following:
· Radon-Nikodym theorem via martingales
· Optimal stopping problems and the Snell envelope
· Ergodic theorem and strong laws via martingale techniques
· Concentration inequalities for martingale differences and examples from balls-into-bins, hashing, and randomized rounding
· Brief introduction to Robbins-Monro stochastic approximation and stochastic gradient descent
3. Markov chains
· Structure of Markov chains: transition kernel, Chapman-Kolmogorov equations, stochastic matrices, and transition distributions
· Classification of states: accessibility and communication, communicating classes, recurrence and transience, periodicity and irreducibility, hitting times and return times
· Asymptotic behavior: invariant and stationary distributions, existence and uniqueness, ergodicity, reversibility and the detailed balance condition, brief discussion of convergence to equilibrium, coupling, and mixing times
· Brief introduction to continuous-time Markov chains
4. Applications of Markov chains, chosen from the following:
· Random walks on graphs
· Markovian algorithms for statistical inference, data science, and machine learning: Metropolis-Hastings and Gibbs sampler
· Hidden Markov models
· Possible brief discussion of Markov decision processes, Bellman recursion, and reinforcement learning
5. Transition from discrete time to continuous time:
· Weak convergence of measures: equivalent formulations, Portmanteau theorem
· Tightness: relative compactness, brief discussion of Prokhorov's theorem
· Convergence of processes: path spaces, Kolmogorov existence theorem, weak convergence of processes
· Donsker's principle: rescaled random walk, convergence to Brownian motion, interpretation as a functional limit theorem
· Conditional expectation
· Martingales: fundamental examples, stopping times and stopped processes, Doob decomposition, predictable transforms, optional stopping theorems, Doob maximal inequalities, upcrossing inequality, almost sure and L^1, L^2 convergence
2. Applications of martingales, chosen from the following:
· Radon-Nikodym theorem via martingales
· Optimal stopping problems and the Snell envelope
· Ergodic theorem and strong laws via martingale techniques
· Concentration inequalities for martingale differences and examples from balls-into-bins, hashing, and randomized rounding
· Brief introduction to Robbins-Monro stochastic approximation and stochastic gradient descent
3. Markov chains
· Structure of Markov chains: transition kernel, Chapman-Kolmogorov equations, stochastic matrices, and transition distributions
· Classification of states: accessibility and communication, communicating classes, recurrence and transience, periodicity and irreducibility, hitting times and return times
· Asymptotic behavior: invariant and stationary distributions, existence and uniqueness, ergodicity, reversibility and the detailed balance condition, brief discussion of convergence to equilibrium, coupling, and mixing times
· Brief introduction to continuous-time Markov chains
4. Applications of Markov chains, chosen from the following:
· Random walks on graphs
· Markovian algorithms for statistical inference, data science, and machine learning: Metropolis-Hastings and Gibbs sampler
· Hidden Markov models
· Possible brief discussion of Markov decision processes, Bellman recursion, and reinforcement learning
5. Transition from discrete time to continuous time:
· Weak convergence of measures: equivalent formulations, Portmanteau theorem
· Tightness: relative compactness, brief discussion of Prokhorov's theorem
· Convergence of processes: path spaces, Kolmogorov existence theorem, weak convergence of processes
· Donsker's principle: rescaled random walk, convergence to Brownian motion, interpretation as a functional limit theorem
Prerequisites for admission
Basic probability at the level of an introductory course: construction of a probability space, random variables and random vectors, induced measures, main notions of convergence, law of large numbers, central limit theorem.
Teaching methods
The course consists of 42 hours of lectures and 36 hours of exercise classes. Part of the exercise classes will be devoted to examples, case studies and discussion of applications.
Teaching Resources
The main references are:
- D. Williams, Probability with Martingales.
- J. R. Norris, Markov Chains.
- P. Billingsley, Convergence of Probability Measures.
- P. Billingsley, Probability and Measure.ù
Other refererences will be given during the lectures.
- D. Williams, Probability with Martingales.
- J. R. Norris, Markov Chains.
- P. Billingsley, Convergence of Probability Measures.
- P. Billingsley, Probability and Measure.ù
Other refererences will be given during the lectures.
Assessment methods and Criteria
· a short written part consisting of exercises
· an oral part devoted to assessing the understanding of the theoretical results, their proofs, and their applications.
The final grade takes both parts into account and is expressed on a scale of 30.
· an oral part devoted to assessing the understanding of the theoretical results, their proofs, and their applications.
The final grade takes both parts into account and is expressed on a scale of 30.
MATH-03/B - Probability and Mathematical Statistics - University credits: 9
Exercises: 36 hours
Lessons: 42 hours
Lessons: 42 hours
Professors:
Campi Luciano, Ugolini Stefania
Professor(s)
Reception:
Please write an email
Room of the teacher or online room