Natural Interaction

A.Y. 2021/2022
Overall hours
Learning objectives
The course describes and analyses theory and techniques for the design of artificial agents that are able to:
- perceive the user's behavioural signals, via contactless sensors (Kinect,Leap motion, video camere eye-trackers, etc) and extract their most relevant cues
- infer and predict, from such cues in a given context, user's behavioural state
- provide the appropriate feedback to the user
To such end, the course concerns: a rigorous introduction to the neurobiological and psychological models of natural interaction through gestures, postures and gaze; statistical machine learning and inference for modelling the dynamics of interaction
Expected learning outcomes
Upon completion of the course students will be able to:
1. Define the methodology and the most appropriate techniques for modelling interaction agents dealing with uncertainty
2. Measure and analyse behavioural signals, and extract their most informative cues
3. Design and implement simple interaction agents to be exploited in different applications such as smart living ambients, behavioural biometrics, entertainment, video surveillance.
These objectives are measured via a combination of three components: the project realisation, the project technical report and the oral discussion. The final grade is formed by assessing the software developed, the project report, and then using the oral discussion for fine tuning.
Course syllabus and organization

Single session

Lesson period
Second semester
Lectures will be given either in asynchronous or synchronous mode (or hybrid mode, depending on the epidemiological scenario) via Zoom video conference system. In all modalities, lectures will be recorded and made available on the Ariel web site of the course
Course syllabus
The course aims at providing the methodological background to address the modelling problem for detecting and understanding human behavioural signals as captured by a variety of sensors (cameras, microphones, Kinect, Leap motion, etc.). Such background involves both the psychological/neurobiological bases of non verbal behaviours and probabilistic machine-learning methods to set up computational models of behavioural/social signal understanding.
The course articulates in two parts:
Part 1 Neurobiological and psychological models of nonverbal behaviour
- Gaze interaction
- Gesture interaction
- Posture interaction
Part 2 Probabilistic models and machine-learning techniques for the analysis of nonverbal behaviour
- Probabilistic models: inference and learning
- Supervised modelling: classic techniques and deep learning
- Unsupervised modelling: classic techniques and deep learning
Practical examples and applications
Prerequisites for admission
Students are strongly encouraged to preliminarily attend the courses of Statistical methods for machine learning, Artificial Intelligence and Computer Vision. Python programming capabilities are welcomed for project work.
Teaching methods
Lectures on general aspects of natural interaction and, while developing the projects, hands-on activities within the PHUSE Lab
Teaching Resources
Lectures notes, live notes and slides, books, chapters, papers, videos and code are available on the course web site hosted by Ariel (
Assessment methods and Criteria
The examination concerns the development of a project and an oral discussion. The project is assigned based on ongoing natural interaction research within the Perceptual computing and HUman SEnsing Laboratory (PHUSE Lab).
The project involves the realisation, and the experimental validation of a prototype addressing a specific case study. At the end of the project, the student should release a scientific report (illustrating the problem, results achieved, bibliography) and the code implementing the prototype.
The oral discussion aims at critically assessing project's motivation and results so far obtained, and the capability of the student to frame the latter within the theoretical topics presented in lectures.
INF/01 - INFORMATICS - University credits: 6
Lessons: 48 hours
Educational website(s)