Multimedia Systems and Interaction Design
A.Y. 2025/2026
Learning objectives
The course aims to provide students with advanced knowledge of
interactive multimedia systems, with a focus on modeling, management,
search, and use of multimedia content in complex and data-driven contexts.
Upon completion of the course, students will be able to:
- understand and design interactive multimedia systems based on advanced
data models;
- apply Information Retrieval and Multimedia Retrieval techniques,
including vector representations and semantic embeddings;
- integrate generative Artificial Intelligence technologies (Large
Language Models, Retrieval-Augmented Generation, conversational agents)
in a controlled and reliable manner;
- evaluate the user experience (UX) and technological acceptance of
multimedia and AI-driven systems;
- analyze the architectural and real-time constraints (QoS and QoE) that
influence the quality of interaction.
The course provides adequate theoretical and methodological preparation
for access to advanced research and industrial development contexts in
multimedia systems and interactive AI.
interactive multimedia systems, with a focus on modeling, management,
search, and use of multimedia content in complex and data-driven contexts.
Upon completion of the course, students will be able to:
- understand and design interactive multimedia systems based on advanced
data models;
- apply Information Retrieval and Multimedia Retrieval techniques,
including vector representations and semantic embeddings;
- integrate generative Artificial Intelligence technologies (Large
Language Models, Retrieval-Augmented Generation, conversational agents)
in a controlled and reliable manner;
- evaluate the user experience (UX) and technological acceptance of
multimedia and AI-driven systems;
- analyze the architectural and real-time constraints (QoS and QoE) that
influence the quality of interaction.
The course provides adequate theoretical and methodological preparation
for access to advanced research and industrial development contexts in
multimedia systems and interactive AI.
Expected learning outcomes
Knowledge and Understanding.
The student will acquire in-depth knowledge of:
- data models for multimedia content;
- multimedia retrieval and multimodal retrieval architectures;
- text, image, audio, and video retrieval techniques;
- generative AI systems applied to interaction;
- user experience evaluation methodologies.
Ability to apply knowledge and understanding.
The student will be able to:
- design systems for searching and accessing multimedia content;
- use semantic representations and embeddings for retrieval;
- develop RAG pipelines and AI-based conversational systems;
- critically evaluate alternative technological solutions.
Making independent judgments.
The student will develop the following skills:
- critically analyze multimedia and AI-driven architectures;
- evaluate the trade-offs between technical quality and perceived quality;
- Evaluate the reliability, usability, and acceptability of designed
systems.
Communication skills.
The student will be able to:
- clearly and rigorously describe architectures and design solutions;
- discuss technological choices using appropriate technical language;
- present design and experimental results.
Learning skills.
The student will acquire the conceptual tools to:
- independently update themselves on emerging technologies in the field;
- deepen research topics in multimedia, IR, and interactive AI.
The student will acquire in-depth knowledge of:
- data models for multimedia content;
- multimedia retrieval and multimodal retrieval architectures;
- text, image, audio, and video retrieval techniques;
- generative AI systems applied to interaction;
- user experience evaluation methodologies.
Ability to apply knowledge and understanding.
The student will be able to:
- design systems for searching and accessing multimedia content;
- use semantic representations and embeddings for retrieval;
- develop RAG pipelines and AI-based conversational systems;
- critically evaluate alternative technological solutions.
Making independent judgments.
The student will develop the following skills:
- critically analyze multimedia and AI-driven architectures;
- evaluate the trade-offs between technical quality and perceived quality;
- Evaluate the reliability, usability, and acceptability of designed
systems.
Communication skills.
The student will be able to:
- clearly and rigorously describe architectures and design solutions;
- discuss technological choices using appropriate technical language;
- present design and experimental results.
Learning skills.
The student will acquire the conceptual tools to:
- independently update themselves on emerging technologies in the field;
- deepen research topics in multimedia, IR, and interactive AI.
Lesson period: Second four month period
Assessment methods: Esame
Assessment result: voto verbalizzato in trentesimi
Single course
This course can be attended as a single course.
Course syllabus and organization
Single session
Responsible
Lesson period
Second four month period
Course syllabus
General Introduction
- Multimedia and digital communication: definitions, application domains, trends
- Recap of Interaction Design fundamentals: principles, processes, mental models, affordance, and mapping
Design of Interactive Systems for Multimedia Content
- Multimedia data modeling
- Object-Relational Database Management Systems (ORDBMS)
- Multimedia extensions in Oracle (e.g., Oracle Multimedia)
Management of Digital Texts
- Information Retrieval models: boolean, vector space, probabilistic
- Text IR vs relational databases
- Oracle Text and IR systems such as Solr
Management of Digital Images
- Visual perception and color models (RGB, CMYK, HSV)
- Formats and standards for digital images: GIF, PNG, TIFF, JPEG
- Image retrieval: from low-level content to semantic representations
- Image IR techniques: Query by Sample Example (QBSE), MediaPipe Image Embedders, OpenAI CLIP
Management of Audio Content
- Sampling, quantization, signal-to-noise ratio
- Standards and compression: WAV, AIFF, MP3, AAC
- Psychoacoustics: critical bands, spatial and temporal masking
Management of Video Content
- Color models for video, compression techniques (predictive coding, motion vectors)
- Video standards: MPEG, H.264
- Integrated audio-video management and IR with Oracle Multimedia
Multimedia Information Systems and Metadata
- Standards for content description: MPEG-7
- Metadata languages, ontologies, and semantic enrichment
Interactive Systems and Generative AI
- Design of Question Answering systems
- Designing with Generative AI
- Retrieval-Augmented Generation (RAG): architecture and applications
- Prompt engineering for optimizing generative experiences
Conversational Experience Design
- Conversation models and design of dialog flows
- Building conversational agents with DialogFlow and LLM-based chatbots
- Criteria for coherence, context-awareness, personalization, and engagement
Architectures and Interaction in Real-Time Environments
- Operating systems and challenges for real-time interactive media
- Synchronization and temporal management of multimedia data
- Quality of Service: technical parameters and perceived quality
- Scheduling and network protocols for real-time multimedia (e.g., RTP, RTCP)
User Experience Evaluation Methods
- Usability, accessibility, and UX in multimedia systems
- Empirical and analytical evaluation techniques: Quantitative measures: SUS (System Usability Scale), UTAUT, CTS, NASA-TLX and Qualitative feedback
- Evaluation of generative AI integration in multimedia projects
Analysis of Technology Adoption and Acceptance Potential
- Project-based and lab activities
- Hands-on sessions with advanced tools and technologies (e.g., Oracle Text, DialogFlow, MediaPipe, LLM-based tools)
- Case study analysis of real-world applications
- Multimedia and digital communication: definitions, application domains, trends
- Recap of Interaction Design fundamentals: principles, processes, mental models, affordance, and mapping
Design of Interactive Systems for Multimedia Content
- Multimedia data modeling
- Object-Relational Database Management Systems (ORDBMS)
- Multimedia extensions in Oracle (e.g., Oracle Multimedia)
Management of Digital Texts
- Information Retrieval models: boolean, vector space, probabilistic
- Text IR vs relational databases
- Oracle Text and IR systems such as Solr
Management of Digital Images
- Visual perception and color models (RGB, CMYK, HSV)
- Formats and standards for digital images: GIF, PNG, TIFF, JPEG
- Image retrieval: from low-level content to semantic representations
- Image IR techniques: Query by Sample Example (QBSE), MediaPipe Image Embedders, OpenAI CLIP
Management of Audio Content
- Sampling, quantization, signal-to-noise ratio
- Standards and compression: WAV, AIFF, MP3, AAC
- Psychoacoustics: critical bands, spatial and temporal masking
Management of Video Content
- Color models for video, compression techniques (predictive coding, motion vectors)
- Video standards: MPEG, H.264
- Integrated audio-video management and IR with Oracle Multimedia
Multimedia Information Systems and Metadata
- Standards for content description: MPEG-7
- Metadata languages, ontologies, and semantic enrichment
Interactive Systems and Generative AI
- Design of Question Answering systems
- Designing with Generative AI
- Retrieval-Augmented Generation (RAG): architecture and applications
- Prompt engineering for optimizing generative experiences
Conversational Experience Design
- Conversation models and design of dialog flows
- Building conversational agents with DialogFlow and LLM-based chatbots
- Criteria for coherence, context-awareness, personalization, and engagement
Architectures and Interaction in Real-Time Environments
- Operating systems and challenges for real-time interactive media
- Synchronization and temporal management of multimedia data
- Quality of Service: technical parameters and perceived quality
- Scheduling and network protocols for real-time multimedia (e.g., RTP, RTCP)
User Experience Evaluation Methods
- Usability, accessibility, and UX in multimedia systems
- Empirical and analytical evaluation techniques: Quantitative measures: SUS (System Usability Scale), UTAUT, CTS, NASA-TLX and Qualitative feedback
- Evaluation of generative AI integration in multimedia projects
Analysis of Technology Adoption and Acceptance Potential
- Project-based and lab activities
- Hands-on sessions with advanced tools and technologies (e.g., Oracle Text, DialogFlow, MediaPipe, LLM-based tools)
- Case study analysis of real-world applications
Prerequisites for admission
It is advisable but not mandatory to have basic knowledge of subjects such as databases and Human-Computer Interaction.
Teaching methods
Lectures with discussions and presentations of case studies
Teaching Resources
Slides of the lectures in MyARIEL
In the slides are reported books, articles websites useful both for studying and insight.
Basic Readings:
- Z. Li, M. Drew. Fundamentals of Multimedia, Pearson Educational, 2004
- Multimedia Applications. In B. Fuhrt (ed), Handbook of Internet Computing, CRC Press, 2000
In the slides are reported books, articles websites useful both for studying and insight.
Basic Readings:
- Z. Li, M. Drew. Fundamentals of Multimedia, Pearson Educational, 2004
- Multimedia Applications. In B. Fuhrt (ed), Handbook of Internet Computing, CRC Press, 2000
Assessment methods and Criteria
Methods of verification are:
1. Written test (max grade 30/30)
2. Oral test, Thematic Deepening, Project (optional) that will result in an increase or decrease in the grade of the written test by + or - 3 points
The evaluation criteria of the written test concern:
- Knowledge of the content
- Adherence of the response to the track
- Ability to make connections
- The relevance of the contents in relation to the question
The evaluation criteria of the oral test/thematic deepening/project concern:
- the focus of the issue
- logical rigour and use of technical language
- fairness and clarity of exposure
- The critical awareness, personal interpretation skills, richness and relevance of the oral presentation
1. Written test (max grade 30/30)
2. Oral test, Thematic Deepening, Project (optional) that will result in an increase or decrease in the grade of the written test by + or - 3 points
The evaluation criteria of the written test concern:
- Knowledge of the content
- Adherence of the response to the track
- Ability to make connections
- The relevance of the contents in relation to the question
The evaluation criteria of the oral test/thematic deepening/project concern:
- the focus of the issue
- logical rigour and use of technical language
- fairness and clarity of exposure
- The critical awareness, personal interpretation skills, richness and relevance of the oral presentation
Educational website(s)
Professor(s)