Genomic big data management and computing

Many projects in the genomics field rely on increasingly large data sets, analyzing, for example, genomes of thousands of individuals affected by a particular disease. It is paramount to understand how large data sets can be managed and processed in an efficient way and how next-generation sequencing processing pipelines and workflows can be used to benefit such large-scale projects.

The objective of the course is to illustrate and discuss key aspects regarding the management, processing and analysis of big data for genomics (mainly data obtained by Next-Generation Sequencing), as well as introduce some of the existing approaches, analysis systems and technologies used. Practical applications will be illustrated using both dedicated programming and query languages (PySpark, GMQL), and specific computational platforms and distributed systems (Galaxy, Apache Spark, Cloud Computing). Also "downstream" analysis examples to underscore the necessity of big data management and computing in genomics will be illustrated.

Given the vastness of the topics presented, the ultimate goal of the course is not an in-depth knowledge of specific data analysis approaches, but rather to provide a broad overview of different solutions paired with the understanding of strengths and weaknesses of different methodologies and computing environments for managing scientific workflows used for big data analysis in the field of genomics.