Preface

Welcome to 4C16. When this module began in 2017, the shockwave from the AlexNet paper four years prior was still reverberating through the research community. We were the first to offer a dedicated Deep Learning module in Ireland, driven by a sense of urgency to keep pace with what looked liked an ongoing revolution.

The essence of Deep Learning is relatively easy to comprehend. I have a slide deck for introducing Deep Learning in 5 minutes, another for Convolutional Neural Networks in 5 minutes, and a third for Large Language Models, also in 5 minutes. In less than half an hour, a secondary school student could realistically grasp its core mechanisms. But we need to delve deeper in its foundations.

Thus, from its inception, our philosophy for 4C16 was clear: to train practitioners with a deep understanding of both the mathematical foundations and the practical application of Deep Learning. We knew our students would come from diverse backgrounds—some with stronger coding skills, some more confortable with the mathematical foundations; the challenge was to create a curriculum that could suit everyone.

To bridge the gap between theory and practice for everyone, we engineered our own solution: a sophisticated, in-house learning web platform. This system provides students with seamless access to the Google Cloud Platform (now Colab), through a web-based terminal and Jupyter environment. The labs, built on the Keras framework, are automatically assessed via Git, providing instant, formative feedback. This platform was our answer to the challenge of scalability and quality, and nearly a decade later, it remains the robust bedrock of our teaching and remains without equal.

Of course, both the field and the module have evolved significantly. Looking back at the first year’s introductory handout, my focus was on stressing that this was a revolution. In 2017, some were still doubtful, viewing Deep Learning as just another fad that would be superseded by the next shiny Machine Learning method. But even then, the signs of a profound paradigm shift were visible, and I felt they needed to be explicitly stated. Today, the importance of the topic needs no justification, and the same slides now serve as a brief historical reminder.

As Deep Learning has advanced, so has 4C16. In 2020, the module expanded from 5 to 10 ECTS, and the curriculum has progressively integrated critical new topics, including Variational Autoencoders, Transformers, Large Language Models. The emergence of generative AI, has not only become a topic of study but has also impacted the module delivery itself.

Today, the field of Deep Learning has reached a certain maturity. It may no longer be the raw, unexplored frontier it was in 2017, but it has become a vast and indispensable territory in science and engineering. Its mathematical principles draw from statistics, signal processing, computer vision, and linguistics, making it a truly interdisciplinary pursuit. This module, 4C16, has matured alongside it and now serves as an essential foundation, with more advanced topics branching into specialised modules like EEP55C34.

François Pitié