https://lvb.informatik.uni-bremen.de/ims/03-ims-allm.pdf
Dieser Kurs findet auf Englisch statt/The course is in English.
Prior knowledge:
Basic knowledge of artificial intelligence (AI) and machine learning (ML), as acquired from
one of the introductory and foundational machine learning, AI, or cognitive systems lectures.
Advanced course knowledge, e.g. as acquired in "Life-long Machine Learning" or "Advanced
Machine Learning", is not required, but will be beneficial to engage with more recent in-depth
perspectives at the frontier of current research.
Learning Outcomes:
Traditional machine learning studies the design of models and training algorithms to learn to
solve tasks from data. In contrast to humans, who excel at adaptivity and benefit greatly from
learning curricula, machine learners are largely tailored to stationary data and static test
scenarios. Throughout the seminar, students will engage with state-of-the-art literature that
advances the frontiers of lifelong learning machines. They will improve their ability to
navigate scientific works, critically assess contributions and prospects, inspect the newest
algorithms, and identify persisting limitations and open research questions. In the process,
students will refine their scientific presentation skills and strengthen their competence in
participating in advanced machine learning discourse.
Content:
The seminar will cover different elements required to transform static machine learners into
lifelong learning machines. At the start of the seminar, students will get to select from a
choice of topics spanning a variety of modern perspectives and state-of-the-art lifelong
learning algorithms. In prospective in-depth engagement with the chosen literature as the
starting point, presentations will be prepared and held in front of the seminar participants to
fuel weekly scientific discourse. Discussed advances towards lifelong learning machines
encompass, but are not limited to:
* Memory: what consistutes memorable data points and episodes? How can we construct
concrete algorithms to identify and extract memorable experience? What types of
memories do humans possess and how do machine learning algorithms acommodate
them?
* Learning curricula: how can we identify suitable learning curricula for machine learning?
Which modern algorithms exist to enable models to benefit from structured curricula and
data streams?
* Continual optimization: how can we transcend the greediness of machine learning
optimizers? What algorithmic advances exist to effectively regularize past parameters,
representations, and knowledge?
* Adaptive and modular architectures: how can we dynamically modify opaque
architectures, such as deep neural networks? Which techniques exist to modularize
functions and disambiguate entangled representations?
* Lifelong ML theory: what are the advances in transferring machine learning theory to
lifelong machine learning? What are the insights and obstacles learned from grounded
advances analyzable, linear systems?
* Evaluation: How do we evaluate dimensions with inherent trade-offs, e.g. the pursuit of
stability and plasticity? How do we select hyperparameters in scenarios that are
constantly evolving? What techniques exist to enable fair comparisons across a
plethora of metrics of interest?
