The summer school aims to bring together former and current RTG members as well as invited experts from around the world to discuss and explore topics central to the RTG, including:

• Artificial Intelligence
• Inverse Problems
• Data Science
• Optimization
• Statistics

Beyond the scientific program, this gathering offers an opportunity to reconnect, reflect on shared experiences within the RTG, and explore future research directions. Participants can look forward to an inspiring lineup of talks, engaging discussions, and a conference dinner on the evening of September 25.

For more information please visit the Official Webpage.

The goal of this summer school was to explore topics like Koopman and transfer operator theory, data-driven approximations and kernel analog forecasting and targeted advanced Master students, PhD students as well as young researchers who are interested in the intersection of dynamical systems and machine learning. The participation was free of charge but with prior registration.

For more information please visit the Official Webpage.

The course was targeted to practitioners and application-oriented statisticians interested in machine learning. It started with a (brief) introduction to machine learning and covered a broad range of interpretability methods such as partial dependence plots, feature importance and Shapley values. The workshop was splitted into theory sessions, i.e. lectures, and practical sessions in R.

For more information on the course, visit the corresponding webpage.

The digital award ceremony took place on 16 March, you can find more information here. The award winners of the last years can be found here.

Louisa continues her research work in our working group within the Helmholtz School for Marine Data Science with the PhD project "MarDATA - Monitoring a stressed ice shelf - Machine learning algorithms to detect icequakes in 20 years of seismological records at Neumayer station, antarctica" in cooperation with the Alfred Wegener Institute.

Abstract

This thesis’s primary focus is on the application of deep learning methods to inverse problems. More specifically, it tries to advance the application of inverse problems for cases where straight forward supervised training is not feasible due to a lack of training data.

The thesis begins by introducing basic concepts concerning inverse problems, deep learning, and the latter’s application to the former. The second part of the thesis presents five papers to which the author contributed.

While the first of the papers aims to advance our understanding of neural networks by analyzing changes in properties of the data propagating through networks, the latter four are concerned with the direct application of deep learning methods to inverse problems. Two of these four papers – one more theoretical, the other focusing on magnetic particle imaging – investigate the concept of regularization by architecture, otherwise known as the deep image prior approach. Regularization by architecture is a deep learning paradigm for reconstruction that does not require any training data at all. Of the other two papers, one is concerned with training a denoiser, which, even during training, does not require any kind of ground truth data. The last of the five papers introduces a learned strongly convex penalty term based on an input-convex neural network architecture. The strong convexity allows us to formally show regularizing properties.
 

Abstract

This thesis is a compound of various works of the author and coworkers on the application of neural networks and deep learning for solving inverse problems.

The first application is in Materials Science. The development of new structural materials with desirable properties has become one of the most challenging tasks for engineers. We use Neural Networks to learn how the parameters influence the material properties and focus mainly on solving the corresponding inverse problem. Given desired properties, a material should have, we aim at finding the production parameters we need to obtain it.

The second application is in Computed Tomography (CT), which is one of the most valuable technologies in modern medical imaging. It allows a non-invasive acquisition of the inside of the human body using X-rays. In this work, we examine the application of deep learning methods for the reconstruction of CT images in the context of a low-data regime. Firstly, we review the current challenges and existing approaches and obtain quantitative results after training them with different amounts of data. We find that the learned primal-dual method has an outstanding performance in terms of reconstruction quality and data efficiency. However, in general, end-to-end learned techniques have two issues: a) lack of classical guarantees in inverse problems and b) lack of generalization when not trained with enough data. To overcome these issues, we bring in the Deep Image Prior (DIP) approach in combination with classical regularization. The proposed methods improve the state-of-the-art results in the low data-regime.

Finally, we analyze specific network designs and discuss different interpretations of the DIP approach. We obtain theoretical results by introducing the idea of viewing these approaches as the optimization of Tikhonov functionals rather than optimizing net\-works. Additionally, we provide numerical examples that verify our findings.

Abstract

This thesis is concerned with works in connection to double backpropagation, which is a phenomenon that arises when first-order optimization methods are applied to a neural network’s loss function, if this contains derivatives. Since feedforward neural networks are constructed in a layerwise fashion, the successive application of the chain rule throughout the layers of these networks yields the desired derivatives according to the famous backpropagation procedure. If these derivatives in turn appear in the form of a loss function, training the neural network results in said double backpropagation. In this thesis, an extensive analysis of the properties of double backpropagation is performed. This includes the calculation of the gradients themselves, for whose coordinateindependent representation in Hilbert spaces a theory of adjoints of bilinear operators is developed. The explicit calculation of the weight gradients allows for a reduction in computational complexity by roughly a third for a common special case. Furthermore, empirical results are presented which demonstrate a ’pseudo-smoothing’ effect on this loss landscape, when using the popular rectified linear units in combination with batch optimization. From an application-perspective, double backpropagation can be used for reducing a neural network’s vulnerability to adversarial attacks. Such an increase in adversarial robustness has been shown to improve the structure of saliency maps, i.e. gradients indicating the discriminative portions of an input image. This work offers an explanation of this so far unexplained phenomenon by considering the alignment between an input image and its saliency map. These findings are verified for networks robustified with double backpropagation. Tumor typing of imaging mass spectrometry data is an active area of research, which aims to determine the correct type of tumor of a patient’s cancerous tissue obtained during surgery. While ’classical’ methods from machine learning have been successfully applied to this problem, in this thesis a neural network approach is presented, for which a task-adapted architecture called IsotopeNet is developed. This architecture beats both a classical baseline as well as a more standard neural network architecture on two challenging datasets. This approach however yields unsatisfactory accuracies on a multilaboratory study. Using an attribution method called layerwise relevance propagation, the reason for this failure is determined to stem from measurement artifacts induced by the multi-laboratory setting. By penalizing this layerwise relevance propagation with a sparsity-inducing penalty term (a novel method which is named deep relevance regularization), the performance of the neural network approach is greatly improved.

For mor information visit Dr. Lisa Horvath's webpage.

It is well known that certain properties of the pitchfork bifurcation are destroyed when noise is added (see Crauel and Flandoli, Additive noise destroys a pitchfork bifurcation, Journal of Dynamics and Differential Equations, 1998). In this talk, I would like to present three different points of view that demonstrate that the bifurcation is still present. The first two approaches concern a finite-time and local analysis of the (unbounded noise) random dynamical system, revealing structural changes at the bifurcation point that can be described by the dichotomy spectrum and conditional Lyapunov exponents. In the last part of the talk, we study the situation for bounded noise, and show that the pitchfork bifurcation can be described by means of a discontinuous bifurcation of minimal invariant sets. Joint work with Mark Callaway, Thai Son Doan, Maximilian Engel, Jeroen Lamb, and Christian Rodrigues.