As a part of the strategic funding initiative in the field of AI, the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) will first focus on supporting early career researchers in this field. The DFG invites suitably qualified early career researchers in the field of AI to submit proposals to the Emmy Noether Program. This call is aimed at individuals from a wide range of disciplines who specialise in the field of AI method development. Dr. Felix Dietrich, Chair of Scientific Computing (Prof. Hans-Joachim Bungartz), is among the junior research group leaders who are supported in the frame of the second stage of the review process. His proposal "Harmonic Artificial Intelligence through linear operators" in the field of "Methods of Artificial Intelligence" was approved.
Simulating complex, highly interconnected systems such as the climate, biology, or society typically involve methods from the "traditional" field of scientific computing. These methods are usually reliable and explainable through their foundations in rigorous mathematics. Examples are efficient space discretization schemes such as sparse grids and scalable, parallel solvers for partial differential equations. Unfortunately, most of them are not immediately applicable to the extremely high-dimensional, heterogeneous, and scattered data where neural networks are usually used. Those methods employed in the AI community, on the other hand, are typically not reliable or explainable in the traditional sense, and pose problems illustrated through adversarial examples and brittle generalization results.
Towards the goal of explainable, reliable, and efficient AI, Dr. Felix Dietrich proposes to connect the two worlds of scientific HPC and deep learning, forming a concept he calls "Harmonic AI". Specifically, he will combine linear operator theory and deep learning methods through harmonic analysis. The benefit of a link between AI and linear operators is bi-directional. Inference, classification, and training of neural networks will be understood mostly in terms of linear algebra. This will open the field to much more mathematical rigor and enable more mathematicians to work on AI methods. Simultaneously, applied AI researchers obtain reliable methods that can currently only be found for problems outside the field, such as Finite Element Methods or iterative Newton-Raphson solvers.
The first three years of the project are devoted to explainability of AI by bridging the gap to rigorous mathematics: Leveraging the common principles between the Laplace operator, Gaussian processes, and neural networks, Harmonic AI will connect AI and linear algebra. This will allow the new research group to explain unsupervised data representation algorithms.
The second core objective in this first phase is to bridge the theory between the linear Koopman operator and deep neural networks. The new research group plans to bring ideas from dynamical systems theory to the AI community, to explain the layered neural networks and stochastic optimization algorithms in terms of dynamical evolution operators. The core objective in the second phase (years four to six) is to devise robust and reliable numerical algorithms harnessing the connection between linear operators and neural networks from phase A. The algorithms will be integrated in an HPC environment. Dr. Dietrich and his researchers will aim to release a free and open source software for Harmonic AI to make its algorithms available to a broad research community. Throughout the project, to disseminate, demonstrate, and test the new methods in a proof of concept, they will collaborate with the simulation groups from Prof. Christian Mendl (Quantum Computing Group) studying quantum dynamics and from Prof. Gerta Köster (Department 7 of Computer Science and Mathematics, University of Applied Sciences Munich) studying human crowds.