Institute of Applied Analysis
University Ulm
Helmholtzstraße 18
89081 Ulm
Room E.10
Tel.: +49 731 50-23560
andre.schlichting@uni-ulm.de
News and Updates
- Preprint: Singular-limit analysis of gradient descent with noise injection
Together with Anna Shalova and Mark Peletier, we study the limiting dynamics of a large class of noisy gradient descent systems in the overparameterized regime. In this regime the set of global minimizers of the loss is large, and when initialized in a neighbourhood of this zero-loss set a noisy gradient descent algorithm slowly evolves along this set. In some cases this slow evolution has been related to better generalisation properties. We characterize this evolution for the broad class of noisy gradient descent systems in the limit of small step size. Our results show that the structure of the noise affects not just the form of the limiting process, but also the time scale at which the evolution takes place. We apply the theory to Dropout, label noise and classical SGD (minibatching) noise, and show that these evolve on different two time scales. Classical SGD even yields a trivial evolution on both time scales, implying that additional noise is required for regularization. The results are inspired by the training of neural networks, but the theorems apply to noisy gradient descent of any loss that has a non-trivial zero-loss set.
The preprint is arXiv:2404.12293. - Preprint: Variational convergence of exchange-driven stochastic particle systems in the thermodynamic limit
Together with Chun Yin Lam we consider the thermodynamic limit of mean-field stochastic particle systems on a complete graph. The evolution of occupation number at each vertex is driven by particle exchange with its rate depending on the population of the starting vertex and the destination vertex, including zero-range and misanthrope process. We show that under a detailed balance condition and suitable growth assumptions on the exchange rate, the evolution equation of the law of the particle density can be seen as a generalised gradient flow equation related to the large deviation rate functional.
We show the variational convergence of the gradient structures based on the energy dissipation principle, which coincides with the large deviation rate function of the finite system. The convergence of the system in this variational sense is established based on compactness of the density and flux and Γ-lower-semicontinuity of the energy dissipation functional along solutions to the continuity equation. The driving free energy Γ-converges in the thermodynamic limit, after taking possible condensation phenomena into account.
The preprint is on arXiv:2401.06696. - Preprint: Modelling and numerical analysis for a durotaxis model in “Piezo1-induced durotaxis of pancreatic stellate cells depends on TRPC1 and TRPV4 channels”
In this interdisciplinary project with a group from the Institute of Physiology II in Münster and the Department of Medicine at Duke, together with Angela Stevens we contributed a mathematical model and numerical analysis for a phenomenological model of durotaxis with mechanosensitive ion channels.
For details see the preprint bioRxiv:2023.12.22.572956 and the supplementary material about the numerics. - Preprint: A structure preserving discretization for the Derrida-Lebowitz-Speer-Spohn equation based on diffusive transport:
Together with Daniel Matthes, Eva-Maria Rott and Giuseppe Savaré we propose a spatial discretization of the fourth-order nonlinear DLSS equation on the circle. Our choice of discretization is motivated by a novel gradient flow formulation with respect to a metric that generalizes martingale transport. The discrete dynamics inherits this gradient flow structure, and in addition further properties, such as an alternative gradient flow formulation in the Wasserstein distance, contractivity in the Hellinger distance, and monotonicity of several Lypunov functionals. Our main result is the convergence in the limit of vanishing mesh size. The proof relies an a discrete version of a nonlinear functional inequality between integral expressions involving second order derivatives. arXiv:2312.13284
- Conference announcement: PDE & Probability in interaction: functional inequalities, optimal transport and particle systems
Together with Pierre Monmarché (Sorbonne Université), Julien Reygner (École des Ponts ParisTech), and Marielle Simon (Université de Lille), we are delighted to announce the upcoming workshop “PDE & Probability in interaction: functional inequalities, optimal transport and particle systems”.
The event will be held from January 22 to 26, 2024, at CIRM in Marseille.
Registrations are now open on the website: https://conferences.cirm-math.fr/2988.html
This workshop will also feature two courses, delivered by J. LEHEC (Université de Poitiers) and M. GOLDMAN (Université Paris Cité) on functional inequalities in high dimensions and random matching problems, respectively.
Invited talks will be given by:
Nathalie Ayi (Sorbonne Université)
Roland Bauerschmidt* (University of Cambridge)
Maria Bruna (University of Cambridge)
Kleber Carrapotoso (École Polytechnique, Palaiseau)
Giovanni Conforti (École Polytechnique, Palaiseau)
Alex Delalande (Lagrange Center, Paris)
François Delarue (Université Côte d’Azur)
Alex Dunlap (NYU Courant)
Rishabh Gvalani (MPI MIS Leipzig)
Martin Huesmann (University of Münster)
Jean-Francois Mehdi Jabir (HSE Moscow)
Jasper Hoeksema (TU Eindhoven)
Bo’az Klartag (Weizmann Institute of Science)
Daniel Lacker (Columbia University)
Jean-Christophe Mourrat (ENS Lyon)
Emanuela Radici (University of L’Aquila)
Milica Tomasevic (École Polytechnique, Palaiseau)
Dario Trevisan (Pisa University)
Isabelle Tristani (ENS Paris)
Haava Yoldas (TU Delft)