A new arXiv paper unifies a decade of scattered tricks for training neural networks on curved, non-Euclidean data into one reusable toolkit.
The thesis generalizes batch normalization, previously customized per manifold, to broad classes of Lie groups and gyrogroups. It extends multinomial logistic regression from ordinary Euclidean space to SPD (symmetric positive definite) manifolds and then to general Riemannian manifolds. It also builds new networks for specific geometric representations: an unconstrained model of hyperbolic space, a Busemann-based approach to hyperbolic learning, and networks for full-rank correlation matrices. Finally, it introduces adaptive Riemannian metrics on SPD manifolds, including learnable Log-Euclidean geometries and faster, more numerically stable Cholesky-based geometries.
Manifold-valued data shows up constantly in real applications: covariance matrices in signal processing, correlation matrices in genomics, and hierarchical structures better suited to hyperbolic space than flat Euclidean embeddings. Until now, using these geometries in deep learning meant either crude Euclidean approximations or bespoke, numerically fragile operations rebuilt for every manifold. Reusable modules that work across geometries could cut the engineering overhead that has kept Riemannian deep learning a niche pursuit.
It is still a thesis, not a shipped library, and the abstract offers no benchmark numbers against strong Euclidean baselines - so whether these generalized modules actually beat the manifold-specific ones they replace remains an open question.