recent publications

1. Physics-Informed Neural Compression of High-Dimensional Plasma Data

   Galletti, G., Cranganore, G., Hornsby, W., Zanisi, L., Carey, N., Pamela, S., Brandstetter, J., and Paischer, F.

   2026

   url Blog Code
2. MHNfs: Prompting In-Context Bioactivity Predictions for Low-Data Drug Discovery

   Schimunek, J., Luukkonen, S., and Klambauer, G.

   Journal of Chemical Information and Modeling 2025
3. Bio-xLSTM: Generative modeling, representation and in-context learning of biological and chemical sequences

   Schmidinger, N., Schneckenreiter, L., Seidl, P., Schimunek, J., Hoedt, P., Brandstetter, J., Mayr, A., Luukkonen, S., Hochreiter, S., and Klambauer, G.

   In The Thirteenth International Conference on Learning Representations 2025

   url
4. Parameter Efficient Fine-tuning via Explained Variance Adaptation

   Paischer, F., Hauzenberger, L., Schmied, T., Alkin, B., Deisenroth, M., and Hochreiter, S.

   In The Thirty-ninth Annual Conference on Neural Information Processing Systems 2025

   url Code
5. NeurIPS

   GyroSwin: 5D Surrogates for Gyrokinetic Plasma Turbulence Simulations

   Paischer, F., Galletti, G., Hornsby, W., Setinek, P., Zanisi, L., Carey, N., Pamela, S., and Brandstetter, J.

   In 2025

   Abs url Blog Code

   Nuclear fusion plays a pivotal role in the quest for reliable and sustainable energy production. A major roadblock to viable fusion power is understanding plasma turbulence, which significantly impairs plasma confinement, and is vital for next-generation reactor design. Plasma turbulence is governed by the nonlinear gyrokinetic equation, which evolves a 5D distribution function over time. Due to its high computational cost, reduced-order models are often employed in practice to approximate turbulent transport of energy. However, they omit nonlinear effects unique to the full 5D dynamics. To tackle this, we introduce GyroSwin, the first scalable 5D neural surrogate that can model 5D nonlinear gyrokinetic simulations, thereby capturing the physical phenomena neglected by reduced models, while providing accurate estimates of turbulent heat transport. GyroSwin (i) extends hierarchical Vision Transformers to 5D, (ii) introduces cross-attention and integration modules for latent 3D\leftrightarrow5D interactions between electrostatic potential fields and the distribution function, and (iii) performs channelwise mode separation inspired by nonlinear physics. We demonstrate that GyroSwin outperforms widely used reduced numerics on heat flux prediction, captures the turbulent energy cascade, and reduces the cost of fully resolved nonlinear gyrokinetics by three orders of magnitude while remaining physically verifiable. GyroSwin shows promising scaling laws, tested up to one billion parameters, paving the way for scalable neural surrogates for gyrokinetic simulations of plasma turbulence.

6. Universal Physics Transformers

   Alkin, B., Fürst, A., Schmid, S., Gruber, L., Holzleitner, M., and Brandstetter, J.

   arXiv preprint arXiv:2402.12365 2024

   Abs url Code

   Neural operators, serving as physics surrogate models, have recently gained increased interest. With ever increasing problem complexity, the natural question arises: what is an efficient way to scale neural operators to larger and more complex simulations - most importantly by taking into account different types of simulation datasets. This is of special interest since, akin to their numerical counterparts, different techniques are used across applications, even if the underlying dynamics of the systems are similar. Whereas the flexibility of transformers has enabled unified architectures across domains, neural operators mostly follow a problem specific design, where GNNs are commonly used for Lagrangian simulations and grid-based models predominate Eulerian simulations. We introduce Universal Physics Transformers (UPTs), an efficient and unified learning paradigm for a wide range of spatio-temporal problems. UPTs operate without grid- or particle-based latent structures, enabling flexibility and scalability across meshes and particles. UPTs efficiently propagate dynamics in the latent space, emphasized by inverse encoding and decoding techniques. Finally, UPTs allow for queries of the latent space representation at any point in space-time. We demonstrate diverse applicability and efficacy of UPTs in mesh-based fluid simulations, and steady-state Reynolds averaged Navier-Stokes simulations, and Lagrangian-based dynamics.

7. Potential predictors for deterioration of renal function after transfusion

   Tschoellitsch, T., Moser, P., Maletzky, A., Seidl, P., Böck, C., Roland, T., Ludwig, H., Süssner, S., Hochreiter, S., and Meier, J.

   Anesthesia & Analgesia 2024
8. MIM-Refiner: A Contrastive Learning Boost from Intermediate Pre-Trained Representations

   Alkin, B., Miklautz, L., Hochreiter, S., and Brandstetter, J.

   arXiv preprint arXiv:2402.10093 2024

   Abs url Code

   We introduce MIM (Masked Image Modeling)-Refiner, a contrastive learning boost for pre-trained MIM models. MIM-Refiner is motivated by the insight that strong representations within MIM models generally reside in intermediate layers. Accordingly, MIM-Refiner leverages multiple contrastive heads that are connected to different intermediate layers. In each head, a modified nearest neighbor objective constructs semantic clusters that capture semantic information which improves performance on downstream tasks, including off-the-shelf and fine-tuning settings. The refinement process is short and simple - yet highly effective. Within a few epochs, we refine the features of MIM models from subpar to state-of-the-art, off-the-shelf features. Refining a ViT-H, pre-trained with data2vec 2.0 on ImageNet-1K, sets a new state-of-the-art in linear probing (84.7%) and low-shot classification among models that are pre-trained on ImageNet-1K. MIM-Refiner efficiently combines the advantages of MIM and ID objectives and compares favorably against previous state-of-the-art SSL models on a variety of benchmarks such as low-shot classification, long-tailed classification, clustering and semantic segmentation.

9. ICML

   A Diffusion Model Framework for Unsupervised Neural Combinatorial Optimization

   Sanokowski, S., Hochreiter, S., and Lehner, S.

   Proceedings of the 41st International Conference on Machine Learning 2024

   url Code
10. arXiv

    Energy-based hopfield boosting for out-of-distribution detection

    Hofmann, C., Schmid, S., Lehner, B., Klotz, D., and Hochreiter, S.

    arXiv preprint arXiv:2405.08766 2024

    Abs url Code

    Out-of-distribution (OOD) detection is critical when deploying machine learning models in the real world. Outlier exposure methods, which incorporate auxiliary outlier data in the training process, can drastically improve OOD detection performance compared to approaches without advanced training strategies. We introduce Hopfield Boosting, a boosting approach, which leverages modern Hopfield energy (MHE) to sharpen the decision boundary between the in-distribution and OOD data. Hopfield Boosting encourages the model to concentrate on hard-to-distinguish auxiliary outlier examples that lie close to the decision boundary between in-distribution and auxiliary outlier data. Our method achieves a new state-of-the-art in OOD detection with outlier exposure, improving the FPR95 metric from 2.28 to 0.92 on CIFAR-10 and from 11.76 to 7.94 on CIFAR-100.