Deep Learning

Artificial Intelligence (AI) has recently revolutionised various fields of science and has also started to pervade commercial applications in an unprecedented manner. Despite great successes, most of AI’s enormous potential is still to be realised. The recent surge of AI can be attributed to advances in the machine learning field known as “Deep Learning”, that is, large deeply-layered artificial neural networks (ANNs) trained by modern learning algorithms on massive datasets. In its core, Deep Learning discovers multiple levels of distributed representations of the input, with higher levels representing more abstract concepts. These representations led to impressive successes in different research areas. In particular, artificial neural networks considerably improved the performance in computer vision, speech recognition, and internet advertising.

Sepp Hochreiter, heading this research group, is considered a pioneer of Deep Learning with his discovery of the vanishing gradient problem and the invention of long-short term memory (LSTM) networks.

recent publications in Deep Learning:

  1. arXiv
    Semantic HELM: An Interpretable Memory for Reinforcement Learning
    Paischer, F., Adler, T., Hofmarcher, M., and Hochreiter, S.
    2023
  2. arXiv
    SITTA: A Semantic Image-Text Alignment for Image Captioning
    Paischer, F., Adler, T., Hofmarcher, M., and Hochreiter, S.
    2023
  3. arXiv
    Quantification of Uncertainty with Adversarial Models
    Schweighofer, K., Aichberger, L., Ielanski, M., Klambauer, G., and Hochreiter, S.
    2023
  4. arXiv
    Boundary Graph Neural Networks for 3D Simulations
    Mayr, A., Lehner, S., Mayrhofer, A., Kloss, C., Hochreiter, S., and Brandstetter, J.
    2023
  5. arXiv
    Contrastive Tuning: A Little Help to Make Masked Autoencoders Forget
    Lehner, J., Alkin, B., Fürst, A., Rumetshofer, E., Miklautz, L., and Hochreiter, S.
    2023
  6. AAAI
    Boundary Graph Neural Networks for 3D Simulations
    Mayr, A., Lehner, S., Mayrhofer, A., Kloss, C., Hochreiter, S., and Brandstetter, J.
    Proceedings of the AAAI Conference on Artificial Intelligence 2023
  7. ICLR
    Normalization is dead, long live normalization!
    Hoedt, P., Hochreiter, S., and Klambauer, G.
    In ICLR Blog Track 2022
  8. ML4PS
    One Network to Approximate Them All: Amortized Variational Inference of Ising Ground States
    Sanokowski, S., Berghammer, W., Johannes, K., Hochreiter, S., and Lehner, S.
    2022
  9. ML4PS
    Using Shadows to Learn Ground State Properties of Quantum Hamiltonians
    Tran, V., Lewis, L., Huang, H., Kofler, J., Kueng, R., Hochreiter, S., and Lehner, S.
    2022
  10. CoLLAs
    Few-Shot Learning by Dimensionality Reduction in Gradient Space
    Gauch, M., Beck, M., Adler, T., Kotsur, D., Fiel, S., Eghbal-zadeh, H., Brandstetter, J., Kofler, J., Holzleitner, M., Zellinger, W., Klotz, D., Hochreiter, S., and Lehner, S.
    2022
  11. arXiv
    Trusted Artificial Intelligence: Towards Certification of Machine Learning Applications
    Winter, P., Eder, S., Weissenböck, J., Schwald, C., Doms, T., Vogt, T., Hochreiter, S., and Nessler, B.
    2021
  12. arXiv
    Learning 3D Granular Flow Simulations
    Mayr, A., Lehner, S., Mayrhofer, A., Kloss, C., Hochreiter, S., and Brandstetter, J.
    2021
  13. ICML
    MC-LSTM: Mass-Conserving LSTM
    Hoedt, P., Kratzert, F., Klotz, D., Halmich, C., Holzleitner, M., Nearing, G., Hochreiter, S., and Klambauer, G.
    In Proceedings of the 38th International Conference on Machine Learning 2021
  14. bioarXiv
    DeepRC: Immune Repertoire Classification with Attention-Based Deep Massive Multiple Instance Learning
    Widrich, M., Schäfl, B., Pavlović, M., Sandve, G., Hochreiter, S., Greiff, V., and Klambauer, G.
    2020
  15. arXiv
    Cross-Domain Few-Shot Learning by Representation Fusion
    Adler, T., Brandstetter, J., Widrich, M., Mayr, A., Kreil, D., Kopp, M., Klambauer, G., and Hochreiter, S.
    arXiv preprint arXiv:2010.06498 2020
  16. First Order Generative Adversarial Networks
    Seward, C., Unterthiner, T., Bergmann, U., Jetchev, N., and Hochreiter, S.
    2018
  17. Coulomb GANs: Provably Optimal Nash Equilibria via Potential Fields
    Unterthiner, T., Nessler, B., Seward, C., Klambauer, G., Heusel, M., Ramsauer, H., and Hochreiter, S.
    2018
  18. NeurIPS
    Self-Normalizing Neural Networks
    Klambauer, G., Unterthiner, T., Mayr, A., and Hochreiter, S.
    2017
  19. GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
    Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., and Hochreiter, S.
    2017