Deep Learning

Artificial Intelligence (AI) has recently revolutionised various fields of science and has also started to pervade commercial applications in an unprecedented manner. Despite great successes, most of AI’s enormous potential is still to be realised. The recent surge of AI can be attributed to advances in the machine learning field known as “Deep Learning”, that is, large deeply-layered artificial neural networks (ANNs) trained by modern learning algorithms on massive datasets. In its core, Deep Learning discovers multiple levels of distributed representations of the input, with higher levels representing more abstract concepts. These representations led to impressive successes in different research areas. In particular, artificial neural networks considerably improved the performance in computer vision, speech recognition, and internet advertising.

Sepp Hochreiter, heading this research group, is considered a pioneer of Deep Learning with his discovery of the vanishing gradient problem and the invention of long-short term memory (LSTM) networks.

recent publications in Deep Learning:

  1. ICLR
    Normalization is dead, long live normalization!
    Hoedt, P., Hochreiter, S., and Klambauer, G.
    In ICLR Blog Track 2022
  2. CoLLAs
    Few-Shot Learning by Dimensionality Reduction in Gradient Space
    Gauch, M., Beck, M., Adler, T., Kotsur, D., Fiel, S., Eghbal-zadeh, H., Brandstetter, J., Kofler, J., Holzleitner, M., Zellinger, W., Klotz, D., Hochreiter, S., and Lehner, S.
  3. arXiv
    Learning 3D Granular Flow Simulations
    Mayr, A., Lehner, S., Mayrhofer, A., Kloss, C., Hochreiter, S., and Brandstetter, J.
  4. arXiv
    Trusted Artificial Intelligence: Towards Certification of Machine Learning Applications
    Winter, P., Eder, S., Weissenböck, J., Schwald, C., Doms, T., Vogt, T., Hochreiter, S., and Nessler, B.
  5. ICML
    MC-LSTM: Mass-Conserving LSTM
    Hoedt, P., Kratzert, F., Klotz, D., Halmich, C., Holzleitner, M., Nearing, G., Hochreiter, S., and Klambauer, G.
    In Proceedings of the 38th International Conference on Machine Learning 2021
  6. bioarXiv
    DeepRC: Immune Repertoire Classification with Attention-Based Deep Massive Multiple Instance Learning
    Widrich, M., Schäfl, B., Pavlović, M., Sandve, G., Hochreiter, S., Greiff, V., and Klambauer, G.
  7. arXiv
    Cross-Domain Few-Shot Learning by Representation Fusion
    Adler, T., Brandstetter, J., Widrich, M., Mayr, A., Kreil, D., Kopp, M., Klambauer, G., and Hochreiter, S.
    arXiv preprint arXiv:2010.06498 2020
  8. First Order Generative Adversarial Networks
    Seward, C., Unterthiner, T., Bergmann, U., Jetchev, N., and Hochreiter, S.
  9. Coulomb GANs: Provably Optimal Nash Equilibria via Potential Fields
    Unterthiner, T., Nessler, B., Seward, C., Klambauer, G., Heusel, M., Ramsauer, H., and Hochreiter, S.
  10. NeurIPS
    Self-Normalizing Neural Networks
    Klambauer, G., Unterthiner, T., Mayr, A., and Hochreiter, S.
  11. GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
    Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., and Hochreiter, S.