no code implementations • 20 Jan 2023 • Colin White, Mahmoud Safari, Rhea Sukthanker, Binxin Ru, Thomas Elsken, Arber Zela, Debadeepta Dey, Frank Hutter
Specialized, high-performing neural architectures are crucial to the success of deep learning in these areas.
Natural Language Understanding Neural Architecture Search +2
no code implementations • 15 Feb 2022 • Thomas Elsken, Arber Zela, Jan Hendrik Metzen, Benedikt Staffler, Thomas Brox, Abhinav Valada, Frank Hutter
The success of deep learning in recent years has lead to a rising demand for neural network architecture engineering.
no code implementations • 8 Jul 2021 • Thomas Elsken, Benedikt Staffler, Arber Zela, Jan Hendrik Metzen, Frank Hutter
While neural architecture search methods have been successful in previous years and led to new state-of-the-art performance on various problems, they have also been criticized for being unstable, being highly sensitive with respect to their hyperparameters, and often not performing better than random search.
1 code implementation • ICML Workshop AutoML 2021 • Julia Guerrero-Viu, Sven Hauns, Sergio Izquierdo, Guilherme Miotto, Simon Schrodi, Andre Biedenkapp, Thomas Elsken, Difan Deng, Marius Lindauer, Frank Hutter
Neural architecture search (NAS) and hyperparameter optimization (HPO) make deep learning accessible to non-experts by automatically finding the architecture of the deep neural network to use and tuning the hyperparameters of the used training pipeline.
1 code implementation • NeurIPS 2021 • Sheheryar Zaidi, Arber Zela, Thomas Elsken, Chris Holmes, Frank Hutter, Yee Whye Teh
On a variety of classification tasks and modern architecture search spaces, we show that the resulting ensembles outperform deep ensembles not only in terms of accuracy but also uncertainty calibration and robustness to dataset shift.
2 code implementations • CVPR 2020 • Thomas Elsken, Benedikt Staffler, Jan Hendrik Metzen, Frank Hutter
The recent progress in neural architecture search (NAS) has allowed scaling the automated design of neural architectures to real-world domains, such as object detection and semantic segmentation.
no code implementations • 30 Sep 2019 • Christoph Schorn, Thomas Elsken, Sebastian Vogel, Armin Runge, Andre Guntoro, Gerd Ascheid
It is thus desirable to exploit optimization potential for error resilience and efficiency also at the algorithmic side, e. g., by optimizing the architecture of the DNN.
1 code implementation • ICLR 2020 • Arber Zela, Thomas Elsken, Tonmoy Saikia, Yassine Marrakchi, Thomas Brox, Frank Hutter
Differentiable Architecture Search (DARTS) has attracted a lot of attention due to its simplicity and small search costs achieved by a continuous relaxation and an approximation of the resulting bi-level optimization problem.
1 code implementation • 16 Aug 2018 • Thomas Elsken, Jan Hendrik Metzen, Frank Hutter
Deep Learning has enabled remarkable progress over the last years on a variety of tasks, such as image recognition, speech recognition, and machine translation.
no code implementations • ICLR 2019 • Thomas Elsken, Jan Hendrik Metzen, Frank Hutter
Neural Architecture Search aims at automatically finding neural architectures that are competitive with architectures designed by human experts.
3 code implementations • ICLR 2018 • Thomas Elsken, Jan-Hendrik Metzen, Frank Hutter
Neural networks have recently had a lot of success for many tasks.