ADADELTA: An Adaptive Learning Rate Method

Matthew D. Zeiler

doi:https://doi.org/10.48550/arxiv.1212.5701

ADADELTA: An Adaptive Learning Rate Method

2012 · arXiv (Cornell University) · 5,532 citations · 24 from inside this corpus

The source holds an abstract for this work, but its best open-access copy is under no open licence, which does not permit us to republish the text. Read it at the source below.

26 of 26 neighbouring works in this corpus. Blue is what this paper cites; orange is what cites it, and a dashed line is one neighbour citing another. Only the largest labels are drawn — every node carries its full title on hover.

this paper works it cites works citing it node size = global citations · hover for the full title

Citation edge evidence

Each row is directed from the citing work to the cited work. Source status remains readable without the interactive graph.

Adam: A Method for Stochastic Optimization→ADADELTA: An Adaptive Learning Rate MethodLegacy edge — source detail unavailable
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Transla…→ADADELTA: An Adaptive Learning Rate MethodLegacy edge — source detail unavailable
Deep learning in neural networks: An overview→ADADELTA: An Adaptive Learning Rate MethodLegacy edge — source detail unavailable
Neural Machine Translation by Jointly Learning to Align and Translate→ADADELTA: An Adaptive Learning Rate MethodLegacy edge — source detail unavailable
Convolutional Neural Networks for Sentence Classification→ADADELTA: An Adaptive Learning Rate MethodLegacy edge — source detail unavailable
Domain-Adversarial Training of Neural Networks→ADADELTA: An Adaptive Learning Rate MethodLegacy edge — source detail unavailable
[No title in the source record — Edinburgh Research Explorer (University of Edinburgh)]→ADADELTA: An Adaptive Learning Rate MethodLegacy edge — source detail unavailable
On the Properties of Neural Machine Translation: Encoder–Decoder Approaches→ADADELTA: An Adaptive Learning Rate MethodLegacy edge — source detail unavailable
An overview of gradient descent optimization algorithms→ADADELTA: An Adaptive Learning Rate MethodLegacy edge — source detail unavailable
Neural Architectures for Named Entity Recognition→ADADELTA: An Adaptive Learning Rate MethodLegacy edge — source detail unavailable
Deep Convolutional Neural Networks for Image Classification: A Comprehensive Review→ADADELTA: An Adaptive Learning Rate MethodLegacy edge — source detail unavailable
Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Transla…→ADADELTA: An Adaptive Learning Rate MethodLegacy edge — source detail unavailable
End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF→ADADELTA: An Adaptive Learning Rate MethodLegacy edge — source detail unavailable
Optimization as a Model for Few-Shot Learning→ADADELTA: An Adaptive Learning Rate MethodLegacy edge — source detail unavailable
Abstractive Text Summarization using Sequence-to-sequence RNNs and Beyond→ADADELTA: An Adaptive Learning Rate MethodLegacy edge — source detail unavailable
Attention-Based Bidirectional Long Short-Term Memory Networks for Relation Classification→ADADELTA: An Adaptive Learning Rate MethodLegacy edge — source detail unavailable
Knowledge Graph Embedding via Dynamic Mapping Matrix→ADADELTA: An Adaptive Learning Rate MethodLegacy edge — source detail unavailable
Deep Contextualized Word Representations→ADADELTA: An Adaptive Learning Rate MethodLegacy edge — source detail unavailable
SGDR: Stochastic Gradient Descent with Warm Restarts→ADADELTA: An Adaptive Learning Rate MethodLegacy edge — source detail unavailable
Asynchronous Methods for Deep Reinforcement Learning→ADADELTA: An Adaptive Learning Rate MethodLegacy edge — source detail unavailable
A Gift from Knowledge Distillation: Fast Optimization, Network Minimization and Transfer …→ADADELTA: An Adaptive Learning Rate MethodLegacy edge — source detail unavailable
Automated Machine Learning→ADADELTA: An Adaptive Learning Rate MethodLegacy edge — source detail unavailable
Improved Adam Optimizer for Deep Neural Networks→ADADELTA: An Adaptive Learning Rate MethodLegacy edge — source detail unavailable
A Survey on Deep Learning→ADADELTA: An Adaptive Learning Rate MethodLegacy edge — source detail unavailable
ADADELTA: An Adaptive Learning Rate Method→Learning representations by back-propagating errorsLegacy edge — source detail unavailable
ADADELTA: An Adaptive Learning Rate Method→Context-Dependent Pre-Trained Deep Neural Networks for Large-Vocabulary Speech RecognitionLegacy edge — source detail unavailable
Deep learning in neural networks: An overview→Context-Dependent Pre-Trained Deep Neural Networks for Large-Vocabulary Speech RecognitionLegacy edge — source detail unavailable
Neural Machine Translation by Jointly Learning to Align and Translate→Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Transla…Legacy edge — source detail unavailable
Context-Dependent Pre-Trained Deep Neural Networks for Large-Vocabulary Speech Recognition→Learning representations by back-propagating errorsLegacy edge — source detail unavailable
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Transla…→Context-Dependent Pre-Trained Deep Neural Networks for Large-Vocabulary Speech RecognitionLegacy edge — source detail unavailable
Asynchronous Methods for Deep Reinforcement Learning→Adam: A Method for Stochastic OptimizationLegacy edge — source detail unavailable
Neural Architectures for Named Entity Recognition→Adam: A Method for Stochastic OptimizationLegacy edge — source detail unavailable
SGDR: Stochastic Gradient Descent with Warm Restarts→Adam: A Method for Stochastic OptimizationLegacy edge — source detail unavailable
Deep Convolutional Neural Networks for Image Classification: A Comprehensive Review→Learning representations by back-propagating errorsLegacy edge — source detail unavailable
Deep Convolutional Neural Networks for Image Classification: A Comprehensive Review→Adam: A Method for Stochastic OptimizationLegacy edge — source detail unavailable
Deep Convolutional Neural Networks for Image Classification: A Comprehensive Review→Domain-Adversarial Training of Neural NetworksLegacy edge — source detail unavailable
Deep Convolutional Neural Networks for Image Classification: A Comprehensive Review→Deep learning in neural networks: An overviewLegacy edge — source detail unavailable
A Gift from Knowledge Distillation: Fast Optimization, Network Minimization and Transfer …→Adam: A Method for Stochastic OptimizationLegacy edge — source detail unavailable
Deep Contextualized Word Representations→Neural Architectures for Named Entity RecognitionLegacy edge — source detail unavailable
A Survey on Deep Learning→Adam: A Method for Stochastic OptimizationLegacy edge — source detail unavailable
A Survey on Deep Learning→Deep learning in neural networks: An overviewLegacy edge — source detail unavailable
A Survey on Deep Learning→Neural Machine Translation by Jointly Learning to Align and TranslateLegacy edge — source detail unavailable
A Survey on Deep Learning→Context-Dependent Pre-Trained Deep Neural Networks for Large-Vocabulary Speech RecognitionLegacy edge — source detail unavailable
A Survey on Deep Learning→Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Transla…Legacy edge — source detail unavailable
Improved Adam Optimizer for Deep Neural Networks→Adam: A Method for Stochastic OptimizationLegacy edge — source detail unavailable
Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Transla…→Context-Dependent Pre-Trained Deep Neural Networks for Large-Vocabulary Speech RecognitionLegacy edge — source detail unavailable
[No title in the source record — Edinburgh Research Explorer (University of Edinburgh)]→Neural Machine Translation by Jointly Learning to Align and TranslateLegacy edge — source detail unavailable
[No title in the source record — Edinburgh Research Explorer (University of Edinburgh)]→Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Transla…Legacy edge — source detail unavailable
End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF→Adam: A Method for Stochastic OptimizationLegacy edge — source detail unavailable
End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF→Neural Architectures for Named Entity RecognitionLegacy edge — source detail unavailable
End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF→On the Properties of Neural Machine Translation: Encoder–Decoder ApproachesLegacy edge — source detail unavailable
Abstractive Text Summarization using Sequence-to-sequence RNNs and Beyond→Neural Machine Translation by Jointly Learning to Align and TranslateLegacy edge — source detail unavailable
On the Properties of Neural Machine Translation: Encoder–Decoder Approaches→Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Transla…Legacy edge — source detail unavailable
Automated Machine Learning→Adam: A Method for Stochastic OptimizationLegacy edge — source detail unavailable
Automated Machine Learning→Optimization as a Model for Few-Shot LearningLegacy edge — source detail unavailable

What this paper cites, inside the corpus

Paper	Year	Cited
Learning representations by back-propagating errors	1986	31,688
Context-Dependent Pre-Trained Deep Neural Networks for Large-Vocabulary Speech Recognition	2011	3,089

What cites it, inside the corpus

Paper	Year	Cited
Adam: A Method for Stochastic Optimization	2014	84,698
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Transla…	2014	25,048
Deep learning in neural networks: An overview	2014	18,236
Neural Machine Translation by Jointly Learning to Align and Translate	2014	14,615
Convolutional Neural Networks for Sentence Classification	2014	13,987
Domain-Adversarial Training of Neural Networks	2017	7,702
[No title in the source record — Edinburgh Research Explorer (University of Edinburgh)]		7,291
On the Properties of Neural Machine Translation: Encoder–Decoder Approaches	2014	6,683
An overview of gradient descent optimization algorithms	2016	4,807
Neural Architectures for Named Entity Recognition	2016	4,482
Deep Convolutional Neural Networks for Image Classification: A Comprehensive Review	2017	3,570
Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Transla…	2014	3,518
End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF	2016	2,595
Optimization as a Model for Few-Shot Learning	2017	2,436
Abstractive Text Summarization using Sequence-to-sequence RNNs and Beyond	2016	2,233
Attention-Based Bidirectional Long Short-Term Memory Networks for Relation Classification	2016	2,062
Knowledge Graph Embedding via Dynamic Mapping Matrix	2015	1,868
Deep Contextualized Word Representations	2018	1,783
SGDR: Stochastic Gradient Descent with Warm Restarts	2016	1,735
Asynchronous Methods for Deep Reinforcement Learning	2016	1,689
A Gift from Knowledge Distillation: Fast Optimization, Network Minimization and Transfer …	2017	1,660
Automated Machine Learning	2019	1,451
Improved Adam Optimizer for Deep Neural Networks	2018	1,408
A Survey on Deep Learning	2018	1,398

Links

Download BibTeX · Download RIS · DOI · Open access copy · OpenAlex record

Topics

Neural Networks and Applications	Computer Science
Domain Adaptation and Few-Shot Learning	Computer Science
Machine Learning and Algorithms	Computer Science

Is this record sound?

complete

Nothing in this record contradicts itself and no field we check is missing.

supports1 author record(s) attached.
supports6 reference(s) recorded.
neutralThe DOI carries no year to check against.
supportsA title is present.

Provenance

Everything above was read from one stored OpenAlex payload, fetched 2026-09-04T03:58:44+00:00.

sha256 648f8491aaceaa34…