Who Cited It

On the importance of initialization and momentum in deep learning

2013 · 3,523 citations · 18 from inside this corpus

Ilya Sutskever, James Martens low, George E. Dahl, Geoffrey E. Hinton

The source holds an abstract for this work, but its best open-access copy is under no open licence, which does not permit us to republish the text. Read it at the source below.

On the importance of initialization and momentum in deep learning (2013)On the importance of initiali…Long Short-Term Memory (1997)Long Short-Term MemoryReducing the Dimensionality of Data with Neural Networks (2006)Reducing the Dimensionality o…A Fast Learning Algorithm for Deep Belief Nets (2006)A Fast Learning Algorithm for…Understanding the difficulty of training deep feedforward neural networks (2010)Understanding the difficulty …Learning long-term dependencies with gradient descent is difficult (1994)Learning long-term dependenci…Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communi… (2004)Harnessing Nonlinearity: Pred…Context-Dependent Pre-Trained Deep Neural Networks for Large-Vocabulary Speech Recognition (2011)Context-Dependent Pre-Trained…Neural Networks: Tricks of the Trade (2012)Neural Networks: Tricks of th…Deep Neural Networks for Acoustic Modeling in Speech Recognition (2012)Deep Neural Networks for Acou…Acoustic Modeling Using Deep Belief Networks (2011)Acoustic Modeling Using Deep …Adam: A Method for Stochastic Optimization (2014)Adam: A Method for Stochastic…Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Sh… (2015)Batch Normalization: Accelera…Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Sh… (2024)Batch Normalization: Accelera…Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning (2017)Inception-v4, Inception-ResNe…Deep Learning in Medical Image Analysis (2017)Deep Learning in Medical Imag…Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning (2017)Inception-v4, Inception-ResNe…Neural Architecture Search with Reinforcement Learning (2016)Neural Architecture Search wi…Deep Convolutional Neural Networks for Image Classification: A Comprehensive Review (2017)Deep Convolutional Neural Net…Character-level Convolutional Networks for Text Classification (2015)Character-level Convolutional…Boosting Adversarial Attacks with Momentum (2018)Boosting Adversarial Attacks …FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence (2020)FixMatch: Simplifying Semi-Su…Deep Speech 2: End-to-End Speech Recognition in English and Mandarin (2015)Deep Speech 2: End-to-End Spe…Convolutional Sequence to Sequence Learning (2017)Convolutional Sequence to Seq…A State-of-the-Art Survey on Deep Learning Theory and Architectures (2019)A State-of-the-Art Survey on …Deep Speech: Scaling up end-to-end speech recognition (2014)Deep Speech: Scaling up end-t…Learning Temporal Regularity in Video Sequences (2016)Learning Temporal Regularity …An Empirical Exploration of Recurrent Network Architectures (2015)An Empirical Exploration of R…A Survey on Deep Learning (2018)A Survey on Deep Learning
28 of 28 neighbouring works in this corpus. Blue is what this paper cites; orange is what cites it, and a dashed line is one neighbour citing another. Only the largest labels are drawn — every node carries its full title on hover.
this paper works it cites works citing it node size = global citations · hover for the full title

What this paper cites, inside the corpus

PaperYearCited
Long Short-Term Memory1997101,359
Reducing the Dimensionality of Data with Neural Networks200621,247
A Fast Learning Algorithm for Deep Belief Nets200616,540
Understanding the difficulty of training deep feedforward neural networks201012,673
Learning long-term dependencies with gradient descent is difficult19948,610
Harnessing Nonlinearity: Predicting Chaotic Systems and Saving Energy in Wireless Communi…20043,862
Context-Dependent Pre-Trained Deep Neural Networks for Large-Vocabulary Speech Recognition20113,089
Neural Networks: Tricks of the Trade20122,159
Deep Neural Networks for Acoustic Modeling in Speech Recognition20121,907
Acoustic Modeling Using Deep Belief Networks20111,752

What cites it, inside the corpus

PaperYearCited
Adam: A Method for Stochastic Optimization201484,698
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Sh…201524,404
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Sh…202415,663
Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning201712,734
Deep Learning in Medical Image Analysis20174,917
Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning20174,487
Neural Architecture Search with Reinforcement Learning20163,881
Deep Convolutional Neural Networks for Image Classification: A Comprehensive Review20173,570
Character-level Convolutional Networks for Text Classification20153,280
Boosting Adversarial Attacks with Momentum20183,040
FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence20202,271
Deep Speech 2: End-to-End Speech Recognition in English and Mandarin20152,181
Convolutional Sequence to Sequence Learning20171,903
A State-of-the-Art Survey on Deep Learning Theory and Architectures20191,619
Deep Speech: Scaling up end-to-end speech recognition20141,519
Learning Temporal Regularity in Video Sequences20161,407
An Empirical Exploration of Recurrent Network Architectures20151,406
A Survey on Deep Learning20181,398

Links

OpenAlex record

Topics

Stochastic Gradient Optimization TechniquesComputer Science
Advanced Neural Network ApplicationsComputer Science
Privacy-Preserving Technologies in DataComputer Science

Is this record sound?

complete

Nothing in this record contradicts itself and no field we check is missing.

  • supports4 author record(s) attached.
  • supports28 reference(s) recorded.
  • neutralThe DOI carries no year to check against.
  • supportsA title is present.

Provenance

Everything above was read from one stored OpenAlex payload, fetched 2026-09-04T03:58:46+00:00.

sha256 bba2969b3567609a…