Distilling the Knowledge in a Neural Network
Geoffrey E. Hinton, Oriol Vinyals, Jay B. Dean
The source holds an abstract for this work, but its best open-access copy is under no open licence, which does not permit us to republish the text. Read it at the source below.
this paper
works it cites
works citing it
node size = global citations · hover for the full title
Citation edge evidence
- Xception: Deep Learning with Depthwise Separable Convolutions→Distilling the Knowledge in a Neural NetworkLegacy edge — source detail unavailable
- Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challe…→Distilling the Knowledge in a Neural NetworkLegacy edge — source detail unavailable
- Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)→Distilling the Knowledge in a Neural NetworkLegacy edge — source detail unavailable
- Emerging Properties in Self-Supervised Vision Transformers→Distilling the Knowledge in a Neural NetworkLegacy edge — source detail unavailable
- DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter→Distilling the Knowledge in a Neural NetworkLegacy edge — source detail unavailable
- Membership Inference Attacks Against Machine Learning Models→Distilling the Knowledge in a Neural NetworkLegacy edge — source detail unavailable
- Deep Reinforcement Learning: A Brief Survey→Distilling the Knowledge in a Neural NetworkLegacy edge — source detail unavailable
- Learning without Forgetting→Distilling the Knowledge in a Neural NetworkLegacy edge — source detail unavailable
- Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer→Distilling the Knowledge in a Neural NetworkLegacy edge — source detail unavailable
- Unsupervised Feature Learning via Non-parametric Instance Discrimination→Distilling the Knowledge in a Neural NetworkLegacy edge — source detail unavailable
- Deep Convolutional Neural Networks for Image Classification: A Comprehensive Review→Distilling the Knowledge in a Neural NetworkLegacy edge — source detail unavailable
- Grad-CAM++: Generalized Gradient-Based Visual Explanations for Deep Convolutional Networks→Distilling the Knowledge in a Neural NetworkLegacy edge — source detail unavailable
- Continual lifelong learning with neural networks: A review→Distilling the Knowledge in a Neural NetworkLegacy edge — source detail unavailable
- Language Models are Few-Shot Learners→Distilling the Knowledge in a Neural NetworkLegacy edge — source detail unavailable
- On the Opportunities and Risks of Foundation Models→Distilling the Knowledge in a Neural NetworkLegacy edge — source detail unavailable
- Deep visual domain adaptation: A survey→Distilling the Knowledge in a Neural NetworkLegacy edge — source detail unavailable
- Ensemble deep learning: A review→Distilling the Knowledge in a Neural NetworkLegacy edge — source detail unavailable
- Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey→Distilling the Knowledge in a Neural NetworkLegacy edge — source detail unavailable
- Machine Learning Interpretability: A Survey on Methods and Metrics→Distilling the Knowledge in a Neural NetworkLegacy edge — source detail unavailable
- Adversarial Examples: Attacks and Defenses for Deep Learning→Distilling the Knowledge in a Neural NetworkLegacy edge — source detail unavailable
- ZOO→Distilling the Knowledge in a Neural NetworkLegacy edge — source detail unavailable
- On Calibration of Modern Neural Networks→Distilling the Knowledge in a Neural NetworkLegacy edge — source detail unavailable
- TinyBERT: Distilling BERT for Natural Language Understanding→Distilling the Knowledge in a Neural NetworkLegacy edge — source detail unavailable
- AutoML: A survey of the state-of-the-art→Distilling the Knowledge in a Neural NetworkLegacy edge — source detail unavailable
- Text Data Augmentation for Deep Learning→Distilling the Knowledge in a Neural NetworkLegacy edge — source detail unavailable
- Explainable Artificial Intelligence (XAI): What we know and what is left to attain Trustw…→Distilling the Knowledge in a Neural NetworkLegacy edge — source detail unavailable
- Relational Knowledge Distillation→Distilling the Knowledge in a Neural NetworkLegacy edge — source detail unavailable
- A Gift from Knowledge Distillation: Fast Optimization, Network Minimization and Transfer …→Distilling the Knowledge in a Neural NetworkLegacy edge — source detail unavailable
- A continual learning survey: Defying forgetting in classification tasks→Distilling the Knowledge in a Neural NetworkLegacy edge — source detail unavailable
- Unsupervised Data Augmentation for Consistency Training→Distilling the Knowledge in a Neural NetworkLegacy edge — source detail unavailable
- Distilling the Knowledge in a Neural Network→Dropout: a simple way to prevent neural networks from overfittingLegacy edge — source detail unavailable
- Distilling the Knowledge in a Neural Network→Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Fou…Legacy edge — source detail unavailable
- Distilling the Knowledge in a Neural Network→Ensemble Methods in Machine LearningLegacy edge — source detail unavailable
- Distilling the Knowledge in a Neural Network→Improving neural networks by preventing co-adaptation of feature detectorsLegacy edge — source detail unavailable
- Distilling the Knowledge in a Neural Network→Adaptive Mixtures of Local ExpertsLegacy edge — source detail unavailable
- Distilling the Knowledge in a Neural Network→Model compressionLegacy edge — source detail unavailable
- Improving neural networks by preventing co-adaptation of feature detectors→Adaptive Mixtures of Local ExpertsLegacy edge — source detail unavailable
- Membership Inference Attacks Against Machine Learning Models→Dropout: a simple way to prevent neural networks from overfittingLegacy edge — source detail unavailable
- Deep Convolutional Neural Networks for Image Classification: A Comprehensive Review→Improving neural networks by preventing co-adaptation of feature detectorsLegacy edge — source detail unavailable
- Deep Convolutional Neural Networks for Image Classification: A Comprehensive Review→Dropout: a simple way to prevent neural networks from overfittingLegacy edge — source detail unavailable
- Deep Convolutional Neural Networks for Image Classification: A Comprehensive Review→Xception: Deep Learning with Depthwise Separable ConvolutionsLegacy edge — source detail unavailable
- Unsupervised Data Augmentation for Consistency Training→Xception: Deep Learning with Depthwise Separable ConvolutionsLegacy edge — source detail unavailable
- Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey→Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Fou…Legacy edge — source detail unavailable
- Threat of Adversarial Attacks on Deep Learning in Computer Vision: A Survey→ZOOLegacy edge — source detail unavailable
- Relational Knowledge Distillation→Model compressionLegacy edge — source detail unavailable
- Relational Knowledge Distillation→A Gift from Knowledge Distillation: Fast Optimization, Network Minimization and Transfer …Legacy edge — source detail unavailable
- Adversarial Examples: Attacks and Defenses for Deep Learning→Membership Inference Attacks Against Machine Learning ModelsLegacy edge — source detail unavailable
- Adversarial Examples: Attacks and Defenses for Deep Learning→ZOOLegacy edge — source detail unavailable
- Machine Learning Interpretability: A Survey on Methods and Metrics→Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)Legacy edge — source detail unavailable
- AutoML: A survey of the state-of-the-art→Learning without ForgettingLegacy edge — source detail unavailable
- AutoML: A survey of the state-of-the-art→Xception: Deep Learning with Depthwise Separable ConvolutionsLegacy edge — source detail unavailable
- AutoML: A survey of the state-of-the-art→A Gift from Knowledge Distillation: Fast Optimization, Network Minimization and Transfer …Legacy edge — source detail unavailable
- Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challe…→Model compressionLegacy edge — source detail unavailable
- Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challe…→Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)Legacy edge — source detail unavailable
- Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challe…→Adversarial Examples: Attacks and Defenses for Deep LearningLegacy edge — source detail unavailable
- Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer→Dropout: a simple way to prevent neural networks from overfittingLegacy edge — source detail unavailable
- Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer→DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighterLegacy edge — source detail unavailable
- Language Models are Few-Shot Learners→Unsupervised Data Augmentation for Consistency TrainingLegacy edge — source detail unavailable
- Language Models are Few-Shot Learners→DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighterLegacy edge — source detail unavailable
- Language Models are Few-Shot Learners→TinyBERT: Distilling BERT for Natural Language UnderstandingLegacy edge — source detail unavailable
- A continual learning survey: Defying forgetting in classification tasks→Improving neural networks by preventing co-adaptation of feature detectorsLegacy edge — source detail unavailable
- A continual learning survey: Defying forgetting in classification tasks→Dropout: a simple way to prevent neural networks from overfittingLegacy edge — source detail unavailable
- A continual learning survey: Defying forgetting in classification tasks→Continual lifelong learning with neural networks: A reviewLegacy edge — source detail unavailable
- TinyBERT: Distilling BERT for Natural Language Understanding→DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighterLegacy edge — source detail unavailable
- Ensemble deep learning: A review→Ensemble Methods in Machine LearningLegacy edge — source detail unavailable
- Ensemble deep learning: A review→Dropout: a simple way to prevent neural networks from overfittingLegacy edge — source detail unavailable
- Emerging Properties in Self-Supervised Vision Transformers→Unsupervised Feature Learning via Non-parametric Instance DiscriminationLegacy edge — source detail unavailable
- Emerging Properties in Self-Supervised Vision Transformers→Unsupervised Data Augmentation for Consistency TrainingLegacy edge — source detail unavailable
- Text Data Augmentation for Deep Learning→Dropout: a simple way to prevent neural networks from overfittingLegacy edge — source detail unavailable
- Text Data Augmentation for Deep Learning→Unsupervised Data Augmentation for Consistency TrainingLegacy edge — source detail unavailable
- Text Data Augmentation for Deep Learning→AutoML: A survey of the state-of-the-artLegacy edge — source detail unavailable
- Text Data Augmentation for Deep Learning→DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighterLegacy edge — source detail unavailable
- On the Opportunities and Risks of Foundation Models→Ensemble Methods in Machine LearningLegacy edge — source detail unavailable
- On the Opportunities and Risks of Foundation Models→Membership Inference Attacks Against Machine Learning ModelsLegacy edge — source detail unavailable
- On the Opportunities and Risks of Foundation Models→Continual lifelong learning with neural networks: A reviewLegacy edge — source detail unavailable
- On the Opportunities and Risks of Foundation Models→Unsupervised Feature Learning via Non-parametric Instance DiscriminationLegacy edge — source detail unavailable
- On the Opportunities and Risks of Foundation Models→DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighterLegacy edge — source detail unavailable
- On the Opportunities and Risks of Foundation Models→Language Models are Few-Shot LearnersLegacy edge — source detail unavailable
- Explainable Artificial Intelligence (XAI): What we know and what is left to attain Trustw…→Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)Legacy edge — source detail unavailable
- Explainable Artificial Intelligence (XAI): What we know and what is left to attain Trustw…→Machine Learning Interpretability: A Survey on Methods and MetricsLegacy edge — source detail unavailable
- Explainable Artificial Intelligence (XAI): What we know and what is left to attain Trustw…→Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challe…Legacy edge — source detail unavailable
What this paper cites, inside the corpus
What cites it, inside the corpus
Links
Download BibTeX · Download RIS · DOI · Open access copy · OpenAlex record
Topics
| Neural Networks and Applications | Computer Science |
| Time Series Analysis and Forecasting | Computer Science |
| Topic Modeling | Computer Science |
Is this record sound?
complete
Nothing in this record contradicts itself and no field we check is missing.
- supports3 author record(s) attached.
- supports9 reference(s) recorded.
- neutralThe DOI carries no year to check against.
- supportsA title is present.
Provenance
sha256 7e3d99a592f7f61f…