Text Data Augmentation for Deep Learning
Connor Shorten, Taghi M. Khoshgoftaar, Borko Furht
Natural Language Processing (NLP) is one of the most captivating applications of Deep Learning. In this survey, we consider how the Data Augmentation training strategy can aid in its development. We begin with the major motifs of Data Augmentation summarized into strengthening local decision boundaries, brute force training, causality and counterfactual examples, and the distinction between meaning and form. We follow these motifs with a concrete list of augmentation frameworks that have been developed for text data. Deep Learning generally struggles with the measurement of generalization and characterization of overfitting. We highlight studies that cover how augmentations can construct test sets for generalization. NLP is at an early stage in applying Data Augmentation compared to Computer Vision. We highlight the key differences and promising ideas that have yet to be tested in NLP. For the sake of practical implementation, we describe tools that facilitate Data Augmentation such as the use of consistency regularization, controllers, and offline and online augmentation pipelines, to preview a few. Finally, we discuss interesting topics around Data Augmentation in NLP such as task-specific augmentations, the use of prior knowledge in self-supervised learning versus Data Augmentation, intersections with transfer and multi-task learning, and ideas for AI-GAs (AI-Generating Algorithms). We hope this paper inspires further research interest in Text Data Augmentation.
this paper
works it cites
works citing it
node size = global citations · hover for the full title
Citation edge evidence
- Text Data Augmentation for Deep Learning→Dropout: a simple way to prevent neural networks from overfittingLegacy edge — source detail unavailable
- Text Data Augmentation for Deep Learning→BERT: Pre-training of Deep Bidirectional Transformers for Language UnderstandingLegacy edge — source detail unavailable
- Text Data Augmentation for Deep Learning→SMOTE: Synthetic Minority Over-sampling TechniqueLegacy edge — source detail unavailable
- Text Data Augmentation for Deep Learning→WordNetLegacy edge — source detail unavailable
- Text Data Augmentation for Deep Learning→Distilling the Knowledge in a Neural NetworkLegacy edge — source detail unavailable
- Text Data Augmentation for Deep Learning→Momentum Contrast for Unsupervised Visual Representation LearningLegacy edge — source detail unavailable
- Text Data Augmentation for Deep Learning→Sentence-BERT: Sentence Embeddings using Siamese BERT-NetworksLegacy edge — source detail unavailable
- Text Data Augmentation for Deep Learning→Building a Large Annotated Corpus of English: The Penn TreebankLegacy edge — source detail unavailable
- Text Data Augmentation for Deep Learning→A Simple Framework for Contrastive Learning of Visual RepresentationsLegacy edge — source detail unavailable
- Text Data Augmentation for Deep Learning→A survey of transfer learningLegacy edge — source detail unavailable
- Text Data Augmentation for Deep Learning→Advances and Open Problems in Federated LearningLegacy edge — source detail unavailable
- Text Data Augmentation for Deep Learning→Causality: Models, Reasoning and InferenceLegacy edge — source detail unavailable
- Text Data Augmentation for Deep Learning→DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighterLegacy edge — source detail unavailable
- Text Data Augmentation for Deep Learning→GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language UnderstandingLegacy edge — source detail unavailable
- Text Data Augmentation for Deep Learning→Affordance-Compiled Intelligence: Observable-Only Cognitive Impedance Matching for No-Met…Legacy edge — source detail unavailable
- Text Data Augmentation for Deep Learning→Survey on deep learning with class imbalanceLegacy edge — source detail unavailable
- Text Data Augmentation for Deep Learning→Causality: models, reasoning, and inferenceLegacy edge — source detail unavailable
- Text Data Augmentation for Deep Learning→A Survey on Knowledge Graphs: Representation, Acquisition, and ApplicationsLegacy edge — source detail unavailable
- Text Data Augmentation for Deep Learning→EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Ta…Legacy edge — source detail unavailable
- Text Data Augmentation for Deep Learning→RUSBoost: A Hybrid Approach to Alleviating Class ImbalanceLegacy edge — source detail unavailable
- Text Data Augmentation for Deep Learning→GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language UnderstandingLegacy edge — source detail unavailable
- Text Data Augmentation for Deep Learning→AutoML: A survey of the state-of-the-artLegacy edge — source detail unavailable
- Text Data Augmentation for Deep Learning→Unsupervised Data Augmentation for Consistency TrainingLegacy edge — source detail unavailable
- Distilling the Knowledge in a Neural Network→Dropout: a simple way to prevent neural networks from overfittingLegacy edge — source detail unavailable
- RUSBoost: A Hybrid Approach to Alleviating Class Imbalance→SMOTE: Synthetic Minority Over-sampling TechniqueLegacy edge — source detail unavailable
- Survey on deep learning with class imbalance→SMOTE: Synthetic Minority Over-sampling TechniqueLegacy edge — source detail unavailable
- Unsupervised Data Augmentation for Consistency Training→Distilling the Knowledge in a Neural NetworkLegacy edge — source detail unavailable
- Unsupervised Data Augmentation for Consistency Training→BERT: Pre-training of Deep Bidirectional Transformers for Language UnderstandingLegacy edge — source detail unavailable
- AutoML: A survey of the state-of-the-art→Distilling the Knowledge in a Neural NetworkLegacy edge — source detail unavailable
- AutoML: A survey of the state-of-the-art→SMOTE: Synthetic Minority Over-sampling TechniqueLegacy edge — source detail unavailable
- Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks→BERT: Pre-training of Deep Bidirectional Transformers for Language UnderstandingLegacy edge — source detail unavailable
- EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Ta…→WordNetLegacy edge — source detail unavailable
- EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Ta…→BERT: Pre-training of Deep Bidirectional Transformers for Language UnderstandingLegacy edge — source detail unavailable
- DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter→Distilling the Knowledge in a Neural NetworkLegacy edge — source detail unavailable
- DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter→GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language UnderstandingLegacy edge — source detail unavailable
- DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter→BERT: Pre-training of Deep Bidirectional Transformers for Language UnderstandingLegacy edge — source detail unavailable
- A Survey on Knowledge Graphs: Representation, Acquisition, and Applications→WordNetLegacy edge — source detail unavailable
- A Simple Framework for Contrastive Learning of Visual Representations→Unsupervised Data Augmentation for Consistency TrainingLegacy edge — source detail unavailable
- Affordance-Compiled Intelligence: Observable-Only Cognitive Impedance Matching for No-Met…→GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language UnderstandingLegacy edge — source detail unavailable
- Affordance-Compiled Intelligence: Observable-Only Cognitive Impedance Matching for No-Met…→BERT: Pre-training of Deep Bidirectional Transformers for Language UnderstandingLegacy edge — source detail unavailable
- Momentum Contrast for Unsupervised Visual Representation Learning→BERT: Pre-training of Deep Bidirectional Transformers for Language UnderstandingLegacy edge — source detail unavailable
- Momentum Contrast for Unsupervised Visual Representation Learning→A Simple Framework for Contrastive Learning of Visual RepresentationsLegacy edge — source detail unavailable
What this paper cites, inside the corpus
Links
Download BibTeX · Download RIS · DOI · Open access copy · OpenAlex record
Topics
| Topic Modeling | Computer Science |
| Text and Document Classification Technologies | Computer Science |
| Machine Learning in Healthcare | Computer Science |
Is this record sound?
complete
Nothing in this record contradicts itself and no field we check is missing.
- supports3 author record(s) attached.
- supports129 reference(s) recorded.
- neutralThe DOI carries no year to check against.
- supportsA title is present.
Provenance
sha256 7bc26169e93749d1…