Know What You Don’t Know: Unanswerable Questions for SQuAD
Pranav Rajpurkar, Robin Jia, Percy Liang
Extractive reading comprehension systems can often locate the correct answer to a question in a context document, but they also tend to make unreliable guesses on questions for which the correct answer is not stated in the context. Existing datasets either focus exclusively on answerable questions, or use automatically generated unanswerable questions that are easy to identify. To address these weaknesses, we present SQUADRUN, a new dataset that combines the existing Stanford Question Answering Dataset (SQuAD) with over 50,000 unanswerable questions written adversarially by crowdworkers to look similar to answerable ones. To do well on SQUADRUN, systems must not only answer questions when possible, but also determine when no answer is supported by the paragraph and abstain from answering. SQUADRUN is a challenging natural language understanding task for existing models: a strong neural system that gets 86% F1 on SQuAD achieves only 66% F1 on SQUADRUN. We release SQUADRUN to the community as the successor to SQuAD.
this paper
works it cites
works citing it
node size = global citations · hover for the full title
Citation edge evidence
- HISTORIAE, History of Socio-Cultural Transformation as Linguistic Data Science. A Humanit…→Know What You Don’t Know: Unanswerable Questions for SQuADLegacy edge — source detail unavailable
- ALBERT: A Lite BERT for Self-supervised Learning of Language\n Representations→Know What You Don’t Know: Unanswerable Questions for SQuADLegacy edge — source detail unavailable
- Language Models are Few-Shot Learners→Know What You Don’t Know: Unanswerable Questions for SQuADLegacy edge — source detail unavailable
- On the Opportunities and Risks of Foundation Models→Know What You Don’t Know: Unanswerable Questions for SQuADLegacy edge — source detail unavailable
- Natural Questions: A Benchmark for Question Answering Research→Know What You Don’t Know: Unanswerable Questions for SQuADLegacy edge — source detail unavailable
- XLNet: Generalized Autoregressive Pretraining for Language Understanding→Know What You Don’t Know: Unanswerable Questions for SQuADLegacy edge — source detail unavailable
- HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering→Know What You Don’t Know: Unanswerable Questions for SQuADLegacy edge — source detail unavailable
- TinyBERT: Distilling BERT for Natural Language Understanding→Know What You Don’t Know: Unanswerable Questions for SQuADLegacy edge — source detail unavailable
- Deep Learning--based Text Classification→Know What You Don’t Know: Unanswerable Questions for SQuADLegacy edge — source detail unavailable
- Know What You Don’t Know: Unanswerable Questions for SQuAD→SQuAD: 100,000+ Questions for Machine Comprehension of TextLegacy edge — source detail unavailable
- Know What You Don’t Know: Unanswerable Questions for SQuAD→Teaching Machines to Read and ComprehendLegacy edge — source detail unavailable
- Know What You Don’t Know: Unanswerable Questions for SQuAD→Teaching Machines to Read and ComprehendLegacy edge — source detail unavailable
- HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering→SQuAD: 100,000+ Questions for Machine Comprehension of TextLegacy edge — source detail unavailable
- Natural Questions: A Benchmark for Question Answering Research→HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question AnsweringLegacy edge — source detail unavailable
- Natural Questions: A Benchmark for Question Answering Research→Teaching Machines to Read and ComprehendLegacy edge — source detail unavailable
- Natural Questions: A Benchmark for Question Answering Research→SQuAD: 100,000+ Questions for Machine Comprehension of TextLegacy edge — source detail unavailable
- XLNet: Generalized Autoregressive Pretraining for Language Understanding→HISTORIAE, History of Socio-Cultural Transformation as Linguistic Data Science. A Humanit…Legacy edge — source detail unavailable
- SQuAD: 100,000+ Questions for Machine Comprehension of Text→Teaching Machines to Read and ComprehendLegacy edge — source detail unavailable
- HISTORIAE, History of Socio-Cultural Transformation as Linguistic Data Science. A Humanit…→SQuAD: 100,000+ Questions for Machine Comprehension of TextLegacy edge — source detail unavailable
- ALBERT: A Lite BERT for Self-supervised Learning of Language\n Representations→XLNet: Generalized Autoregressive Pretraining for Language UnderstandingLegacy edge — source detail unavailable
- ALBERT: A Lite BERT for Self-supervised Learning of Language\n Representations→SQuAD: 100,000+ Questions for Machine Comprehension of TextLegacy edge — source detail unavailable
- ALBERT: A Lite BERT for Self-supervised Learning of Language\n Representations→HISTORIAE, History of Socio-Cultural Transformation as Linguistic Data Science. A Humanit…Legacy edge — source detail unavailable
- Language Models are Few-Shot Learners→Natural Questions: A Benchmark for Question Answering ResearchLegacy edge — source detail unavailable
- Language Models are Few-Shot Learners→HISTORIAE, History of Socio-Cultural Transformation as Linguistic Data Science. A Humanit…Legacy edge — source detail unavailable
- Language Models are Few-Shot Learners→TinyBERT: Distilling BERT for Natural Language UnderstandingLegacy edge — source detail unavailable
- TinyBERT: Distilling BERT for Natural Language Understanding→SQuAD: 100,000+ Questions for Machine Comprehension of TextLegacy edge — source detail unavailable
- TinyBERT: Distilling BERT for Natural Language Understanding→HISTORIAE, History of Socio-Cultural Transformation as Linguistic Data Science. A Humanit…Legacy edge — source detail unavailable
- TinyBERT: Distilling BERT for Natural Language Understanding→ALBERT: A Lite BERT for Self-supervised Learning of Language\n RepresentationsLegacy edge — source detail unavailable
- Deep Learning--based Text Classification→SQuAD: 100,000+ Questions for Machine Comprehension of TextLegacy edge — source detail unavailable
- Deep Learning--based Text Classification→HISTORIAE, History of Socio-Cultural Transformation as Linguistic Data Science. A Humanit…Legacy edge — source detail unavailable
- Deep Learning--based Text Classification→ALBERT: A Lite BERT for Self-supervised Learning of Language\n RepresentationsLegacy edge — source detail unavailable
- On the Opportunities and Risks of Foundation Models→SQuAD: 100,000+ Questions for Machine Comprehension of TextLegacy edge — source detail unavailable
- On the Opportunities and Risks of Foundation Models→HISTORIAE, History of Socio-Cultural Transformation as Linguistic Data Science. A Humanit…Legacy edge — source detail unavailable
- On the Opportunities and Risks of Foundation Models→ALBERT: A Lite BERT for Self-supervised Learning of Language\n RepresentationsLegacy edge — source detail unavailable
- On the Opportunities and Risks of Foundation Models→Language Models are Few-Shot LearnersLegacy edge — source detail unavailable
What this paper cites, inside the corpus
| Paper | Year | Cited |
|---|---|---|
| SQuAD: 100,000+ Questions for Machine Comprehension of Text | 2016 | 6,435 |
| Teaching Machines to Read and Comprehend | 2015 | 1,939 |
| Teaching Machines to Read and Comprehend | 2015 | 1,519 |
What cites it, inside the corpus
Links
Download BibTeX · Download RIS · DOI · Open access copy · OpenAlex record
Topics
| Topic Modeling | Computer Science |
| Natural Language Processing Techniques | Computer Science |
| Scientific Computing and Data Management | Decision Sciences |
Is this record sound?
complete
Nothing in this record contradicts itself and no field we check is missing.
- supports3 author record(s) attached.
- supports28 reference(s) recorded.
- neutralThe DOI carries no year to check against.
- supportsA title is present.
Provenance
sha256 86a6d6715ff540a0…