Longformer: The Long-Document Transformer

Iz Beltagy; Matthew E. Peters; Arman Cohan

doi:https://doi.org/10.48550/arxiv.2004.05150

Longformer: The Long-Document Transformer

2020 · arXiv (Cornell University) · 2,206 citations · 4 from inside this corpus

Iz Beltagy, Matthew E. Peters, Arman Cohan

The source holds an abstract for this work, but its best open-access copy is under no open licence, which does not permit us to republish the text. Read it at the source below.

11 of 11 neighbouring works in this corpus. Blue is what this paper cites; orange is what cites it, and a dashed line is one neighbour citing another. Only the largest labels are drawn — every node carries its full title on hover.

this paper works it cites works citing it node size = global citations · hover for the full title

Citation edge evidence

Each row is directed from the citing work to the cited work. Source status remains readable without the interactive graph.

Transformers: State-of-the-Art Natural Language Processing→Longformer: The Long-Document TransformerLegacy edge — source detail unavailable
Survey of Hallucination in Natural Language Generation→Longformer: The Long-Document TransformerLegacy edge — source detail unavailable
HuggingFace's Transformers: State-of-the-art Natural Language Processing→Longformer: The Long-Document TransformerLegacy edge — source detail unavailable
On the Opportunities and Risks of Foundation Models→Longformer: The Long-Document TransformerLegacy edge — source detail unavailable
Longformer: The Long-Document Transformer→BERT: Pre-training of Deep Bidirectional Transformers for Language UnderstandingLegacy edge — source detail unavailable
Longformer: The Long-Document Transformer→HISTORIAE, History of Socio-Cultural Transformation as Linguistic Data Science. A Humanit…Legacy edge — source detail unavailable
Longformer: The Long-Document Transformer→Sequence to Sequence Learning with Neural NetworksLegacy edge — source detail unavailable
Longformer: The Long-Document Transformer→Learning Word Vectors for Sentiment AnalysisLegacy edge — source detail unavailable
Longformer: The Long-Document Transformer→Transformer-XL: Attentive Language Models beyond a Fixed-Length ContextLegacy edge — source detail unavailable
Longformer: The Long-Document Transformer→HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question AnsweringLegacy edge — source detail unavailable
Longformer: The Long-Document Transformer→Unsupervised Data Augmentation for Consistency TrainingLegacy edge — source detail unavailable
Unsupervised Data Augmentation for Consistency Training→Learning Word Vectors for Sentiment AnalysisLegacy edge — source detail unavailable
Unsupervised Data Augmentation for Consistency Training→BERT: Pre-training of Deep Bidirectional Transformers for Language UnderstandingLegacy edge — source detail unavailable
Transformer-XL: Attentive Language Models beyond a Fixed-Length Context→BERT: Pre-training of Deep Bidirectional Transformers for Language UnderstandingLegacy edge — source detail unavailable
HISTORIAE, History of Socio-Cultural Transformation as Linguistic Data Science. A Humanit…→BERT: Pre-training of Deep Bidirectional Transformers for Language UnderstandingLegacy edge — source detail unavailable
Transformers: State-of-the-Art Natural Language Processing→Transformer-XL: Attentive Language Models beyond a Fixed-Length ContextLegacy edge — source detail unavailable
Transformers: State-of-the-Art Natural Language Processing→HISTORIAE, History of Socio-Cultural Transformation as Linguistic Data Science. A Humanit…Legacy edge — source detail unavailable
HuggingFace's Transformers: State-of-the-art Natural Language Processing→HISTORIAE, History of Socio-Cultural Transformation as Linguistic Data Science. A Humanit…Legacy edge — source detail unavailable
On the Opportunities and Risks of Foundation Models→BERT: Pre-training of Deep Bidirectional Transformers for Language UnderstandingLegacy edge — source detail unavailable
On the Opportunities and Risks of Foundation Models→Transformer-XL: Attentive Language Models beyond a Fixed-Length ContextLegacy edge — source detail unavailable
On the Opportunities and Risks of Foundation Models→HISTORIAE, History of Socio-Cultural Transformation as Linguistic Data Science. A Humanit…Legacy edge — source detail unavailable
On the Opportunities and Risks of Foundation Models→Transformers: State-of-the-Art Natural Language ProcessingLegacy edge — source detail unavailable

What this paper cites, inside the corpus

Paper	Year	Cited
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding	2019	33,416
HISTORIAE, History of Socio-Cultural Transformation as Linguistic Data Science. A Humanit…	2019	17,489
Sequence to Sequence Learning with Neural Networks	2014	13,351
Learning Word Vectors for Sentiment Analysis	2011	3,294
Transformer-XL: Attentive Language Models beyond a Fixed-Length Context	2019	3,202
HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering	2018	1,743
Unsupervised Data Augmentation for Consistency Training	2019	1,624

What cites it, inside the corpus

Paper	Year	Cited
Transformers: State-of-the-Art Natural Language Processing	2020	8,295
Survey of Hallucination in Natural Language Generation	2022	4,279
HuggingFace's Transformers: State-of-the-art Natural Language Processing	2019	3,147
On the Opportunities and Risks of Foundation Models	2021	2,265

Links

Download BibTeX · Download RIS · DOI · Open access copy · OpenAlex record

Topics

Topic Modeling	Computer Science
Natural Language Processing Techniques	Computer Science
Computational Physics and Python Applications	Computer Science

Is this record sound?

partial

One field of this record is missing or disagrees with another. What is shown below is what the source publishes.

supports3 author record(s) attached.
supports54 reference(s) recorded.
weakensThe DOI names 2004 but the record dates this to 2,020. One of the two is about a different paper.
supportsA title is present.

Provenance

Everything above was read from one stored OpenAlex payload, fetched 2026-09-04T03:58:50+00:00.

sha256 86a6d6715ff540a0…