Effective Approaches to Attention-based Neural Machine Translation

Thang Luong; Hieu Pham; Christopher D. Manning

doi:https://doi.org/10.18653/v1/d15-1166

Effective Approaches to Attention-based Neural Machine Translation

2015 · 8,617 citations · 9 from inside this corpus

Thang Luong, Hieu Pham, Christopher D. Manning

An attentional mechanism has lately been used to improve neural machine translation (NMT) by selectively focusing on parts of the source sentence during translation. However, there has been little work exploring useful architectures for attention-based NMT. This paper examines two simple and effective classes of attentional mechanism: a global approach which always attends to all source words and a local one that only looks at a subset of source words at a time. We demonstrate the effectiveness of both approaches on the WMT translation tasks between English and German in both directions. With local attention, we achieve a significant gain of 5.0 BLEU points over non-attentional systems that already incorporate known techniques such as dropout. Our ensemble model using different attention architectures yields a new state-of-the-art result in the WMT'15 English to German translation task with 25.9 BLEU points, an improvement of 1.0 BLEU points over the existing best system backed by NMT and an n-gram reranker. 1

15 of 15 neighbouring works in this corpus. Blue is what this paper cites; orange is what cites it, and a dashed line is one neighbour citing another. Only the largest labels are drawn — every node carries its full title on hover.

this paper works it cites works citing it node size = global citations · hover for the full title

Citation edge evidence

Each row is directed from the citing work to the cited work. Source status remains readable without the interactive graph.

A Comprehensive Survey on Graph Neural Networks→Effective Approaches to Attention-based Neural Machine TranslationLegacy edge — source detail unavailable
[No title in the source record — Edinburgh Research Explorer (University of Edinburgh)]→Effective Approaches to Attention-based Neural Machine TranslationLegacy edge — source detail unavailable
Attention Is All You Need→Effective Approaches to Attention-based Neural Machine TranslationLegacy edge — source detail unavailable
A Comprehensive Survey on Graph Neural Networks→Effective Approaches to Attention-based Neural Machine TranslationLegacy edge — source detail unavailable
fairseq: A Fast, Extensible Toolkit for Sequence Modeling→Effective Approaches to Attention-based Neural Machine TranslationLegacy edge — source detail unavailable
[No title in the source record — Edinburgh Research Explorer (University of Edinburgh)]→Effective Approaches to Attention-based Neural Machine TranslationLegacy edge — source detail unavailable
Relational inductive biases, deep learning, and graph networks→Effective Approaches to Attention-based Neural Machine TranslationLegacy edge — source detail unavailable
Google’s Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation→Effective Approaches to Attention-based Neural Machine TranslationLegacy edge — source detail unavailable
Deep Learning--based Text Classification→Effective Approaches to Attention-based Neural Machine TranslationLegacy edge — source detail unavailable
Effective Approaches to Attention-based Neural Machine Translation→Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Transla…Legacy edge — source detail unavailable
Effective Approaches to Attention-based Neural Machine Translation→BLEULegacy edge — source detail unavailable
Effective Approaches to Attention-based Neural Machine Translation→Neural Machine Translation by Jointly Learning to Align and TranslateLegacy edge — source detail unavailable
Effective Approaches to Attention-based Neural Machine Translation→Sequence to Sequence Learning with Neural NetworksLegacy edge — source detail unavailable
Effective Approaches to Attention-based Neural Machine Translation→Statistical phrase-based translationLegacy edge — source detail unavailable
Effective Approaches to Attention-based Neural Machine Translation→Recurrent Neural Network RegularizationLegacy edge — source detail unavailable
Recurrent Neural Network Regularization→Sequence to Sequence Learning with Neural NetworksLegacy edge — source detail unavailable
Recurrent Neural Network Regularization→Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Transla…Legacy edge — source detail unavailable
Sequence to Sequence Learning with Neural Networks→BLEULegacy edge — source detail unavailable
Neural Machine Translation by Jointly Learning to Align and Translate→Statistical phrase-based translationLegacy edge — source detail unavailable
Statistical phrase-based translation→BLEULegacy edge — source detail unavailable
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Transla…→Statistical phrase-based translationLegacy edge — source detail unavailable
Google’s Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation→Sequence to Sequence Learning with Neural NetworksLegacy edge — source detail unavailable
Google’s Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation→Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Transla…Legacy edge — source detail unavailable
Google’s Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation→[No title in the source record — Edinburgh Research Explorer (University of Edinburgh)]Legacy edge — source detail unavailable
Attention Is All You Need→[No title in the source record — Edinburgh Research Explorer (University of Edinburgh)]Legacy edge — source detail unavailable
Relational inductive biases, deep learning, and graph networks→Attention Is All You NeedLegacy edge — source detail unavailable
A Comprehensive Survey on Graph Neural Networks→Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Transla…Legacy edge — source detail unavailable
fairseq: A Fast, Extensible Toolkit for Sequence Modeling→[No title in the source record — Edinburgh Research Explorer (University of Edinburgh)]Legacy edge — source detail unavailable
[No title in the source record — Edinburgh Research Explorer (University of Edinburgh)]→Sequence to Sequence Learning with Neural NetworksLegacy edge — source detail unavailable
[No title in the source record — Edinburgh Research Explorer (University of Edinburgh)]→Neural Machine Translation by Jointly Learning to Align and TranslateLegacy edge — source detail unavailable
[No title in the source record — Edinburgh Research Explorer (University of Edinburgh)]→Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Transla…Legacy edge — source detail unavailable
[No title in the source record — Edinburgh Research Explorer (University of Edinburgh)]→Sequence to Sequence Learning with Neural NetworksLegacy edge — source detail unavailable
[No title in the source record — Edinburgh Research Explorer (University of Edinburgh)]→Neural Machine Translation by Jointly Learning to Align and TranslateLegacy edge — source detail unavailable
[No title in the source record — Edinburgh Research Explorer (University of Edinburgh)]→Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Transla…Legacy edge — source detail unavailable
[No title in the source record — Edinburgh Research Explorer (University of Edinburgh)]→[No title in the source record — Edinburgh Research Explorer (University of Edinburgh)]Legacy edge — source detail unavailable
A Comprehensive Survey on Graph Neural Networks→Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Transla…Legacy edge — source detail unavailable
A Comprehensive Survey on Graph Neural Networks→Relational inductive biases, deep learning, and graph networksLegacy edge — source detail unavailable

What this paper cites, inside the corpus

Paper	Year	Cited
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Transla…	2014	25,048
BLEU	2001	21,963
Neural Machine Translation by Jointly Learning to Align and Translate	2014	14,615
Sequence to Sequence Learning with Neural Networks	2014	13,351
Statistical phrase-based translation	2003	3,278
Recurrent Neural Network Regularization	2014	2,289

What cites it, inside the corpus

Paper	Year	Cited
A Comprehensive Survey on Graph Neural Networks	2020	9,855
[No title in the source record — Edinburgh Research Explorer (University of Edinburgh)]		7,291
Attention Is All You Need	2025	7,133
A Comprehensive Survey on Graph Neural Networks	2020	3,307
fairseq: A Fast, Extensible Toolkit for Sequence Modeling	2019	2,511
[No title in the source record — Edinburgh Research Explorer (University of Edinburgh)]		2,485
Relational inductive biases, deep learning, and graph networks	2018	2,406
Google’s Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation	2017	1,759
Deep Learning--based Text Classification	2021	1,525

Links

Download BibTeX · Download RIS · DOI · Open access copy · OpenAlex record

Topics

Natural Language Processing Techniques	Computer Science
Topic Modeling	Computer Science
Multimodal Machine Learning Applications	Computer Science

Is this record sound?

complete

Nothing in this record contradicts itself and no field we check is missing.

supports3 author record(s) attached.
supports20 reference(s) recorded.
neutralThe DOI carries no year to check against.
supportsA title is present.

Provenance

Everything above was read from one stored OpenAlex payload, fetched 2026-09-04T03:58:43+00:00.

sha256 5cad55ac5d4d41e1…