Who Cited It

ERNIE: Enhanced Language Representation with Informative Entities

2019 · 1,435 citations · 4 from inside this corpus

Zhengyan Zhang, Xu Han, Zhiyuan Liu, Xin Jiang, Maosong Sun

Neural language representation models such as BERT pre-trained on large-scale corpora can well capture rich semantic patterns from plain text, and be fine-tuned to consistently improve the performance of various NLP tasks. However, the existing pre-trained language models rarely consider incorporating knowledge graphs (KGs), which can provide rich structured knowledge facts for better language understanding. We argue that informative entities in KGs can enhance language representation with external knowledge. In this paper, we utilize both large-scale textual corpora and KGs to train an enhanced language representation model (ERNIE), which can take full advantage of lexical, syntactic, and knowledge information simultaneously. The experimental results have demonstrated that ERNIE achieves significant improvements on various knowledge-driven tasks, and meanwhile is comparable with the state-of-the-art model BERT on other common NLP tasks. The source code and experiment details of this paper can be obtained from https:// github.com/thunlp/ERNIE.

ERNIE: Enhanced Language Representation with Informative Entities (2019)ERNIE: Enhanced Language Repr…Exploiting Generative AI to Scale up Intelligent Tutoring Systems (2023)Exploiting Generative AI to S…AI-Assisted Pipeline for Dynamic Generation of Trustworthy Health Supplement Content at S… (2018)AI-Assisted Pipeline for Dyna…Glove: Global Vectors for Word Representation (2014)Glove: Global Vectors for Wor…BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding (2019)BERT: Pre-training of Deep Bi…Distributed Representations of Words and Phrases and their Compositionality (2013)Distributed Representations o…Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank (2013)Recursive Deep Models for Sem…SQuAD: 100,000+ Questions for Machine Comprehension of Text (2016)SQuAD: 100,000+ Questions for…A unified architecture for natural language processing (2008)A unified architecture for na…Translating Embeddings for Modeling Multi-relational Data (2013)Translating Embeddings for Mo…GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding (2018)GLUE: A Multi-Task Benchmark …Gaussian Error Linear Units (GELUs) (2016)Gaussian Error Linear Units (…Word Representations: A Simple and General Method for Semi-Supervised Learning (2010)Word Representations: A Simpl…The PASCAL Recognising Textual Entailment Challenge (2006)The PASCAL Recognising Textua…Introduction to the CoNLL-2002 Shared Task: Language-Independent Named Entity Recognition (2002)Introduction to the CoNLL-200…A Survey on Knowledge Graphs: Representation, Acquisition, and Applications (2021)A Survey on Knowledge Graphs:…On the Opportunities and Risks of Foundation Models (2021)A Survey on Deep Learning for Named Entity Recognition (2020)A Survey on Deep Learning for…Pre-trained models for natural language processing: A survey (2020)Pre-trained models for natura…
18 of 18 neighbouring works in this corpus. Blue is what this paper cites; orange is what cites it, and a dashed line is one neighbour citing another. Only the largest labels are drawn — every node carries its full title on hover.
this paper works it cites works citing it node size = global citations · hover for the full title

Citation edge evidence

Each row is directed from the citing work to the cited work. Source status remains readable without the interactive graph.

What this paper cites, inside the corpus

PaperYearCited
Exploiting Generative AI to Scale up Intelligent Tutoring Systems202379,071
AI-Assisted Pipeline for Dynamic Generation of Trustworthy Health Supplement Content at S…201846,036
Glove: Global Vectors for Word Representation201434,067
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding201933,416
Distributed Representations of Words and Phrases and their Compositionality201318,054
Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank20136,849
SQuAD: 100,000+ Questions for Machine Comprehension of Text20166,435
A unified architecture for natural language processing20085,207
Translating Embeddings for Modeling Multi-relational Data20135,188
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding20184,055
Gaussian Error Linear Units (GELUs)20163,197
Word Representations: A Simple and General Method for Semi-Supervised Learning20101,941
The PASCAL Recognising Textual Entailment Challenge20061,672
Introduction to the CoNLL-2002 Shared Task: Language-Independent Named Entity Recognition20021,583

What cites it, inside the corpus

PaperYearCited
A Survey on Knowledge Graphs: Representation, Acquisition, and Applications20212,862
On the Opportunities and Risks of Foundation Models20212,265
A Survey on Deep Learning for Named Entity Recognition20201,534
Pre-trained models for natural language processing: A survey20201,521

Links

Download BibTeX · Download RIS · DOI · Open access copy · OpenAlex record

Topics

Topic ModelingComputer Science
Natural Language Processing TechniquesComputer Science
Text Readability and SimplificationComputer Science

Is this record sound?

complete

Nothing in this record contradicts itself and no field we check is missing.

  • supports5 author record(s) attached.
  • supports59 reference(s) recorded.
  • neutralThe DOI carries no year to check against.
  • supportsA title is present.

Provenance

Everything above was read from one stored OpenAlex payload, fetched 2026-09-04T03:58:59+00:00.

sha256 59007f057c7dae86…