Reinforcement Learning: A Survey

Leslie Pack Kaelbling; Michael L. Littman; Andrew Moore

doi:https://doi.org/10.1613/jair.301

Reinforcement Learning: A Survey

1996 · Journal of Artificial Intelligence Research · 8,953 citations · 12 from inside this corpus

Leslie Pack Kaelbling low, Michael L. Littman, Andrew Moore

This paper surveys the field of reinforcement learning from a computer-science perspective. It is written to be accessible to researchers familiar with machine learning. Both the historical basis of the field and a broad selection of current work are summarized. Reinforcement learning is the problem faced by an agent that learns behavior through trial-and-error interactions with a dynamic environment. The work described here has a resemblance to work in psychology, but differs considerably in the details and in the use of the word ``reinforcement.'' The paper discusses central issues of reinforcement learning, including trading off exploration and exploitation, establishing the foundations of the field via Markov decision theory, learning from delayed reinforcement, constructing empirical models to accelerate learning, making use of generalization and hierarchy, and coping with hidden state. It concludes with a survey of some implemented systems and an assessment of the practical utility of current methods for reinforcement learning.

29 of 29 neighbouring works in this corpus. Blue is what this paper cites; orange is what cites it, and a dashed line is one neighbour citing another. Only the largest labels are drawn — every node carries its full title on hover.

this paper works it cites works citing it node size = global citations · hover for the full title

Citation edge evidence

Each row is directed from the citing work to the cited work. Source status remains readable without the interactive graph.

Deep learning in neural networks: An overview→Reinforcement Learning: A SurveyLegacy edge — source detail unavailable
Ant colony system: a cooperative learning approach to the traveling salesman problem→Reinforcement Learning: A SurveyLegacy edge — source detail unavailable
Deep Reinforcement Learning with Double Q-Learning→Reinforcement Learning: A SurveyLegacy edge — source detail unavailable
Machine Learning: Algorithms, Real-World Applications and Research Directions→Reinforcement Learning: A SurveyLegacy edge — source detail unavailable
Deep Reinforcement Learning with Double Q-Learning→Reinforcement Learning: A SurveyLegacy edge — source detail unavailable
Evolving Neural Networks through Augmenting Topologies→Reinforcement Learning: A SurveyLegacy edge — source detail unavailable
Reinforcement learning in robotics: A survey→Reinforcement Learning: A SurveyLegacy edge — source detail unavailable
Deep Learning: A Comprehensive Overview on Techniques, Taxonomy, Applications and Researc…→Reinforcement Learning: A SurveyLegacy edge — source detail unavailable
A Comprehensive Survey of Multiagent Reinforcement Learning→Reinforcement Learning: A SurveyLegacy edge — source detail unavailable
Transfer Learning for Reinforcement Learning Domains: A Survey→Reinforcement Learning: A SurveyLegacy edge — source detail unavailable
A Survey on Deep Learning for Named Entity Recognition→Reinforcement Learning: A SurveyLegacy edge — source detail unavailable
Adaptive Computation and Machine Learning→Reinforcement Learning: A SurveyLegacy edge — source detail unavailable
Reinforcement Learning: A Survey→Genetic algorithms in search, optimization, and machine learningLegacy edge — source detail unavailable
Reinforcement Learning: A Survey→Adaptation in Natural and Artificial SystemsLegacy edge — source detail unavailable
Reinforcement Learning: A Survey→Genetic Algorithms in Search, Optimization and Machine LearningLegacy edge — source detail unavailable
Reinforcement Learning: A Survey→Q-learningLegacy edge — source detail unavailable
Reinforcement Learning: A Survey→Markov Decision Processes: Discrete Stochastic Dynamic Programming.Legacy edge — source detail unavailable
Reinforcement Learning: A Survey→Simple statistical gradient-following algorithms for connectionist reinforcement learningLegacy edge — source detail unavailable
Reinforcement Learning: A Survey→Learning from Delayed RewardsLegacy edge — source detail unavailable
Reinforcement Learning: A Survey→Some Studies in Machine Learning Using the Game of CheckersLegacy edge — source detail unavailable
Reinforcement Learning: A Survey→Learning to Predict by the Methods of Temporal DifferencesLegacy edge — source detail unavailable
Reinforcement Learning: A Survey→Dynamic Programming and Markov Processes.Legacy edge — source detail unavailable
Reinforcement Learning: A Survey→Markov games as a framework for multi-agent reinforcement learningLegacy edge — source detail unavailable
Reinforcement Learning: A Survey→A New Approach to Manipulator Control: The Cerebellar Model Articulation Controller (CMAC)Legacy edge — source detail unavailable
Reinforcement Learning: A Survey→Markov Decision Processes: Discrete Stochastic Dynamic ProgrammingLegacy edge — source detail unavailable
Reinforcement Learning: A Survey→Learning Automata: An IntroductionLegacy edge — source detail unavailable
Reinforcement Learning: A Survey→Some studies in machine learning using the game of checkersLegacy edge — source detail unavailable
Reinforcement Learning: A Survey→Classifier Fitness Based on AccuracyLegacy edge — source detail unavailable
Reinforcement Learning: A Survey→Integrated Architectures for Learning, Planning, and Reacting Based on Approximating Dyna…Legacy edge — source detail unavailable
Integrated Architectures for Learning, Planning, and Reacting Based on Approximating Dyna…→Learning to Predict by the Methods of Temporal DifferencesLegacy edge — source detail unavailable
Markov games as a framework for multi-agent reinforcement learning→Dynamic Programming and Markov Processes.Legacy edge — source detail unavailable
Markov games as a framework for multi-agent reinforcement learning→Some Studies in Machine Learning Using the Game of CheckersLegacy edge — source detail unavailable
Markov games as a framework for multi-agent reinforcement learning→Q-learningLegacy edge — source detail unavailable
Reinforcement learning in robotics: A survey→Integrated Architectures for Learning, Planning, and Reacting Based on Approximating Dyna…Legacy edge — source detail unavailable
Reinforcement learning in robotics: A survey→Markov Decision Processes: Discrete Stochastic Dynamic Programming.Legacy edge — source detail unavailable
Reinforcement learning in robotics: A survey→Simple statistical gradient-following algorithms for connectionist reinforcement learningLegacy edge — source detail unavailable
Reinforcement learning in robotics: A survey→Adaptive Computation and Machine LearningLegacy edge — source detail unavailable
Classifier Fitness Based on Accuracy→A New Approach to Manipulator Control: The Cerebellar Model Articulation Controller (CMAC)Legacy edge — source detail unavailable
Classifier Fitness Based on Accuracy→Q-learningLegacy edge — source detail unavailable
Deep learning in neural networks: An overview→Adaptation in Natural and Artificial SystemsLegacy edge — source detail unavailable
Deep learning in neural networks: An overview→Evolving Neural Networks through Augmenting TopologiesLegacy edge — source detail unavailable
Deep learning in neural networks: An overview→Simple statistical gradient-following algorithms for connectionist reinforcement learningLegacy edge — source detail unavailable
Deep learning in neural networks: An overview→Some studies in machine learning using the game of checkersLegacy edge — source detail unavailable
Deep learning in neural networks: An overview→Learning from Delayed RewardsLegacy edge — source detail unavailable
Deep learning in neural networks: An overview→Genetic algorithms in search, optimization, and machine learningLegacy edge — source detail unavailable
Deep learning in neural networks: An overview→Some Studies in Machine Learning Using the Game of CheckersLegacy edge — source detail unavailable
Deep learning in neural networks: An overview→Q-learningLegacy edge — source detail unavailable
Transfer Learning for Reinforcement Learning Domains: A Survey→Learning to Predict by the Methods of Temporal DifferencesLegacy edge — source detail unavailable
Transfer Learning for Reinforcement Learning Domains: A Survey→Markov Decision Processes: Discrete Stochastic Dynamic Programming.Legacy edge — source detail unavailable
Transfer Learning for Reinforcement Learning Domains: A Survey→Simple statistical gradient-following algorithms for connectionist reinforcement learningLegacy edge — source detail unavailable
Transfer Learning for Reinforcement Learning Domains: A Survey→Learning from Delayed RewardsLegacy edge — source detail unavailable
A Comprehensive Survey of Multiagent Reinforcement Learning→Integrated Architectures for Learning, Planning, and Reacting Based on Approximating Dyna…Legacy edge — source detail unavailable
A Comprehensive Survey of Multiagent Reinforcement Learning→Markov games as a framework for multi-agent reinforcement learningLegacy edge — source detail unavailable
A Comprehensive Survey of Multiagent Reinforcement Learning→Learning to Predict by the Methods of Temporal DifferencesLegacy edge — source detail unavailable
A Comprehensive Survey of Multiagent Reinforcement Learning→Markov Decision Processes: Discrete Stochastic Dynamic Programming.Legacy edge — source detail unavailable
A Comprehensive Survey of Multiagent Reinforcement Learning→Q-learningLegacy edge — source detail unavailable
Learning to Predict by the Methods of Temporal Differences→Some studies in machine learning using the game of checkersLegacy edge — source detail unavailable
Learning to Predict by the Methods of Temporal Differences→Some Studies in Machine Learning Using the Game of CheckersLegacy edge — source detail unavailable
Evolving Neural Networks through Augmenting Topologies→Genetic Algorithms in Search, Optimization and Machine LearningLegacy edge — source detail unavailable
Simple statistical gradient-following algorithms for connectionist reinforcement learning→Learning Automata: An IntroductionLegacy edge — source detail unavailable
Simple statistical gradient-following algorithms for connectionist reinforcement learning→Learning to Predict by the Methods of Temporal DifferencesLegacy edge — source detail unavailable
Simple statistical gradient-following algorithms for connectionist reinforcement learning→Learning from Delayed RewardsLegacy edge — source detail unavailable
Ant colony system: a cooperative learning approach to the traveling salesman problem→Learning from Delayed RewardsLegacy edge — source detail unavailable
Deep Reinforcement Learning with Double Q-Learning→Integrated Architectures for Learning, Planning, and Reacting Based on Approximating Dyna…Legacy edge — source detail unavailable
Deep Reinforcement Learning with Double Q-Learning→Learning to Predict by the Methods of Temporal DifferencesLegacy edge — source detail unavailable
Deep Reinforcement Learning with Double Q-Learning→Learning from Delayed RewardsLegacy edge — source detail unavailable
Deep Reinforcement Learning with Double Q-Learning→Integrated Architectures for Learning, Planning, and Reacting Based on Approximating Dyna…Legacy edge — source detail unavailable
Deep Reinforcement Learning with Double Q-Learning→Learning from Delayed RewardsLegacy edge — source detail unavailable
Machine Learning: Algorithms, Real-World Applications and Research Directions→Markov Decision Processes: Discrete Stochastic Dynamic Programming.Legacy edge — source detail unavailable
Machine Learning: Algorithms, Real-World Applications and Research Directions→Markov Decision Processes: Discrete Stochastic Dynamic ProgrammingLegacy edge — source detail unavailable
Deep Learning: A Comprehensive Overview on Techniques, Taxonomy, Applications and Researc…→Markov Decision Processes: Discrete Stochastic Dynamic Programming.Legacy edge — source detail unavailable
Deep Learning: A Comprehensive Overview on Techniques, Taxonomy, Applications and Researc…→Machine Learning: Algorithms, Real-World Applications and Research DirectionsLegacy edge — source detail unavailable
Q-learning→Integrated Architectures for Learning, Planning, and Reacting Based on Approximating Dyna…Legacy edge — source detail unavailable
Q-learning→Learning from Delayed RewardsLegacy edge — source detail unavailable

What this paper cites, inside the corpus

Paper	Year	Cited
Genetic algorithms in search, optimization, and machine learning	1989	49,344
Adaptation in Natural and Artificial Systems	1992	35,569
Genetic Algorithms in Search, Optimization and Machine Learning	1988	17,777
Q-learning	1992	9,099
Markov Decision Processes: Discrete Stochastic Dynamic Programming.	1995	8,450
Simple statistical gradient-following algorithms for connectionist reinforcement learning	1992	7,519
Learning from Delayed Rewards	1989	5,473
Some Studies in Machine Learning Using the Game of Checkers	1959	4,416
Learning to Predict by the Methods of Temporal Differences	1988	3,970
Dynamic Programming and Markov Processes.	1961	3,461
Markov games as a framework for multi-agent reinforcement learning	1994	2,715
A New Approach to Manipulator Control: The Cerebellar Model Articulation Controller (CMAC)	1975	2,176
Markov Decision Processes: Discrete Stochastic Dynamic Programming	1995	1,898
Learning Automata: An Introduction	1989	1,561
Some studies in machine learning using the game of checkers	2000	1,449
Classifier Fitness Based on Accuracy	1995	1,396
Integrated Architectures for Learning, Planning, and Reacting Based on Approximating Dyna…	1990	1,375

What cites it, inside the corpus

Paper	Year	Cited
Deep learning in neural networks: An overview	2014	18,236
Ant colony system: a cooperative learning approach to the traveling salesman problem	1997	8,077
Deep Reinforcement Learning with Double Q-Learning	2016	6,271
Machine Learning: Algorithms, Real-World Applications and Research Directions	2021	5,368
Deep Reinforcement Learning with Double Q-Learning	2016	3,516
Evolving Neural Networks through Augmenting Topologies	2002	3,298
Reinforcement learning in robotics: A survey	2013	3,154
Deep Learning: A Comprehensive Overview on Techniques, Taxonomy, Applications and Researc…	2021	2,537
A Comprehensive Survey of Multiagent Reinforcement Learning	2008	2,256
Transfer Learning for Reinforcement Learning Domains: A Survey	2009	1,564
A Survey on Deep Learning for Named Entity Recognition	2020	1,534
Adaptive Computation and Machine Learning	2012	1,437

Links

Download BibTeX · Download RIS · DOI · Open access copy · OpenAlex record

Topics

Reinforcement Learning in Robotics	Computer Science
Evolutionary Algorithms and Applications	Computer Science
Metaheuristic Optimization Algorithms Research	Computer Science

Is this record sound?

complete

Nothing in this record contradicts itself and no field we check is missing.

supports3 author record(s) attached.
supports189 reference(s) recorded.
neutralThe DOI carries no year to check against.
supportsA title is present.

Provenance

Everything above was read from one stored OpenAlex payload, fetched 2026-09-04T03:58:43+00:00.

sha256 5cad55ac5d4d41e1…