Variational Inference: A Review for Statisticians
David M. Blei, Alp Kucukelbir low, Jon McAuliffe
One of the core problems of modern statistics is to approximate difficult-to-compute probability densities. This problem is especially important in Bayesian statistics, which frames all inference about unknown quantities as a calculation involving the posterior density. In this article, we review variational inference (VI), a method from machine learning that approximates probability densities through optimization. VI has been used in many applications and tends to be faster than classical methods, such as Markov chain Monte Carlo sampling. The idea behind VI is to first posit a family of densities and then to find a member of that family which is close to the target density. Closeness is measured by Kullback–Leibler divergence. We review the ideas behind mean-field variational inference, discuss the special case of VI applied to exponential family models, present a full example with a Bayesian mixture of Gaussians, and derive a variant that uses stochastic optimization to scale up to massive data. We discuss modern research in VI and highlight important open problems. VI is powerful, but it is not yet well understood. Our hope in writing this article is to catalyze statistical research on this class of algorithms. Supplementary materials for this article are available online.
this paper
works it cites
works citing it
node size = global citations · hover for the full title
Citation edge evidence
- A review of uncertainty quantification in deep learning: Techniques, applications and cha…→Variational Inference: A Review for StatisticiansLegacy edge — source detail unavailable
- dynesty: a dynamic nested sampling package for estimating Bayesian posteriors and evidenc…→Variational Inference: A Review for StatisticiansLegacy edge — source detail unavailable
- Variational Inference: A Review for Statisticians→Latent dirichlet allocationLegacy edge — source detail unavailable
- Variational Inference: A Review for Statisticians→Probabilistic Reasoning in Intelligent Systems: Networks of Plausible InferenceLegacy edge — source detail unavailable
- Variational Inference: A Review for Statisticians→Auto-Encoding Variational BayesLegacy edge — source detail unavailable
- Variational Inference: A Review for Statisticians→Probabilistic topic modelsLegacy edge — source detail unavailable
- Variational Inference: A Review for Statisticians→Bayesian Learning for Neural NetworksLegacy edge — source detail unavailable
- Variational Inference: A Review for Statisticians→Local Computations with Probabilities on Graphical Structures and Their Application to Ex…Legacy edge — source detail unavailable
- Variational Inference: A Review for Statisticians→An Introduction to Variational Methods for Graphical ModelsLegacy edge — source detail unavailable
- Variational Inference: A Review for Statisticians→Graphical Models, Exponential Families, and Variational InferenceLegacy edge — source detail unavailable
- Variational Inference: A Review for Statisticians→Pattern recognition and machine learningLegacy edge — source detail unavailable
- Variational Inference: A Review for Statisticians→Stochastic Backpropagation and Approximate Inference in Deep Generative ModelsLegacy edge — source detail unavailable
- Variational Inference: A Review for Statisticians→A View of the Em Algorithm that Justifies Incremental, Sparse, and other VariantsLegacy edge — source detail unavailable
- Variational Inference: A Review for Statisticians→Bayesian TheoryLegacy edge — source detail unavailable
- Variational Inference: A Review for Statisticians→Introduction to Stochastic Search and OptimizationLegacy edge — source detail unavailable
- Variational Inference: A Review for Statisticians→Mixed Membership Stochastic Blockmodels.Legacy edge — source detail unavailable
- Variational Inference: A Review for Statisticians→Variational inference for Dirichlet process mixturesLegacy edge — source detail unavailable
- Variational Inference: A Review for Statisticians→Bayesian learning via stochastic gradient langevin dynamicsLegacy edge — source detail unavailable
- Variational Inference: A Review for Statisticians→Stochastic variational inferenceLegacy edge — source detail unavailable
- Variational Inference: A Review for Statisticians→Expectation Propagation for approximate Bayesian inferenceLegacy edge — source detail unavailable
- An Introduction to Variational Methods for Graphical Models→Probabilistic Reasoning in Intelligent Systems: Networks of Plausible InferenceLegacy edge — source detail unavailable
- An Introduction to Variational Methods for Graphical Models→A View of the Em Algorithm that Justifies Incremental, Sparse, and other VariantsLegacy edge — source detail unavailable
- Latent dirichlet allocation→An Introduction to Variational Methods for Graphical ModelsLegacy edge — source detail unavailable
- Stochastic Backpropagation and Approximate Inference in Deep Generative Models→Bayesian learning via stochastic gradient langevin dynamicsLegacy edge — source detail unavailable
- Auto-Encoding Variational Bayes→Stochastic variational inferenceLegacy edge — source detail unavailable
- Mixed Membership Stochastic Blockmodels.→An Introduction to Variational Methods for Graphical ModelsLegacy edge — source detail unavailable
- Mixed Membership Stochastic Blockmodels.→Latent dirichlet allocationLegacy edge — source detail unavailable
- Mixed Membership Stochastic Blockmodels.→Graphical Models, Exponential Families, and Variational InferenceLegacy edge — source detail unavailable
- Graphical Models, Exponential Families, and Variational Inference→An Introduction to Variational Methods for Graphical ModelsLegacy edge — source detail unavailable
- Graphical Models, Exponential Families, and Variational Inference→Local Computations with Probabilities on Graphical Structures and Their Application to Ex…Legacy edge — source detail unavailable
- Graphical Models, Exponential Families, and Variational Inference→Latent dirichlet allocationLegacy edge — source detail unavailable
- Graphical Models, Exponential Families, and Variational Inference→Expectation Propagation for approximate Bayesian inferenceLegacy edge — source detail unavailable
- Graphical Models, Exponential Families, and Variational Inference→Variational inference for Dirichlet process mixturesLegacy edge — source detail unavailable
- Graphical Models, Exponential Families, and Variational Inference→Probabilistic Reasoning in Intelligent Systems: Networks of Plausible InferenceLegacy edge — source detail unavailable
- Graphical Models, Exponential Families, and Variational Inference→A View of the Em Algorithm that Justifies Incremental, Sparse, and other VariantsLegacy edge — source detail unavailable
- Variational inference for Dirichlet process mixtures→An Introduction to Variational Methods for Graphical ModelsLegacy edge — source detail unavailable
- Variational inference for Dirichlet process mixtures→Latent dirichlet allocationLegacy edge — source detail unavailable
- Variational inference for Dirichlet process mixtures→Graphical Models, Exponential Families, and Variational InferenceLegacy edge — source detail unavailable
- Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference→Local Computations with Probabilities on Graphical Structures and Their Application to Ex…Legacy edge — source detail unavailable
- Stochastic variational inference→An Introduction to Variational Methods for Graphical ModelsLegacy edge — source detail unavailable
- Stochastic variational inference→Latent dirichlet allocationLegacy edge — source detail unavailable
- Stochastic variational inference→Introduction to Stochastic Search and OptimizationLegacy edge — source detail unavailable
- Stochastic variational inference→Graphical Models, Exponential Families, and Variational InferenceLegacy edge — source detail unavailable
- Stochastic variational inference→Variational inference for Dirichlet process mixturesLegacy edge — source detail unavailable
- Stochastic variational inference→Probabilistic Reasoning in Intelligent Systems: Networks of Plausible InferenceLegacy edge — source detail unavailable
- Stochastic variational inference→Bayesian learning via stochastic gradient langevin dynamicsLegacy edge — source detail unavailable
- Stochastic variational inference→Probabilistic topic modelsLegacy edge — source detail unavailable
- Probabilistic topic models→An Introduction to Variational Methods for Graphical ModelsLegacy edge — source detail unavailable
- Probabilistic topic models→Latent dirichlet allocationLegacy edge — source detail unavailable
- Probabilistic topic models→Graphical Models, Exponential Families, and Variational InferenceLegacy edge — source detail unavailable
- A review of uncertainty quantification in deep learning: Techniques, applications and cha…→An Introduction to Variational Methods for Graphical ModelsLegacy edge — source detail unavailable
- A review of uncertainty quantification in deep learning: Techniques, applications and cha…→Bayesian Learning for Neural NetworksLegacy edge — source detail unavailable
- A review of uncertainty quantification in deep learning: Techniques, applications and cha…→Auto-Encoding Variational BayesLegacy edge — source detail unavailable
- A review of uncertainty quantification in deep learning: Techniques, applications and cha…→Bayesian learning via stochastic gradient langevin dynamicsLegacy edge — source detail unavailable
What this paper cites, inside the corpus
What cites it, inside the corpus
Links
Download BibTeX · Download RIS · DOI · Open access copy · OpenAlex record
Topics
| Gaussian Processes and Bayesian Inference | Computer Science |
| Bayesian Methods and Mixture Models | Computer Science |
| Statistical Methods and Inference | Mathematics |
Is this record sound?
complete
Nothing in this record contradicts itself and no field we check is missing.
- supports3 author record(s) attached.
- supports216 reference(s) recorded.
- neutralThe DOI carries no year to check against.
- supportsA title is present.
Provenance
sha256 86a6d6715ff540a0…