A geometric variational approach to Bayesian inference

Saha, Abhijoy; Bharath, Karthik; Kurtek, Sebastian

Abstract

We propose a novel Riemannian geometric framework for variational inference in Bayesian models based on the nonparametric Fisher–Rao metric on the manifold of probability density functions. Under the square-root density representation, the manifold can be identified with the positive orthant of the unit hypersphere S ∞ in L 2 , and the Fisher–Rao metric reduces to the standard L 2 metric. Exploiting such a Riemannian structure, we formulate the task of approximating the posterior distribution as a variational problem on the hypersphere based on the α-divergence. This provides a tighter lower bound on the marginal distribution when compared to, and a corresponding upper bound unavailable with, approaches based on the Kullback–Leibler divergence. We propose a novel gradient-based algorithm for the variational problem based on Fréchet derivative operators motivated by the geometry of S ∞ , and examine its properties. Through simulations and real data applications, we demonstrate the utility of the proposed geometric framework and algorithm on several Bayesian models.

Citation

Saha, A., Bharath, K., & Kurtek, S. (2020). A geometric variational approach to Bayesian inference. Journal of the American Statistical Association, 115(530), 822-835. https://doi.org/10.1080/01621459.2019.1585253

• Journal Article Type: Article
• Acceptance Date: Jan 30, 2019
• Online Publication Date: Mar 26, 2019
• Publication Date: 2020
• Deposit Date: Mar 8, 2019
• Publicly Available Date: Mar 27, 2020
• Journal: Journal of the American Statistical Association
• Print ISSN: 0162-1459
• Electronic ISSN: 1537-274X
• Publisher: Taylor & Francis Open
• Peer Reviewed: Peer Reviewed
• Volume: 115
• Issue: 530
• Pages: 822-835
• DOI: https://doi.org/10.1080/01621459.2019.1585253
• Keywords: Infinite-dimensional Riemannian optimization; Gradient ascent algorithm; Square- root density; Bayesian density estimation; Bayesian logistic regression
• Public URL: https://nottingham-repository.worktribe.com/output/1620463
• Publisher URL: https://www.tandfonline.com/doi/full/10.1080/01621459.2019.1585253
• Additional Information: This is an Accepted Manuscript of an article published by Taylor & Francis in Journal of the American Statistical Association on 26.03.2019 , available online: http://www.tandfonline.com/10.1080/01621459.2019.1585253