Nyström method with Kernel K-means++ samples as landmarks

Oglic, Dino; Gaertner, Thomas

Abstract

We investigate, theoretically and empirically, the effectiveness of kernel K-means++ samples as landmarks in the Nyström method for low-rank approximation of kernel matrices. Previous empirical studies (Zhang et al., 2008; Kumar et al.,2012) observe that the landmarks obtained using (kernel) K-means clustering define a good low-rank approximation of kernel matrices. However, the existing work does not provide a theoretical guarantee on the approximation error for this approach to landmark selection. We close this gap and provide the first bound on the approximation error of the Nystrom method with kernel K-means++ samples as landmarks. Moreover, for the frequently used Gaussian kernel we provide a theoretically sound motivation for performing Lloyd refinements of kernel K-means++ landmarks in the instance space. We substantiate our theoretical results empirically by comparing the approach to several state-of-the-art algorithms.

Citation

Oglic, D., & Gaertner, T. Nyström method with Kernel K-means++ samples as landmarks. Presented at Proceedings of the 34th International Conference on Machine Learning

• Conference Name: Proceedings of the 34th International Conference on Machine Learning
• End Date: Aug 11, 2017
• Acceptance Date: May 12, 2017
• Publication Date: Aug 6, 2017
• Deposit Date: Jun 14, 2017
• Publicly Available Date: Aug 6, 2017
• Journal: Journal of Machine Learning Research
• Print ISSN: 1532-4435
• Electronic ISSN: 1533-7928
• Publisher: Journal of Machine Learning Research
• Peer Reviewed: Peer Reviewed
• Public URL: https://nottingham-repository.worktribe.com/output/876618
• Publisher URL: http://proceedings.mlr.press/v70/oglic17a
• Related Public URLs: https://2017.icml.cc/
• Contract Date: Jun 14, 2017