Large-Scale Gaussian Processes with Flexible Adaptive Histogram Kernels
Team
Erik Rodner, Alexander Freytag, Paul Bodesheim, Björn Fröhlich
Overview
How to deal with tens of thousands of examples in an exact Bayesian manner?
Problem Statement
Gaussian Processes suffer from several drawbacks in the general formulation
- training as well as hyperparameter optimization is cubically in the number of examples used
- evaluation is linear in the number of examples used
- computation of the classification uncertainty is quadratic in the number of examples used
- the memory demand is quadratic due to the kernel matrix
Summary
We present new methods for fast exact Gaussian process inference including multi-class classification, hyperparameter optimization, and uncertainty prediction in large-scale scenarios. The key observation of our methods is that the inherent properties of parameterized histogram intersection kernels can be exploited efficiently leading to significant time and memory benefits in several domains. In addition, contributions are made considering provable bounds for hyperparameter optimization, the identification of suitable linear solvers, a new active learning strategy, and incremental learning extensions. Evaluations are based on experiments with the large-scale real-world ImageNet database as well as the utilization of our techniques in the area of pixelwise labeling of images and active learning. The results show that inference can be done within microseconds and that every important piece of the Gaussian process framework (e.g., classification, hyperparameter optimization, variance estimation) can be also used in the presence of tens of thousands of examples.
Code
We released the software for our ECCV and ACCV paper on fast Gaussian process inference with histogram intersection kernels on GitHub. The software is written in C++ and requires our computer vision library NICE-core.
Publications
Large-Scale Gaussian Process Inference with Generalized Histogram Intersection Kernels for Visual Recognition Tasks.
International Journal of Computer Vision (IJCV). 121 (2) : pp. 253-280. 2017.
[bibtex] [pdf] [web] [doi] [abstract]
Beyond Classification - Large-scale Gaussian Process Inference and Uncertainty Prediction.
Big Data Meets Computer Vision: First International Workshop on Large Scale Visual Recognition and Retrieval (NIPS-WS). 2012. This workshop article is a short version of our ACCV 2012 paper.
[bibtex] [pdf] [abstract]
Rapid Uncertainty Computation with Gaussian Processes and Histogram Intersection Kernels.
Asian Conference on Computer Vision (ACCV). Pages 511-524. 2012. Best Paper Honorable Mention Award
[bibtex] [pdf] [web] [doi] [presentation] [abstract]
Large-Scale Gaussian Process Classification with Flexible Adaptive Histogram Kernels.
European Conference on Computer Vision (ECCV). Pages 85-98. 2012.
[bibtex] [pdf] [web] [doi] [supplementary] [abstract]