Large-Scale Gaussian Processes with Flexible Adaptive Histogram Kernels
Team
Erik Rodner, Alexander Freytag, Paul Bodesheim, Björn Fröhlich
Overview
How to deal with tens of thousands of examples in an exact Bayesian manner?
Problem Statement
Gaussian Processes suffer from several drawbacks in the general formulation

  • training as well as hyperparameter optimization is cubically in the number of examples used
  • evaluation is linear in the number of examples used
  • computation of the classification uncertainty is quadratic in the number of examples used
  • the memory demand is quadratic due to the kernel matrix
Summary

We present new methods for fast exact Gaussian process inference including multi-class classification, hyperparameter optimization, and uncertainty prediction in large-scale scenarios. The key observation of our methods is that the inherent properties of parameterized histogram intersection kernels can be exploited efficiently leading to significant time and memory benefits in several domains. In addition, contributions are made considering provable bounds for hyperparameter optimization, the identification of suitable linear solvers, a new active learning strategy, and incremental learning extensions. Evaluations are based on experiments with the large-scale real-world ImageNet database as well as the utilization of our techniques in the area of pixelwise labeling of images and active learning. The results show that inference can be done within microseconds and that every important piece of the Gaussian process framework (e.g., classification, hyperparameter optimization, variance estimation) can be also used in the presence of tens of thousands of examples.

Code

We released the software for our ECCV and ACCV paper on fast Gaussian process inference with histogram intersection kernels on GitHub. The software is written in C++ and requires our computer vision library NICE-core.

Publications
2017
Large-Scale Gaussian Process Inference with Generalized Histogram Intersection Kernels for Visual Recognition Tasks
Erik Rodner and Alexander Freytag and Paul Bodesheim and Björn Fröhlich and Joachim Denzler.
International Journal of Computer Vision (IJCV). 121 (2) : pp. 253-280. 2017.
[bibtex] [pdf] [web] [abstract]
2013
Large-Scale Gaussian Process Multi-Class Classification for Semantic Segmentation and Facade Recognition
Björn Fröhlich and Erik Rodner and Michael Kemmler and Joachim Denzler.
Machine Vision and Applications. 24 (5) : pp. 1043-1053. 2013.
[bibtex] [pdf]
2012
Efficient Semantic Segmentation with Gaussian Processes and Histogram Intersection Kernels
Alexander Freytag and Björn Fröhlich and Erik Rodner and Joachim Denzler.
International Conference on Pattern Recognition (ICPR). Pages 3313-3316. 2012.
[bibtex] [pdf]
Beyond Classification - Large-scale Gaussian Process Inference and Uncertainty Prediction
Alexander Freytag and Erik Rodner and Paul Bodesheim and Joachim Denzler.
Big Data Meets Computer Vision: First International Workshop on Large Scale Visual Recognition and Retrieval (NIPS-WS). 2012. This workshop article is a short version of our ACCV 2012 paper.
[bibtex] [pdf] [abstract]
Rapid Uncertainty Computation with Gaussian Processes and Histogram Intersection Kernels
Alexander Freytag and Erik Rodner and Paul Bodesheim and Joachim Denzler.
Asian Conference on Computer Vision (ACCV). Pages 511-524. 2012. Best Paper Honorable Mention Award
[bibtex] [pdf] [web] [presentation] [abstract]
Large-Scale Gaussian Process Classification using Random Decision Forests
Björn Fröhlich and Erik Rodner and Michael Kemmler and Joachim Denzler.
Pattern Recognition and Image Analysis. Advances in Mathematical Theory and Applications (PRIA). 22 (1) : pp. 113-120. 2012.
[bibtex] [pdf]
Large-Scale Gaussian Process Classification with Flexible Adaptive Histogram Kernels
Erik Rodner and Alexander Freytag and Paul Bodesheim and Joachim Denzler.
European Conference on Computer Vision (ECCV). Pages 85-98. 2012.
[bibtex] [pdf] [web] [supplementary] [abstract]
2011
Efficient Gaussian process classification using random decision forests
Björn Fröhlich and Erik Rodner and Michael Kemmler and Joachim Denzler.
Pattern Recognition and Image Analysis. Advances in Mathematical Theory and Applications (PRIA). 21 : pp. 184-187. 2011. 10.1134/S1054661811020337
[bibtex] [pdf]
2010
Efficient Gaussian Process Classification using Random Decision Forests
Björn Fröhlich and Erik Rodner and Michael Kemmler and Joachim Denzler.
International Conference on Pattern Recognition and Image Analysis (PRIA), St. Petersburg, Russia. Pages 93-96. 2010.
[bibtex] [pdf]