Team

Erik Rodner, Alexander Freytag, Paul Bodesheim, Björn Fröhlich

Overview

How to deal with tens of thousands of examples in an exact Bayesian manner?

Problem Statement

Gaussian Processes suffer from several drawbacks in the general formulation

• training as well as hyperparameter optimization is cubically in the number of examples used
• evaluation is linear in the number of examples used
• computation of the classification uncertainty is quadratic in the number of examples used
• the memory demand is quadratic due to the kernel matrix

Summary

We present new methods for fast exact Gaussian process inference including multi-class classification, hyperparameter optimization, and uncertainty prediction in large-scale scenarios. The key observation of our methods is that the inherent properties of parameterized histogram intersection kernels can be exploited efficiently leading to significant time and memory benefits in several domains. In addition, contributions are made considering provable bounds for hyperparameter optimization, the identification of suitable linear solvers, a new active learning strategy, and incremental learning extensions. Evaluations are based on experiments with the large-scale real-world ImageNet database as well as the utilization of our techniques in the area of pixelwise labeling of images and active learning. The results show that inference can be done within microseconds and that every important piece of the Gaussian process framework (e.g., classification, hyperparameter optimization, variance estimation) can be also used in the presence of tens of thousands of examples.

Code

We released the software for our ECCV and ACCV paper on fast Gaussian process inference with histogram intersection kernels on GitHub. The software is written in C++ and requires our computer vision library NICE-core.