# Gaussian Processes

## Table of Contents

## Results

- Variational framework for learning inducing variables can be intepreted as minimizing a rigorously defined KL-divergence between the approximating and posterior processes. matthews15_spars_variat_method_kullb_leibl

## Useful notes

### Hyperparameters tuning

## IMPORTANT

As of right now, must of the notes regarding this topic can be found in the notes for the book Guassian Proccesses for Machine Learning.

These will be moved in the future.

## Automatic Relevance Determination

Consider the covariance function:

1

The parameter is the *length scale of the function along input dimension *.
This implies that as the function varies less and less
as a function , that is, the *dth* dimension becomes *irrelevant*.

Hence, given data, by learning the *lengthscales*
is is possible to do *automatic feature selection*.

## Resources

- A Tutorial on Gaussian Processes (or why I don't use SVMs) by Zoubin Ghahramani
A short presentation, providing an overview and showing how the objective function
of a SVM is quite similar to a GP, but GP also has other
*nicer*properties. He makes the following notes when comparing GPs with SVMs:- GP incorporates uncertainty
- GP computes ,
*not*as SVM - GP can
**learn the kernel parameters**automatically from data, no matter how flexible we make the kernel - GP can
**learn the regularization parameter**without cross-validation - Can combine
**automatic feature selection**with learning using*automatic relevance determination*(ARD)

## Connection to RKHSs

- If both uses the same kernel, the posterior mean of a GP regression equals the estimator of kernel ridge regression

### Connections between GPs and Kernel Ridge Regression

#### Notation

- non-empty set
- be a function
- Given set of pairs for
Assumption/model:

where is a zero-mean rv. which represents "noise" or uncontrollable error

- If for all ,i .e. no
*output*noise, then we call the problem**interpolation** - in the noise-free/interpolation case

#### Gaussian Process Regression and Interpolation

- Also known as
**Kriging**or**Wiener-Kolmogorov prediction** - Non-parameter method for regression