Gaussian processes for regression

C. K. I. Williams; C. E. Rasmussen

Gaussian processes for regression

C. K. I. Williams, C. E. Rasmussen

Aston University

Research output: Chapter in Book/Published conference output › Chapter

Abstract

The Bayesian analysis of neural networks is difficult because a simple prior over weights implies a complex prior distribution over functions. In this paper we investigate the use of Gaussian process priors over functions, which permit the predictive Bayesian analysis for fixed values of hyperparameters to be carried out exactly using matrix operations. Two methods, using optimization and averaging (via Hybrid Monte Carlo) over hyperparameters have been tested on a number of challenging problems and have produced excellent results.

Original language	English
Title of host publication	Advances in Neural Information Processing Systems 8
Editors	D. S. Touretzky, M. C. Mozer, M. E. Hasselmo
Publisher	MIT
ISBN (Print)	0262201070
Publication status	Published - Jun 1996
Event	Advances in Neural Information Processing Systems 8 - Duration: 1 Jan 1996 → 1 Jan 1996

Conference

Conference	Advances in Neural Information Processing Systems 8
Period	1/01/96 → 1/01/96

Bibliographical note

Copyright of the Massachusetts Institute of Technology Press (MIT Press)

Keywords

Bayesian analysis
neural networks
Gaussian process
predictive
hyperparameters
matrix optimization
averaging
Hybrid Monte Carlo

Access to Document

Gaussian processes for regression
Copyright of the Massachusetts Institute of Technology Press (MIT Press)
Final published version, 708 KB

Cite this

@inbook{bd00c6283869413ab03ce907ce0681d6,

title = "Gaussian processes for regression",

abstract = "The Bayesian analysis of neural networks is difficult because a simple prior over weights implies a complex prior distribution over functions. In this paper we investigate the use of Gaussian process priors over functions, which permit the predictive Bayesian analysis for fixed values of hyperparameters to be carried out exactly using matrix operations. Two methods, using optimization and averaging (via Hybrid Monte Carlo) over hyperparameters have been tested on a number of challenging problems and have produced excellent results.",

keywords = "Bayesian analysis, neural networks, Gaussian process, predictive, hyperparameters, matrix optimization, averaging, Hybrid Monte Carlo",

author = "Williams, {C. K. I.} and Rasmussen, {C. E.}",

note = "Copyright of the Massachusetts Institute of Technology Press (MIT Press); Advances in Neural Information Processing Systems 8 ; Conference date: 01-01-1996 Through 01-01-1996",

year = "1996",

month = jun,

language = "English",

isbn = "0262201070",

editor = "Touretzky, {D. S.} and Mozer, {M. C.} and Hasselmo, {M. E.}",

booktitle = "Advances in Neural Information Processing Systems 8",

publisher = "MIT",

}

TY - CHAP

T1 - Gaussian processes for regression

AU - Williams, C. K. I.

AU - Rasmussen, C. E.

N1 - Copyright of the Massachusetts Institute of Technology Press (MIT Press)

PY - 1996/6

Y1 - 1996/6

N2 - The Bayesian analysis of neural networks is difficult because a simple prior over weights implies a complex prior distribution over functions. In this paper we investigate the use of Gaussian process priors over functions, which permit the predictive Bayesian analysis for fixed values of hyperparameters to be carried out exactly using matrix operations. Two methods, using optimization and averaging (via Hybrid Monte Carlo) over hyperparameters have been tested on a number of challenging problems and have produced excellent results.

AB - The Bayesian analysis of neural networks is difficult because a simple prior over weights implies a complex prior distribution over functions. In this paper we investigate the use of Gaussian process priors over functions, which permit the predictive Bayesian analysis for fixed values of hyperparameters to be carried out exactly using matrix operations. Two methods, using optimization and averaging (via Hybrid Monte Carlo) over hyperparameters have been tested on a number of challenging problems and have produced excellent results.

KW - Bayesian analysis

KW - neural networks

KW - Gaussian process

KW - predictive

KW - hyperparameters

KW - matrix optimization

KW - averaging

KW - Hybrid Monte Carlo

UR - http://mitpress.mit.edu/catalog/item/default.asp?ttype=2&tid=8421

M3 - Chapter

SN - 0262201070

BT - Advances in Neural Information Processing Systems 8

A2 - Touretzky, D. S.

A2 - Mozer, M. C.

A2 - Hasselmo, M. E.

PB - MIT

T2 - Advances in Neural Information Processing Systems 8

Y2 - 1 January 1996 through 1 January 1996

ER -

Gaussian processes for regression

Abstract

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this