Gaussian processes for Bayesian classification via Hybrid Monte Carlo

David Barber, Christopher K. I. Williams

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The full Bayesian method for applying neural networks to a prediction problem is to set up the prior/hyperprior structure for the net and then perform the necessary integrals. However, these integrals are not tractable analytically, and Markov Chain Monte Carlo (MCMC) methods are slow, especially if the parameter space is high-dimensional. Using Gaussian processes we can approximate the weight space integral analytically, so that only a small number of hyperparameters need be integrated over by MCMC methods. We have applied this idea to classification problems, obtaining excellent results on the real-world problems investigated so far.
Original languageEnglish
Title of host publicationAdvances in Neural Information Processing Systems
EditorsM. C. Mozer, M. I. Jordan, T. Petsche
Place of PublicationCambridge, US
PublisherMIT
Pages340-346
Number of pages7
Volume9
ISBN (Print)0262100657
Publication statusPublished - May 1997
Event10th Annual Conference on Neural Information Processing Systems, NIPS 1996 - Denver, CO, United Kingdom
Duration: 2 Dec 19965 Dec 1996

Publication series

NameProceeding of 1996 conference
PublisherMassachusetts Institute of Technology Press (MIT Press)

Conference

Conference10th Annual Conference on Neural Information Processing Systems, NIPS 1996
CountryUnited Kingdom
CityDenver, CO
Period2/12/965/12/96

Fingerprint

Markov processes
Monte Carlo methods
Neural networks

Bibliographical note

Copyright of the Massachusetts Institute of Technology Press (MIT Press)

Keywords

  • Bayesian method
  • neural networks
  • structure for the net
  • integrals
  • Markov Chain Monte Carlo
  • weight space integral

Cite this

Barber, D., & Williams, C. K. I. (1997). Gaussian processes for Bayesian classification via Hybrid Monte Carlo. In M. C. Mozer, M. I. Jordan, & T. Petsche (Eds.), Advances in Neural Information Processing Systems (Vol. 9, pp. 340-346). (Proceeding of 1996 conference). Cambridge, US: MIT.
Barber, David ; Williams, Christopher K. I. / Gaussian processes for Bayesian classification via Hybrid Monte Carlo. Advances in Neural Information Processing Systems. editor / M. C. Mozer ; M. I. Jordan ; T. Petsche. Vol. 9 Cambridge, US : MIT, 1997. pp. 340-346 (Proceeding of 1996 conference).
@inproceedings{e4111ba4fff3439f990174b8c8fce6a9,
title = "Gaussian processes for Bayesian classification via Hybrid Monte Carlo",
abstract = "The full Bayesian method for applying neural networks to a prediction problem is to set up the prior/hyperprior structure for the net and then perform the necessary integrals. However, these integrals are not tractable analytically, and Markov Chain Monte Carlo (MCMC) methods are slow, especially if the parameter space is high-dimensional. Using Gaussian processes we can approximate the weight space integral analytically, so that only a small number of hyperparameters need be integrated over by MCMC methods. We have applied this idea to classification problems, obtaining excellent results on the real-world problems investigated so far.",
keywords = "Bayesian method, neural networks, structure for the net, integrals, Markov Chain Monte Carlo, weight space integral",
author = "David Barber and Williams, {Christopher K. I.}",
note = "Copyright of the Massachusetts Institute of Technology Press (MIT Press)",
year = "1997",
month = "5",
language = "English",
isbn = "0262100657",
volume = "9",
series = "Proceeding of 1996 conference",
publisher = "MIT",
pages = "340--346",
editor = "Mozer, {M. C.} and Jordan, {M. I.} and T. Petsche",
booktitle = "Advances in Neural Information Processing Systems",

}

Barber, D & Williams, CKI 1997, Gaussian processes for Bayesian classification via Hybrid Monte Carlo. in MC Mozer, MI Jordan & T Petsche (eds), Advances in Neural Information Processing Systems. vol. 9, Proceeding of 1996 conference, MIT, Cambridge, US, pp. 340-346, 10th Annual Conference on Neural Information Processing Systems, NIPS 1996, Denver, CO, United Kingdom, 2/12/96.

Gaussian processes for Bayesian classification via Hybrid Monte Carlo. / Barber, David; Williams, Christopher K. I.

Advances in Neural Information Processing Systems. ed. / M. C. Mozer; M. I. Jordan; T. Petsche. Vol. 9 Cambridge, US : MIT, 1997. p. 340-346 (Proceeding of 1996 conference).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Gaussian processes for Bayesian classification via Hybrid Monte Carlo

AU - Barber, David

AU - Williams, Christopher K. I.

N1 - Copyright of the Massachusetts Institute of Technology Press (MIT Press)

PY - 1997/5

Y1 - 1997/5

N2 - The full Bayesian method for applying neural networks to a prediction problem is to set up the prior/hyperprior structure for the net and then perform the necessary integrals. However, these integrals are not tractable analytically, and Markov Chain Monte Carlo (MCMC) methods are slow, especially if the parameter space is high-dimensional. Using Gaussian processes we can approximate the weight space integral analytically, so that only a small number of hyperparameters need be integrated over by MCMC methods. We have applied this idea to classification problems, obtaining excellent results on the real-world problems investigated so far.

AB - The full Bayesian method for applying neural networks to a prediction problem is to set up the prior/hyperprior structure for the net and then perform the necessary integrals. However, these integrals are not tractable analytically, and Markov Chain Monte Carlo (MCMC) methods are slow, especially if the parameter space is high-dimensional. Using Gaussian processes we can approximate the weight space integral analytically, so that only a small number of hyperparameters need be integrated over by MCMC methods. We have applied this idea to classification problems, obtaining excellent results on the real-world problems investigated so far.

KW - Bayesian method

KW - neural networks

KW - structure for the net

KW - integrals

KW - Markov Chain Monte Carlo

KW - weight space integral

UR - http://www.scopus.com/inward/record.url?scp=0000308194&partnerID=8YFLogxK

UR - http://mitpress.mit.edu/catalog/item/default.asp?ttype=2&tid=3990

M3 - Conference contribution

SN - 0262100657

VL - 9

T3 - Proceeding of 1996 conference

SP - 340

EP - 346

BT - Advances in Neural Information Processing Systems

A2 - Mozer, M. C.

A2 - Jordan, M. I.

A2 - Petsche, T.

PB - MIT

CY - Cambridge, US

ER -

Barber D, Williams CKI. Gaussian processes for Bayesian classification via Hybrid Monte Carlo. In Mozer MC, Jordan MI, Petsche T, editors, Advances in Neural Information Processing Systems. Vol. 9. Cambridge, US: MIT. 1997. p. 340-346. (Proceeding of 1996 conference).