Bayesian classification with Gaussian processes

Christopher K. I. Williams; David Barber

Bayesian classification with Gaussian processes

Christopher K. I. Williams, David Barber

Aston University

Research output: Preprint or Working paper › Technical report

Abstract

We consider the problem of assigning an input vector bfx to one of m classes by predicting P(c|bfx) for c = 1, ldots, m. For a two-class problem, the probability of class 1 given bfx is estimated by s(y(bfx)), where s(y) = 1/(1 + e-y). A Gaussian process prior is placed on y(bfx), and is combined with the training data to obtain predictions for new bfx points. We provide a Bayesian treatment, integrating over uncertainty in y and in the parameters that control the Gaussian process prior; the necessary integration over y is carried out using Laplace's approximation. The method is generalized to multi-class problems (m >2) using the softmax function. We demonstrate the effectiveness of the method on a number of datasets.

Original language	English
Place of Publication	Birmingham
Publisher	Aston University
Number of pages	18
ISBN (Print)	NCRG/7/015
Publication status	Unpublished - 13 Dec 1997

Keywords

assigning
input vector
probability
Gaussian process
training data
predictions
Bayesian treatment prior
uncertainty
Laplace
approximation
multi-class problems
softmax function

Access to Document

NCRG_97_015.pdf

Cite this

@techreport{c47c2f5bb21c4da38bafe3cc61f721e7,

title = "Bayesian classification with Gaussian processes",

abstract = "We consider the problem of assigning an input vector bfx to one of m classes by predicting P(c|bfx) for c = 1, ldots, m. For a two-class problem, the probability of class 1 given bfx is estimated by s(y(bfx)), where s(y) = 1/(1 + e-y). A Gaussian process prior is placed on y(bfx), and is combined with the training data to obtain predictions for new bfx points. We provide a Bayesian treatment, integrating over uncertainty in y and in the parameters that control the Gaussian process prior; the necessary integration over y is carried out using Laplace's approximation. The method is generalized to multi-class problems (m >2) using the softmax function. We demonstrate the effectiveness of the method on a number of datasets.",

keywords = "assigning, input vector, probability, Gaussian process, training data, predictions, Bayesian treatment prior, uncertainty, Laplace, approximation, multi-class problems, softmax function",

author = "Williams, {Christopher K. I.} and David Barber",

year = "1997",

month = dec,

day = "13",

language = "English",

isbn = "NCRG/7/015",

publisher = "Aston University",

type = "WorkingPaper",

institution = "Aston University",

}

TY - UNPB

T1 - Bayesian classification with Gaussian processes

AU - Williams, Christopher K. I.

AU - Barber, David

PY - 1997/12/13

Y1 - 1997/12/13

N2 - We consider the problem of assigning an input vector bfx to one of m classes by predicting P(c|bfx) for c = 1, ldots, m. For a two-class problem, the probability of class 1 given bfx is estimated by s(y(bfx)), where s(y) = 1/(1 + e-y). A Gaussian process prior is placed on y(bfx), and is combined with the training data to obtain predictions for new bfx points. We provide a Bayesian treatment, integrating over uncertainty in y and in the parameters that control the Gaussian process prior; the necessary integration over y is carried out using Laplace's approximation. The method is generalized to multi-class problems (m >2) using the softmax function. We demonstrate the effectiveness of the method on a number of datasets.

AB - We consider the problem of assigning an input vector bfx to one of m classes by predicting P(c|bfx) for c = 1, ldots, m. For a two-class problem, the probability of class 1 given bfx is estimated by s(y(bfx)), where s(y) = 1/(1 + e-y). A Gaussian process prior is placed on y(bfx), and is combined with the training data to obtain predictions for new bfx points. We provide a Bayesian treatment, integrating over uncertainty in y and in the parameters that control the Gaussian process prior; the necessary integration over y is carried out using Laplace's approximation. The method is generalized to multi-class problems (m >2) using the softmax function. We demonstrate the effectiveness of the method on a number of datasets.

KW - assigning

KW - input vector

KW - probability

KW - Gaussian process

KW - training data

KW - predictions

KW - Bayesian treatment prior

KW - uncertainty

KW - Laplace

KW - approximation

KW - multi-class problems

KW - softmax function

M3 - Technical report

SN - NCRG/7/015

BT - Bayesian classification with Gaussian processes

PB - Aston University

CY - Birmingham

ER -

Bayesian classification with Gaussian processes

Abstract

Keywords

Access to Document

Fingerprint

Cite this