Regression with input-dependent noise: A Bayesian treatment

Christopher M. Bishop, Cazhaow S. Qazaz

    Research output: Chapter in Book/Report/Conference proceedingChapter

    Abstract

    In most treatments of the regression problem it is assumed that the distribution of target data can be described by a deterministic function of the inputs, together with additive Gaussian noise having constant variance. The use of maximum likelihood to train such models then corresponds to the minimization of a sum-of-squares error function. In many applications a more realistic model would allow the noise variance itself to depend on the input variables. However, the use of maximum likelihood to train such models would give highly biased results. In this paper we show how a Bayesian treatment can allow for an input-dependent variance while overcoming the bias of maximum likelihood.
    Original languageEnglish
    Title of host publicationAdvances in Neural Information Processing Systems 9
    EditorsMichael C. Mozer, Michael I. Jordan, Thomas Petsche
    PublisherMIT
    Pages347-353
    Number of pages7
    Volume9
    ISBN (Print)0262100657
    Publication statusPublished - May 1997
    EventAdvances in Neural Information Processing Systems 1994 - Singapore, Singapore
    Duration: 16 Nov 199418 Nov 1994

    Publication series

    NameProceeding of the 1996 conference
    PublisherMassachusetts Institute of Technology Press (MIT Press)

    Other

    OtherAdvances in Neural Information Processing Systems 1994
    CountrySingapore
    CitySingapore
    Period16/11/9418/11/94

    Fingerprint

    Maximum Likelihood
    Regression
    Dependent
    Square Functions
    Error function
    Gaussian Noise
    Sum of squares
    Biased
    Model
    Target

    Bibliographical note

    Copyright of the Massachusetts Institute of Technology Press (MIT Press)

    Keywords

    • regression problem
    • target data
    • Gaussian noise
    • error function
    • noise variance

    Cite this

    Bishop, C. M., & Qazaz, C. S. (1997). Regression with input-dependent noise: A Bayesian treatment. In M. C. Mozer, M. I. Jordan, & T. Petsche (Eds.), Advances in Neural Information Processing Systems 9 (Vol. 9, pp. 347-353). (Proceeding of the 1996 conference). MIT.
    Bishop, Christopher M. ; Qazaz, Cazhaow S. / Regression with input-dependent noise: A Bayesian treatment. Advances in Neural Information Processing Systems 9. editor / Michael C. Mozer ; Michael I. Jordan ; Thomas Petsche. Vol. 9 MIT, 1997. pp. 347-353 (Proceeding of the 1996 conference).
    @inbook{816edeb99ebc43f6bff87cc8e56b3e84,
    title = "Regression with input-dependent noise: A Bayesian treatment",
    abstract = "In most treatments of the regression problem it is assumed that the distribution of target data can be described by a deterministic function of the inputs, together with additive Gaussian noise having constant variance. The use of maximum likelihood to train such models then corresponds to the minimization of a sum-of-squares error function. In many applications a more realistic model would allow the noise variance itself to depend on the input variables. However, the use of maximum likelihood to train such models would give highly biased results. In this paper we show how a Bayesian treatment can allow for an input-dependent variance while overcoming the bias of maximum likelihood.",
    keywords = "regression problem, target data, Gaussian noise, error function, noise variance",
    author = "Bishop, {Christopher M.} and Qazaz, {Cazhaow S.}",
    note = "Copyright of the Massachusetts Institute of Technology Press (MIT Press)",
    year = "1997",
    month = "5",
    language = "English",
    isbn = "0262100657",
    volume = "9",
    series = "Proceeding of the 1996 conference",
    publisher = "MIT",
    pages = "347--353",
    editor = "Mozer, {Michael C.} and Jordan, {Michael I.} and Thomas Petsche",
    booktitle = "Advances in Neural Information Processing Systems 9",

    }

    Bishop, CM & Qazaz, CS 1997, Regression with input-dependent noise: A Bayesian treatment. in MC Mozer, MI Jordan & T Petsche (eds), Advances in Neural Information Processing Systems 9. vol. 9, Proceeding of the 1996 conference, MIT, pp. 347-353, Advances in Neural Information Processing Systems 1994, Singapore, Singapore, 16/11/94.

    Regression with input-dependent noise: A Bayesian treatment. / Bishop, Christopher M.; Qazaz, Cazhaow S.

    Advances in Neural Information Processing Systems 9. ed. / Michael C. Mozer; Michael I. Jordan; Thomas Petsche. Vol. 9 MIT, 1997. p. 347-353 (Proceeding of the 1996 conference).

    Research output: Chapter in Book/Report/Conference proceedingChapter

    TY - CHAP

    T1 - Regression with input-dependent noise: A Bayesian treatment

    AU - Bishop, Christopher M.

    AU - Qazaz, Cazhaow S.

    N1 - Copyright of the Massachusetts Institute of Technology Press (MIT Press)

    PY - 1997/5

    Y1 - 1997/5

    N2 - In most treatments of the regression problem it is assumed that the distribution of target data can be described by a deterministic function of the inputs, together with additive Gaussian noise having constant variance. The use of maximum likelihood to train such models then corresponds to the minimization of a sum-of-squares error function. In many applications a more realistic model would allow the noise variance itself to depend on the input variables. However, the use of maximum likelihood to train such models would give highly biased results. In this paper we show how a Bayesian treatment can allow for an input-dependent variance while overcoming the bias of maximum likelihood.

    AB - In most treatments of the regression problem it is assumed that the distribution of target data can be described by a deterministic function of the inputs, together with additive Gaussian noise having constant variance. The use of maximum likelihood to train such models then corresponds to the minimization of a sum-of-squares error function. In many applications a more realistic model would allow the noise variance itself to depend on the input variables. However, the use of maximum likelihood to train such models would give highly biased results. In this paper we show how a Bayesian treatment can allow for an input-dependent variance while overcoming the bias of maximum likelihood.

    KW - regression problem

    KW - target data

    KW - Gaussian noise

    KW - error function

    KW - noise variance

    UR - http://mitpress.mit.edu/catalog/item/default.asp?ttype=2&tid=3990

    M3 - Chapter

    SN - 0262100657

    VL - 9

    T3 - Proceeding of the 1996 conference

    SP - 347

    EP - 353

    BT - Advances in Neural Information Processing Systems 9

    A2 - Mozer, Michael C.

    A2 - Jordan, Michael I.

    A2 - Petsche, Thomas

    PB - MIT

    ER -

    Bishop CM, Qazaz CS. Regression with input-dependent noise: A Bayesian treatment. In Mozer MC, Jordan MI, Petsche T, editors, Advances in Neural Information Processing Systems 9. Vol. 9. MIT. 1997. p. 347-353. (Proceeding of the 1996 conference).