A Bayesian approach to on-line learning

Manfred Opper, Ole Winther

Research output: Chapter in Book/Report/Conference proceedingChapter

Abstract

Online learning is discussed from the viewpoint of Bayesian statistical inference. By replacing the true posterior distribution with a simpler parametric distribution, one can define an online algorithm by a repetition of two steps: An update of the approximate posterior, when a new example arrives, and an optimal projection into the parametric family. Choosing this family to be Gaussian, we show that the algorithm achieves asymptotic efficiency. An application to learning in single layer neural networks is given.
Original languageEnglish
Title of host publicationOn-line learning in neural networks
EditorsDavid Saad
Place of PublicationCambridge
PublisherCambridge University Press
Pages363-378
Number of pages16
ISBN (Print)0262194163
DOIs
Publication statusPublished - Jan 1999

Publication series

NamePublications of the Newton Institute
PublisherCambridge University Press
Volume17

Bibliographical note

Copyright of Cambridge University Press Available on Google Books

Keywords

  • Online learning
  • Bayesian statistical inference
  • asymptotic efficiency
  • neural networks

Fingerprint Dive into the research topics of 'A Bayesian approach to on-line learning'. Together they form a unique fingerprint.

  • Research Output

    On-line learning in neural networks

    Saad, D. (ed.), Jan 1999, Cambridge: Cambridge University Press. 408 p. (Publications of the Newton Institute)

    Research output: Book/ReportBook

  • Cite this

    Opper, M., & Winther, O. (1999). A Bayesian approach to on-line learning. In D. Saad (Ed.), On-line learning in neural networks (pp. 363-378). (Publications of the Newton Institute; Vol. 17). Cambridge University Press. https://doi.org/10.2277/0521652634