A Bayesian approach to on-line learning

Manfred Opper, Ole Winther

Research output: Chapter in Book/Published conference outputChapter


Online learning is discussed from the viewpoint of Bayesian statistical inference. By replacing the true posterior distribution with a simpler parametric distribution, one can define an online algorithm by a repetition of two steps: An update of the approximate posterior, when a new example arrives, and an optimal projection into the parametric family. Choosing this family to be Gaussian, we show that the algorithm achieves asymptotic efficiency. An application to learning in single layer neural networks is given.
Original languageEnglish
Title of host publicationOn-line learning in neural networks
EditorsDavid Saad
Place of PublicationCambridge
PublisherCambridge University Press
Number of pages16
ISBN (Print)0262194163
Publication statusPublished - Jan 1999

Publication series

NamePublications of the Newton Institute
PublisherCambridge University Press

Bibliographical note

Copyright of Cambridge University Press Available on Google Books


  • Online learning
  • Bayesian statistical inference
  • asymptotic efficiency
  • neural networks


Dive into the research topics of 'A Bayesian approach to on-line learning'. Together they form a unique fingerprint.

Cite this