Bayesian invariant measurements of generalisation

Huaiyu Zhu, Richard Rohwer

Research output: Contribution to journalArticlepeer-review

Abstract

The problem of evaluating different learning rules and other statistical estimators is analysed. A new general theory of statistical inference is developed by combining Bayesian decision theory with information geometry. It is coherent and invariant. For each sample a unique ideal estimate exists and is given by an average over the posterior. An optimal estimate within a model is given by a projection of the ideal estimate. The ideal estimate is a sufficient statistic of the posterior, so practical learning rules are functions of the ideal estimator. If the sole purpose of learning is to extract information from the data, the learning rule must also approximate the ideal estimator. This framework is applicable to both Bayesian and non-Bayesian methods, with arbitrary statistical models, and to supervised, unsupervised and reinforcement learning schemes.
Original languageEnglish
Pages (from-to)28-31
Number of pages4
JournalNeural Processing Letters
Volume2
Issue number6
Publication statusPublished - Dec 1995

Bibliographical note

Copyright of Springer Verlag. The original publication is available at www.springerlink.com

Keywords

  • learning rules
  • statistical estimators
  • statistical inference
  • decision theory
  • information geometry
  • Bayesian
  • non-Bayesian

Fingerprint

Dive into the research topics of 'Bayesian invariant measurements of generalisation'. Together they form a unique fingerprint.

Cite this