Measurements of generalisation based on information geometry

Huaiyu Zhu, Richard Rohwer

    Research output: Contribution to journalArticle

    Abstract

    Neural networks are statistical models and learning rules are estimators. In this paper a theory for measuring generalisation is developed by combining Bayesian decision theory with information geometry. The performance of an estimator is measured by the information divergence between the true distribution and the estimate, averaged over the Bayesian posterior. This unifies the majority of error measures currently in use. The optimal estimators also reveal some intricate interrelationships among information geometry, Banach spaces and sufficient statistics.
    Original languageEnglish
    JournalAnnals of Mathematics And artificial Intelligence
    Publication statusPublished - 2 Jul 1995
    EventProc. Mathematics of Neural Networks and Applications -
    Duration: 2 Jul 19952 Jul 1995

    Bibliographical note

    Copyright of SpringerLink

    Keywords

    • neural networks
    • Bayesian
    • information geometry
    • estimator
    • error
    • Banach

    Fingerprint Dive into the research topics of 'Measurements of generalisation based on information geometry'. Together they form a unique fingerprint.

  • Cite this