Measurements of generalisation based on information geometry

Huaiyu Zhu, Richard Rohwer

    Research output: Contribution to journalArticle

    Abstract

    Neural networks are statistical models and learning rules are estimators. In this paper a theory for measuring generalisation is developed by combining Bayesian decision theory with information geometry. The performance of an estimator is measured by the information divergence between the true distribution and the estimate, averaged over the Bayesian posterior. This unifies the majority of error measures currently in use. The optimal estimators also reveal some intricate interrelationships among information geometry, Banach spaces and sufficient statistics.
    Original languageEnglish
    JournalAnnals of Mathematics And artificial Intelligence
    Publication statusPublished - 2 Jul 1995
    EventProc. Mathematics of Neural Networks and Applications -
    Duration: 2 Jul 19952 Jul 1995

    Fingerprint

    Information Geometry
    Estimator
    Geometry
    Decision theory
    Banach spaces
    Bayesian Decision Theory
    Statistical Learning
    Sufficient Statistics
    Statistics
    Neural networks
    Statistical Model
    Divergence
    Banach space
    Neural Networks
    Estimate
    Generalization
    Statistical Models

    Bibliographical note

    Copyright of SpringerLink

    Keywords

    • neural networks
    • Bayesian
    • information geometry
    • estimator
    • error
    • Banach

    Cite this

    @article{7e3150b82ff1492bbe7c70eb11b943f6,
    title = "Measurements of generalisation based on information geometry",
    abstract = "Neural networks are statistical models and learning rules are estimators. In this paper a theory for measuring generalisation is developed by combining Bayesian decision theory with information geometry. The performance of an estimator is measured by the information divergence between the true distribution and the estimate, averaged over the Bayesian posterior. This unifies the majority of error measures currently in use. The optimal estimators also reveal some intricate interrelationships among information geometry, Banach spaces and sufficient statistics.",
    keywords = "neural networks, Bayesian, information geometry, estimator, error, Banach",
    author = "Huaiyu Zhu and Richard Rohwer",
    note = "Copyright of SpringerLink",
    year = "1995",
    month = "7",
    day = "2",
    language = "English",
    journal = "Annals of Mathematics And artificial Intelligence",
    issn = "1012-2443",
    publisher = "Springer",

    }

    Measurements of generalisation based on information geometry. / Zhu, Huaiyu; Rohwer, Richard.

    In: Annals of Mathematics And artificial Intelligence, 02.07.1995.

    Research output: Contribution to journalArticle

    TY - JOUR

    T1 - Measurements of generalisation based on information geometry

    AU - Zhu, Huaiyu

    AU - Rohwer, Richard

    N1 - Copyright of SpringerLink

    PY - 1995/7/2

    Y1 - 1995/7/2

    N2 - Neural networks are statistical models and learning rules are estimators. In this paper a theory for measuring generalisation is developed by combining Bayesian decision theory with information geometry. The performance of an estimator is measured by the information divergence between the true distribution and the estimate, averaged over the Bayesian posterior. This unifies the majority of error measures currently in use. The optimal estimators also reveal some intricate interrelationships among information geometry, Banach spaces and sufficient statistics.

    AB - Neural networks are statistical models and learning rules are estimators. In this paper a theory for measuring generalisation is developed by combining Bayesian decision theory with information geometry. The performance of an estimator is measured by the information divergence between the true distribution and the estimate, averaged over the Bayesian posterior. This unifies the majority of error measures currently in use. The optimal estimators also reveal some intricate interrelationships among information geometry, Banach spaces and sufficient statistics.

    KW - neural networks

    KW - Bayesian

    KW - information geometry

    KW - estimator

    KW - error

    KW - Banach

    UR - http://www.springer.com/computer/artificial/journal/10472

    M3 - Article

    JO - Annals of Mathematics And artificial Intelligence

    JF - Annals of Mathematics And artificial Intelligence

    SN - 1012-2443

    ER -