Globally optimal on-line learning rules for multi-layer neural networks

Magnus Rattray*, David Saad

*Corresponding author for this work

    Research output: Contribution to journalArticlepeer-review


    We present a method for determining the globally optimal on-line learning rule for a soft committee machine under a statistical mechanics framework. This rule maximizes the total reduction in generalization error over the whole learning process. A simple example demonstrates that the locally optimal rule, which maximizes the rate of decrease in generalization error, may perform poorly in comparison.

    Original languageEnglish
    Pages (from-to)L771-L776
    JournalJournal of Physics A: Mathematical and General
    Issue number22
    Publication statusPublished - 21 Nov 1997

    Bibliographical note

    Copyright of the Institute of Physics.


    • globally optimal on-line learning
    • soft committee machine
    • error
    • locally optimal rule


    Dive into the research topics of 'Globally optimal on-line learning rules for multi-layer neural networks'. Together they form a unique fingerprint.

    Cite this