Globally optimal on-line learning rules for multi-layer neural networks

Magnus Rattray*, David Saad

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review


We present a method for determining the globally optimal on-line learning rule for a soft committee machine under a statistical mechanics framework. This rule maximizes the total reduction in generalization error over the whole learning process. A simple example demonstrates that the locally optimal rule, which maximizes the rate of decrease in generalization error, may perform poorly in comparison.

Original languageEnglish
Pages (from-to)L771-L776
JournalJournal of Physics A: Mathematical and General
Issue number22
Publication statusPublished - 21 Nov 1997

Bibliographical note

Copyright of the Institute of Physics.


  • globally optimal on-line learning
  • soft committee machine
  • error
  • locally optimal rule


Dive into the research topics of 'Globally optimal on-line learning rules for multi-layer neural networks'. Together they form a unique fingerprint.

Cite this