Globally optimal on-line learning rules

Magnus Rattray, David Saad, Michael I. Jordan (Editor), Michael J. Kearns (Editor), Sara A. Solla (Editor)

Research output: Contribution to journalArticle

Abstract

We present a method for determining the globally optimal on-line learning rule for a soft committee machine under a statistical mechanics framework. This work complements previous results on locally optimal rules, where only the rate of change in generalization error was considered. We maximize the total reduction in generalization error over the whole learning process and show how the resulting rule can significantly outperform the locally optimal rule.
Original languageEnglish
Pages (from-to)322-328
Number of pages7
JournalAdvances in Neural Information Processing Systems
Volume10
Publication statusPublished - Jan 1998
EventAdvances in Neural Information Processing Systems 1994 - Singapore, Singapore
Duration: 16 Nov 199418 Nov 1994

Bibliographical note

Copyright of the Massachusetts Institute of Technology Press (MIT Press)

Keywords

  • on-line learning
  • statistical mechanics
  • generalization error
  • optimal rule
  • resulting rule

Fingerprint Dive into the research topics of 'Globally optimal on-line learning rules'. Together they form a unique fingerprint.

  • Cite this

    Rattray, M., Saad, D., Jordan, M. I. (Ed.), Kearns, M. J. (Ed.), & Solla, S. A. (Ed.) (1998). Globally optimal on-line learning rules. Advances in Neural Information Processing Systems, 10, 322-328.