Globally optimal on-line learning rules

Magnus Rattray, David Saad, Michael I. Jordan (Editor), Michael J. Kearns (Editor), Sara A. Solla (Editor)

Research output: Contribution to journal › Article

Abstract

We present a method for determining the globally optimal on-line learning rule for a soft committee machine under a statistical mechanics framework. This work complements previous results on locally optimal rules, where only the rate of change in generalization error was considered. We maximize the total reduction in generalization error over the whole learning process and show how the resulting rule can significantly outperform the locally optimal rule.
Original languageEnglish
Pages (from-to)322-328
Number of pages7
JournalAdvances in Neural Information Processing Systems
Volume10
Publication statusPublished - Jan 1998
EventAdvances in Neural Information Processing Systems 1994 - Singapore, Singapore
Duration: 16 Nov 1994 → 18 Nov 1994

Fingerprint

Statistical mechanics

Bibliographical note

Copyright of the Massachusetts Institute of Technology Press (MIT Press)

Keywords

  • on-line learning
  • statistical mechanics
  • generalization error
  • optimal rule
  • resulting rule

Cite this

Rattray, M., Saad, D., Jordan, M. I. (Ed.), Kearns, M. J. (Ed.), & Solla, S. A. (Ed.) (1998). Globally optimal on-line learning rules. Advances in Neural Information Processing Systems, 10, 322-328.
Rattray, Magnus ; Saad, David ; Jordan, Michael I. (Editor) ; Kearns, Michael J. (Editor) ; Solla, Sara A. (Editor). / Globally optimal on-line learning rules. In: Advances in Neural Information Processing Systems. 1998 ; Vol. 10. pp. 322-328.
@article{605cdb35e0b84a369b4cbc0675b08ea9,
title = "Globally optimal on-line learning rules",
abstract = "We present a method for determining the globally optimal on-line learning rule for a soft committee machine under a statistical mechanics framework. This work complements previous results on locally optimal rules, where only the rate of change in generalization error was considered. We maximize the total reduction in generalization error over the whole learning process and show how the resulting rule can significantly outperform the locally optimal rule.",
keywords = "on-line learning, statistical mechanics, generalization error, optimal rule, resulting rule",
author = "Magnus Rattray and David Saad and Jordan, {Michael I.} and Kearns, {Michael J.} and Solla, {Sara A.}",
note = "Copyright of the Massachusetts Institute of Technology Press (MIT Press)",
year = "1998",
month = "1",
language = "English",
volume = "10",
pages = "322--328",
journal = "Advances in Neural Information Processing Systems",
issn = "1049-5258",

}

Rattray, M, Saad, D, Jordan, MI (ed.), Kearns, MJ (ed.) & Solla, SA (ed.) 1998, 'Globally optimal on-line learning rules', Advances in Neural Information Processing Systems, vol. 10, pp. 322-328.

Globally optimal on-line learning rules. / Rattray, Magnus; Saad, David; Jordan, Michael I. (Editor); Kearns, Michael J. (Editor); Solla, Sara A. (Editor).

In: Advances in Neural Information Processing Systems, Vol. 10, 01.1998, p. 322-328.

Research output: Contribution to journal › Article

TY - JOUR

T1 - Globally optimal on-line learning rules

AU - Rattray, Magnus

AU - Saad, David

A2 - Jordan, Michael I.

A2 - Kearns, Michael J.

A2 - Solla, Sara A.

N1 - Copyright of the Massachusetts Institute of Technology Press (MIT Press)

PY - 1998/1

Y1 - 1998/1

N2 - We present a method for determining the globally optimal on-line learning rule for a soft committee machine under a statistical mechanics framework. This work complements previous results on locally optimal rules, where only the rate of change in generalization error was considered. We maximize the total reduction in generalization error over the whole learning process and show how the resulting rule can significantly outperform the locally optimal rule.

AB - We present a method for determining the globally optimal on-line learning rule for a soft committee machine under a statistical mechanics framework. This work complements previous results on locally optimal rules, where only the rate of change in generalization error was considered. We maximize the total reduction in generalization error over the whole learning process and show how the resulting rule can significantly outperform the locally optimal rule.

KW - on-line learning

KW - statistical mechanics

KW - generalization error

KW - optimal rule

KW - resulting rule

UR - http://www.scopus.com/inward/record.url?scp=84898946683&partnerID=8YFLogxK

UR - http://mitpress.mit.edu/catalog/item/default.asp?ttype=2&tid=8363

M3 - Article

VL - 10

SP - 322

EP - 328

JO - Advances in Neural Information Processing Systems

JF - Advances in Neural Information Processing Systems

SN - 1049-5258

ER -

Rattray M, Saad D, Jordan MI, (ed.), Kearns MJ, (ed.), Solla SA, (ed.). Globally optimal on-line learning rules. Advances in Neural Information Processing Systems. 1998 Jan;10:322-328.

Access to Document