Globally optimal on-line learning rules

Magnus Rattray; David Saad; Michael I. Jordan; Michael J. Kearns; Sara A. Solla

Globally optimal on-line learning rules

Magnus Rattray, David Saad, Michael I. Jordan (Editor), Michael J. Kearns (Editor), Sara A. Solla (Editor)

Research output: Contribution to journal › Article › peer-review

Abstract

We present a method for determining the globally optimal on-line learning rule for a soft committee machine under a statistical mechanics framework. This work complements previous results on locally optimal rules, where only the rate of change in generalization error was considered. We maximize the total reduction in generalization error over the whole learning process and show how the resulting rule can significantly outperform the locally optimal rule.

Original language	English
Pages (from-to)	322-328
Number of pages	7
Journal	Advances in Neural Information Processing Systems
Volume	10
Publication status	Published - Jan 1998
Event	Advances in Neural Information Processing Systems 1994 - Singapore, Singapore Duration: 16 Nov 1994 → 18 Nov 1994

Bibliographical note

Copyright of the Massachusetts Institute of Technology Press (MIT Press)

Keywords

on-line learning
statistical mechanics
generalization error
optimal rule
resulting rule

Access to Document

Advances_in_Neural_Information_Processing_Advances_10.pdf
Copyright of the Massachusetts Institute of Technology Press (MIT Press)
Final published version, 665 KB

Fingerprint Dive into the research topics of 'Globally optimal on-line learning rules'. Together they form a unique fingerprint.

Cite this

@article{605cdb35e0b84a369b4cbc0675b08ea9,

title = "Globally optimal on-line learning rules",

abstract = "We present a method for determining the globally optimal on-line learning rule for a soft committee machine under a statistical mechanics framework. This work complements previous results on locally optimal rules, where only the rate of change in generalization error was considered. We maximize the total reduction in generalization error over the whole learning process and show how the resulting rule can significantly outperform the locally optimal rule.",

keywords = "on-line learning, statistical mechanics, generalization error, optimal rule, resulting rule",

author = "Magnus Rattray and David Saad and Jordan, {Michael I.} and Kearns, {Michael J.} and Solla, {Sara A.}",

note = "Copyright of the Massachusetts Institute of Technology Press (MIT Press); Advances in Neural Information Processing Systems 1994 ; Conference date: 16-11-1994 Through 18-11-1994",

year = "1998",

month = jan,

language = "English",

volume = "10",

pages = "322--328",

journal = "Advances in Neural Information Processing Systems",

issn = "1049-5258",

publisher = "MIT Press Journals",

}

TY - JOUR

T1 - Globally optimal on-line learning rules

AU - Rattray, Magnus

AU - Saad, David

A2 - Jordan, Michael I.

A2 - Kearns, Michael J.

A2 - Solla, Sara A.

N1 - Copyright of the Massachusetts Institute of Technology Press (MIT Press)

PY - 1998/1

Y1 - 1998/1

N2 - We present a method for determining the globally optimal on-line learning rule for a soft committee machine under a statistical mechanics framework. This work complements previous results on locally optimal rules, where only the rate of change in generalization error was considered. We maximize the total reduction in generalization error over the whole learning process and show how the resulting rule can significantly outperform the locally optimal rule.

AB - We present a method for determining the globally optimal on-line learning rule for a soft committee machine under a statistical mechanics framework. This work complements previous results on locally optimal rules, where only the rate of change in generalization error was considered. We maximize the total reduction in generalization error over the whole learning process and show how the resulting rule can significantly outperform the locally optimal rule.

KW - on-line learning

KW - statistical mechanics

KW - generalization error

KW - optimal rule

KW - resulting rule

UR - http://www.scopus.com/inward/record.url?scp=84898946683&partnerID=8YFLogxK

UR - http://mitpress.mit.edu/catalog/item/default.asp?ttype=2&tid=8363

M3 - Article

VL - 10

SP - 322

EP - 328

JO - Advances in Neural Information Processing Systems

JF - Advances in Neural Information Processing Systems

SN - 1049-5258

T2 - Advances in Neural Information Processing Systems 1994

Y2 - 16 November 1994 through 18 November 1994

ER -