Globally optimal on-line learning rules for multi-layer neural networks

Magnus Rattray; David Saad

doi:10.1088/0305-4470/30/22/005

Globally optimal on-line learning rules for multi-layer neural networks

Magnus Rattray^*, David Saad

^*Corresponding author for this work

Research output: Contribution to journal › Article › peer-review

Abstract

We present a method for determining the globally optimal on-line learning rule for a soft committee machine under a statistical mechanics framework. This rule maximizes the total reduction in generalization error over the whole learning process. A simple example demonstrates that the locally optimal rule, which maximizes the rate of decrease in generalization error, may perform poorly in comparison.

Original language	English
Pages (from-to)	L771-L776
Journal	Journal of Physics A: Mathematical and General
Volume	30
Issue number	22
DOIs	https://doi.org/10.1088/0305-4470/30/22/005
Publication status	Published - 21 Nov 1997

Bibliographical note

Copyright of the Institute of Physics.

Keywords

globally optimal on-line learning
soft committee machine
error
locally optimal rule

Access to Document

10.1088/0305-4470/30/22/005

Globally optimal on-line learning rules for multi-layer neural networks
Copyright of the Institute of Physics.
Accepted author manuscript, 189 KB

Cite this

@article{661611f01fe149a5ab57e01a686d57a5,

title = "Globally optimal on-line learning rules for multi-layer neural networks",

abstract = "We present a method for determining the globally optimal on-line learning rule for a soft committee machine under a statistical mechanics framework. This rule maximizes the total reduction in generalization error over the whole learning process. A simple example demonstrates that the locally optimal rule, which maximizes the rate of decrease in generalization error, may perform poorly in comparison.",

keywords = "globally optimal on-line learning, soft committee machine, error, locally optimal rule",

author = "Magnus Rattray and David Saad",

note = "Copyright of the Institute of Physics.",

year = "1997",

month = nov,

day = "21",

doi = "10.1088/0305-4470/30/22/005",

language = "English",

volume = "30",

pages = "L771--L776",

journal = "Journal of Physics A: Mathematical and General",

issn = "0305-4470",

publisher = "IOP Publishing Ltd.",

number = "22",

}

TY - JOUR

T1 - Globally optimal on-line learning rules for multi-layer neural networks

AU - Rattray, Magnus

AU - Saad, David

N1 - Copyright of the Institute of Physics.

PY - 1997/11/21

Y1 - 1997/11/21

N2 - We present a method for determining the globally optimal on-line learning rule for a soft committee machine under a statistical mechanics framework. This rule maximizes the total reduction in generalization error over the whole learning process. A simple example demonstrates that the locally optimal rule, which maximizes the rate of decrease in generalization error, may perform poorly in comparison.

AB - We present a method for determining the globally optimal on-line learning rule for a soft committee machine under a statistical mechanics framework. This rule maximizes the total reduction in generalization error over the whole learning process. A simple example demonstrates that the locally optimal rule, which maximizes the rate of decrease in generalization error, may perform poorly in comparison.

KW - globally optimal on-line learning

KW - soft committee machine

KW - error

KW - locally optimal rule

UR - http://www.scopus.com/inward/record.url?scp=0031582941&partnerID=8YFLogxK

UR - http://iopscience.iop.org/0305-4470/30/22/005/?ejredirect=.iopscience

U2 - 10.1088/0305-4470/30/22/005

DO - 10.1088/0305-4470/30/22/005

M3 - Article

SN - 0305-4470

VL - 30

SP - L771-L776

JO - Journal of Physics A: Mathematical and General

JF - Journal of Physics A: Mathematical and General

IS - 22

ER -

Globally optimal on-line learning rules for multi-layer neural networks

Abstract

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this