Globally optimal parameters for on-line learning in multilayer neural networks

David Saad, Magnus Rattray

Research output: Contribution to journalArticle

Abstract

We present a framework for calculating globally optimal parameters, within a given time frame, for on-line learning in multilayer neural networks. We demonstrate the capability of this method by computing optimal learning rates in typical learning scenarios. A similar treatment allows one to determine the relevance of related training algorithms based on modifications to the basic gradient descent rule as well as to compare different training methods.
Original languageEnglish
Pages (from-to)2578-2581
Number of pages4
JournalPhysical Review Letters
Volume79
Issue number13
DOIs
Publication statusPublished - 29 Sep 1997

Fingerprint

learning
education
descent
gradients

Bibliographical note

Copyright of the American Physical Society

Keywords

  • on-line learning
  • multilayer neural networks
  • learning rates
  • training algorithms

Cite this

@article{dbb9a8bef4c94b55b4375a32b706eb02,
title = "Globally optimal parameters for on-line learning in multilayer neural networks",
abstract = "We present a framework for calculating globally optimal parameters, within a given time frame, for on-line learning in multilayer neural networks. We demonstrate the capability of this method by computing optimal learning rates in typical learning scenarios. A similar treatment allows one to determine the relevance of related training algorithms based on modifications to the basic gradient descent rule as well as to compare different training methods.",
keywords = "on-line learning, multilayer neural networks, learning rates, training algorithms",
author = "David Saad and Magnus Rattray",
note = "Copyright of the American Physical Society",
year = "1997",
month = "9",
day = "29",
doi = "10.1103/PhysRevLett.79.2578",
language = "English",
volume = "79",
pages = "2578--2581",
journal = "Physical Review Letters",
issn = "0031-9007",
publisher = "American Physical Society",
number = "13",

}

Globally optimal parameters for on-line learning in multilayer neural networks. / Saad, David; Rattray, Magnus.

In: Physical Review Letters, Vol. 79, No. 13, 29.09.1997, p. 2578-2581.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Globally optimal parameters for on-line learning in multilayer neural networks

AU - Saad, David

AU - Rattray, Magnus

N1 - Copyright of the American Physical Society

PY - 1997/9/29

Y1 - 1997/9/29

N2 - We present a framework for calculating globally optimal parameters, within a given time frame, for on-line learning in multilayer neural networks. We demonstrate the capability of this method by computing optimal learning rates in typical learning scenarios. A similar treatment allows one to determine the relevance of related training algorithms based on modifications to the basic gradient descent rule as well as to compare different training methods.

AB - We present a framework for calculating globally optimal parameters, within a given time frame, for on-line learning in multilayer neural networks. We demonstrate the capability of this method by computing optimal learning rates in typical learning scenarios. A similar treatment allows one to determine the relevance of related training algorithms based on modifications to the basic gradient descent rule as well as to compare different training methods.

KW - on-line learning

KW - multilayer neural networks

KW - learning rates

KW - training algorithms

UR - http://www.scopus.com/inward/record.url?scp=0000031088&partnerID=8YFLogxK

UR - http://prola.aps.org/pdf/PRL/v79/i13/p2578_1

U2 - 10.1103/PhysRevLett.79.2578

DO - 10.1103/PhysRevLett.79.2578

M3 - Article

VL - 79

SP - 2578

EP - 2581

JO - Physical Review Letters

JF - Physical Review Letters

SN - 0031-9007

IS - 13

ER -