Dynamics of on-line gradient descent learning for multilayer neural networks

David Saad; Sara A. Solla

Dynamics of on-line gradient descent learning for multilayer neural networks

David Saad, Sara A. Solla

Mathematics

Research output: Contribution to journal › Article › peer-review

Abstract

We consider the problem of on-line gradient descent learning for general two-layer neural networks. An analytic solution is presented and used to investigate the role of the learning rate in controlling the evolution and convergence of the learning process.

Original language	English
Pages (from-to)	302-308
Number of pages	7
Journal	Advances in Neural Information Processing Systems
Volume	8
Publication status	Published - 1996

Bibliographical note

Copyright of Massachusetts Institute of Technology Press (MIT Press) http://mitpress.mit.edu/mitpress/copyright/default.asp

Keywords

on-line
gradient descent learning
general two-layer neural networks
learning rate
learning process.

Access to Document

Dynamics of on-line gradient descent learning for multilayer neural networks
Copyright of Massachusetts Institute of Technology Press (MIT Press) http://mitpress.mit.edu/mitpress/copyright/default.asp
Final published version, 693 KB

Cite this

@article{49c8a447bc6d4d2d80d4cc5795c0229d,

title = "Dynamics of on-line gradient descent learning for multilayer neural networks",

abstract = "We consider the problem of on-line gradient descent learning for general two-layer neural networks. An analytic solution is presented and used to investigate the role of the learning rate in controlling the evolution and convergence of the learning process.",

keywords = "on-line, gradient descent learning, general two-layer neural networks, learning rate, learning process.",

author = "David Saad and Solla, {Sara A.}",

note = "Copyright of Massachusetts Institute of Technology Press (MIT Press) http://mitpress.mit.edu/mitpress/copyright/default.asp",

year = "1996",

language = "English",

volume = "8",

pages = "302--308",

journal = "Advances in Neural Information Processing Systems",

issn = "1049-5258",

publisher = "MIT Press Journals",

}

TY - JOUR

T1 - Dynamics of on-line gradient descent learning for multilayer neural networks

AU - Saad, David

AU - Solla, Sara A.

N1 - Copyright of Massachusetts Institute of Technology Press (MIT Press) http://mitpress.mit.edu/mitpress/copyright/default.asp

PY - 1996

Y1 - 1996

N2 - We consider the problem of on-line gradient descent learning for general two-layer neural networks. An analytic solution is presented and used to investigate the role of the learning rate in controlling the evolution and convergence of the learning process.

AB - We consider the problem of on-line gradient descent learning for general two-layer neural networks. An analytic solution is presented and used to investigate the role of the learning rate in controlling the evolution and convergence of the learning process.

KW - on-line

KW - gradient descent learning

KW - general two-layer neural networks

KW - learning rate

KW - learning process.

UR - http://mitpress.mit.edu/catalog/item/default.asp?ttype=2&tid=8421

M3 - Article

VL - 8

SP - 302

EP - 308

JO - Advances in Neural Information Processing Systems

JF - Advances in Neural Information Processing Systems

SN - 1049-5258

ER -

Dynamics of on-line gradient descent learning for multilayer neural networks

Abstract

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this