Transients and asymptotics of natural gradient learning

Magnus Rattray; David Saad

doi:10.1007/978-1-4471-1599-1_21

Transients and asymptotics of natural gradient learning

Magnus Rattray, David Saad

Aston University

Research output: Chapter in Book/Published conference output › Chapter

Abstract

We analyse natural gradient learning in a two-layer feed-forward neural network using a statistical mechanics framework which is appropriate for large input dimension. We find significant improvement over standard gradient descent in both the transient and asymptotic phases of learning.

Original language	English
Title of host publication	Proceedings of the 8th International Conference on Artificial Neural Networks
Editors	L. Niklasson, M. Boden, T. Ziemke
Publisher	Springer
Pages	165-170
Number of pages	6
Volume	1
ISBN (Print)	3540762639
DOIs	https://doi.org/10.1007/978-1-4471-1599-1_21
Publication status	Published - 1 Sept 1998

Bibliographical note

The original publication is available at www.springerlink.com

Keywords

natural gradient
statistical mechanics
gradient descent
transient
asymptotic

Access to Document

10.1007/978-1-4471-1599-1_21

Transients and Asymptotics of Natural Gradient Learning
The original publication is available at www.springerlink.com
Accepted author manuscript, 112 KB

Cite this

@inbook{ff2c0833fae942f488fc92fa2d6e363a,

title = "Transients and asymptotics of natural gradient learning",

abstract = "We analyse natural gradient learning in a two-layer feed-forward neural network using a statistical mechanics framework which is appropriate for large input dimension. We find significant improvement over standard gradient descent in both the transient and asymptotic phases of learning.",

keywords = "natural gradient, statistical mechanics, gradient descent, transient, asymptotic",

author = "Magnus Rattray and David Saad",

note = "The original publication is available at www.springerlink.com",

year = "1998",

month = sep,

day = "1",

doi = "10.1007/978-1-4471-1599-1_21",

language = "English",

isbn = "3540762639",

volume = "1",

pages = "165--170",

editor = "L. Niklasson and M. Boden and T. Ziemke",

booktitle = "Proceedings of the 8th International Conference on Artificial Neural Networks",

publisher = "Springer",

address = "Germany",

}

TY - CHAP

T1 - Transients and asymptotics of natural gradient learning

AU - Rattray, Magnus

AU - Saad, David

N1 - The original publication is available at www.springerlink.com

PY - 1998/9/1

Y1 - 1998/9/1

N2 - We analyse natural gradient learning in a two-layer feed-forward neural network using a statistical mechanics framework which is appropriate for large input dimension. We find significant improvement over standard gradient descent in both the transient and asymptotic phases of learning.

AB - We analyse natural gradient learning in a two-layer feed-forward neural network using a statistical mechanics framework which is appropriate for large input dimension. We find significant improvement over standard gradient descent in both the transient and asymptotic phases of learning.

KW - natural gradient

KW - statistical mechanics

KW - gradient descent

KW - transient

KW - asymptotic

UR - https://link.springer.com/chapter/10.1007/978-1-4471-1599-1_21

U2 - 10.1007/978-1-4471-1599-1_21

DO - 10.1007/978-1-4471-1599-1_21

M3 - Chapter

SN - 3540762639

VL - 1

SP - 165

EP - 170

BT - Proceedings of the 8th International Conference on Artificial Neural Networks

A2 - Niklasson, L.

A2 - Boden, M.

A2 - Ziemke, T.

PB - Springer

ER -

Transients and asymptotics of natural gradient learning

Abstract

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this