Analysis of natural gradient descent for multilayer neural networks

M. Rattray; D. Saad

Analysis of natural gradient descent for multilayer neural networks

M. Rattray
, D. Saad

Aston University

Manchester Metropolitan University

Research output: Contribution to journal › Article › peer-review

19 Citations (Scopus)

16 Downloads (Pure)

Abstract

Natural gradient descent (NGD) is an on-line algorithm for redefining the steepest descent direction. An analysis of NGD for training a multilayer neural network is presented using statistical mechanics. The performance can be significantly improved using NGD algorithm and can be used for both the transient and asymptotic stages of learning.

Original language	English
Pages (from-to)	4523-4532
Number of pages	10
Journal	Physical Review E
Volume	59
Issue number	4
Publication status	Published - 1 Apr 1999

Bibliographical note

Access to Document

Analysis of natural gradient descent for multilayer neural networks
©1999 The American Physical Society
Final published version, 221 KB

Cite this

@article{f4231704ec904105a47aab988112b468,

title = "Analysis of natural gradient descent for multilayer neural networks",

abstract = "Natural gradient descent (NGD) is an on-line algorithm for redefining the steepest descent direction. An analysis of NGD for training a multilayer neural network is presented using statistical mechanics. The performance can be significantly improved using NGD algorithm and can be used for both the transient and asymptotic stages of learning.",

author = "M. Rattray and D. Saad",

note = "{\textcopyright}1999 The American Physical Society",

year = "1999",

month = apr,

day = "1",

language = "English",

volume = "59",

pages = "4523--4532",

journal = "Physical Review E",

issn = "1063-651X",

publisher = "American Physical Society",

number = "4",

}

TY - JOUR

T1 - Analysis of natural gradient descent for multilayer neural networks

AU - Rattray, M.

AU - Saad, D.

PY - 1999/4/1

Y1 - 1999/4/1

N2 - Natural gradient descent (NGD) is an on-line algorithm for redefining the steepest descent direction. An analysis of NGD for training a multilayer neural network is presented using statistical mechanics. The performance can be significantly improved using NGD algorithm and can be used for both the transient and asymptotic stages of learning.

AB - Natural gradient descent (NGD) is an on-line algorithm for redefining the steepest descent direction. An analysis of NGD for training a multilayer neural network is presented using statistical mechanics. The performance can be significantly improved using NGD algorithm and can be used for both the transient and asymptotic stages of learning.

UR - https://www.scopus.com/pages/publications/0032613639

UR - https://journals.aps.org/pre/pdf/10.1103/PhysRevE.59.4523

M3 - Article

AN - SCOPUS:0032613639

SN - 1063-651X

VL - 59

SP - 4523

EP - 4532

JO - Physical Review E

JF - Physical Review E

IS - 4

ER -

Analysis of natural gradient descent for multilayer neural networks

Abstract

Bibliographical note

Access to Document

Other files and links

Fingerprint

Cite this