Analysis of natural gradient descent for multilayer neural networks

M. Rattray, D. Saad

Research output: Contribution to journalArticlepeer-review


Natural gradient descent (NGD) is an on-line algorithm for redefining the steepest descent direction. An analysis of NGD for training a multilayer neural network is presented using statistical mechanics. The performance can be significantly improved using NGD algorithm and can be used for both the transient and asymptotic stages of learning.
Original languageEnglish
Pages (from-to)4523-4532
Number of pages10
JournalPhysical Review E
Issue number4
Publication statusPublished - 1 Apr 1999

Bibliographical note

©1999 The American Physical Society


Dive into the research topics of 'Analysis of natural gradient descent for multilayer neural networks'. Together they form a unique fingerprint.

Cite this