Computationally efficient natural gradient descent

Abstract

This study examines the use of matrix momentum terms with the aim of creating a more computationally efficient natural gradient descent algorithm for on-line learning. It uses the statistical mechanics framework created by Saad and Solla to describe the evolution of order parameters for this algorithm in a two-layer student-teacher scenario, and compares this with results from matrix-momentum natural gradient learning of real datasets.

Date of Award	1998
Original language	English
Awarding Institution	Aston University