Adaptive back-propagation in on-line learning of multilayer networks

Ansgar H L West, David Saad

Research output: Chapter in Book/Report/Conference proceedingChapter

Abstract

An adaptive back-propagation algorithm is studied and compared with gradient descent (standard back-propagation) for on-line learning in two-layer neural networks with an arbitrary number of hidden units. Within a statistical mechanics framework, both numerical studies and a rigorous analysis show that the adaptive back-propagation method results in faster training by breaking the symmetry between hidden units more efficiently and by providing faster convergence to optimal generalization than gradient descent.
Original languageEnglish
Title of host publicationProceedings of the neural information processing systems
EditorsDavid S Touretzky, Michael C Mozer, Michael E. Hasselmo
Place of PublicationBoston
PublisherMIT
Volume8
ISBN (Print)0262201070
Publication statusPublished - 1996
EventNeural Information Processing Systems 95 -
Duration: 1 Jan 19961 Jan 1996

Conference

ConferenceNeural Information Processing Systems 95
Period1/01/961/01/96

Bibliographical note

Copyright of the Massachusetts Institute of Technology Press (MIT Press)

Keywords

  • adaptive back-propagation
  • algorithm
  • gradient descent
  • neural networks
  • statistical

Fingerprint Dive into the research topics of 'Adaptive back-propagation in on-line learning of multilayer networks'. Together they form a unique fingerprint.

  • Cite this

    West, A. H. L., & Saad, D. (1996). Adaptive back-propagation in on-line learning of multilayer networks. In D. S. Touretzky, M. C. Mozer, & M. E. Hasselmo (Eds.), Proceedings of the neural information processing systems (Vol. 8). MIT.