Adaptive back-propagation in on-line learning of multilayer networks

Ansgar H L West, David Saad

    Research output: Chapter in Book/Published conference outputChapter


    An adaptive back-propagation algorithm is studied and compared with gradient descent (standard back-propagation) for on-line learning in two-layer neural networks with an arbitrary number of hidden units. Within a statistical mechanics framework, both numerical studies and a rigorous analysis show that the adaptive back-propagation method results in faster training by breaking the symmetry between hidden units more efficiently and by providing faster convergence to optimal generalization than gradient descent.
    Original languageEnglish
    Title of host publicationProceedings of the neural information processing systems
    EditorsDavid S Touretzky, Michael C Mozer, Michael E. Hasselmo
    Place of PublicationBoston
    ISBN (Print)0262201070
    Publication statusPublished - 1996
    EventNeural Information Processing Systems 95 -
    Duration: 1 Jan 19961 Jan 1996


    ConferenceNeural Information Processing Systems 95

    Bibliographical note

    Copyright of the Massachusetts Institute of Technology Press (MIT Press)


    • adaptive back-propagation
    • algorithm
    • gradient descent
    • neural networks
    • statistical


    Dive into the research topics of 'Adaptive back-propagation in on-line learning of multilayer networks'. Together they form a unique fingerprint.

    Cite this