On-line learning of unrealizable tasks

Silvia Scarpetta, David Saad

Research output: Contribution to journalArticle

Abstract

The dynamics of on-line learning is investigated for structurally unrealizable tasks in the context of two-layer neural networks with an arbitrary number of hidden neurons. Within a statistical mechanics framework, a closed set of differential equations describing the learning dynamics can be derived, for the general case of unrealizable isotropic tasks. In the asymptotic regime one can solve the dynamics analytically in the limit of large number of hidden neurons, providing an analytical expression for the residual generalization error, the optimal and critical asymptotic training parameters, and the corresponding prefactor of the generalization error decay.
Original languageEnglish
Pages (from-to)5902-5911
Number of pages10
JournalPhysical Review E
Volume60
Issue number5
DOIs
Publication statusPublished - Nov 1999

Bibliographical note

Copyright of the American Physical Society

Keywords

  • on-line learning
  • neural networks
  • neurons
  • asymptotic regime one
  • residual generalization error
  • asymptotic training parameters
  • generalization error decay

Fingerprint Dive into the research topics of 'On-line learning of unrealizable tasks'. Together they form a unique fingerprint.

  • Cite this