Learning with noise and regularizers in multilayer neural networks

David Saad, Sara A. Solla

Research output: Contribution to journalArticle

Abstract

We study the effect of two types of noise, data noise and model noise, in an on-line gradient-descent learning scenario for general two-layer student network with an arbitrary number of hidden units. Training examples are randomly drawn input vectors labeled by a two-layer teacher network with an arbitrary number of hidden units. Data is then corrupted by Gaussian noise affecting either the output or the model itself. We examine the effect of both types of noise on the evolution of order parameters and the generalization error in various phases of the learning process.
Original languageEnglish
Pages (from-to)260-266
Number of pages7
JournalAdvances in Neural Information Processing Systems
Volume9
Publication statusPublished - 1996
Event10th Annual Conference on Neural Information Processing Systems, NIPS 1996 - Denver, CO, United Kingdom
Duration: 2 Dec 19965 Dec 1996

Fingerprint

learning
instructors
descent
random noise
students
education
gradients
output

Bibliographical note

Copiright of Massachusetts Institute of Technology Press (MIT Press)

Keywords

  • noise
  • data noise
  • model noise
  • gradient-descent learning
  • vectors
  • gaussian noise
  • error

Cite this

@article{30d7aeb4742f49ae80463c62d35f690b,
title = "Learning with noise and regularizers in multilayer neural networks",
abstract = "We study the effect of two types of noise, data noise and model noise, in an on-line gradient-descent learning scenario for general two-layer student network with an arbitrary number of hidden units. Training examples are randomly drawn input vectors labeled by a two-layer teacher network with an arbitrary number of hidden units. Data is then corrupted by Gaussian noise affecting either the output or the model itself. We examine the effect of both types of noise on the evolution of order parameters and the generalization error in various phases of the learning process.",
keywords = "noise, data noise, model noise, gradient-descent learning, vectors, gaussian noise, error",
author = "David Saad and Solla, {Sara A.}",
note = "Copiright of Massachusetts Institute of Technology Press (MIT Press)",
year = "1996",
language = "English",
volume = "9",
pages = "260--266",
journal = "Advances in Neural Information Processing Systems",
issn = "1049-5258",

}

Learning with noise and regularizers in multilayer neural networks. / Saad, David; Solla, Sara A.

In: Advances in Neural Information Processing Systems, Vol. 9, 1996, p. 260-266.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Learning with noise and regularizers in multilayer neural networks

AU - Saad, David

AU - Solla, Sara A.

N1 - Copiright of Massachusetts Institute of Technology Press (MIT Press)

PY - 1996

Y1 - 1996

N2 - We study the effect of two types of noise, data noise and model noise, in an on-line gradient-descent learning scenario for general two-layer student network with an arbitrary number of hidden units. Training examples are randomly drawn input vectors labeled by a two-layer teacher network with an arbitrary number of hidden units. Data is then corrupted by Gaussian noise affecting either the output or the model itself. We examine the effect of both types of noise on the evolution of order parameters and the generalization error in various phases of the learning process.

AB - We study the effect of two types of noise, data noise and model noise, in an on-line gradient-descent learning scenario for general two-layer student network with an arbitrary number of hidden units. Training examples are randomly drawn input vectors labeled by a two-layer teacher network with an arbitrary number of hidden units. Data is then corrupted by Gaussian noise affecting either the output or the model itself. We examine the effect of both types of noise on the evolution of order parameters and the generalization error in various phases of the learning process.

KW - noise

KW - data noise

KW - model noise

KW - gradient-descent learning

KW - vectors

KW - gaussian noise

KW - error

UR - http://www.scopus.com/inward/record.url?scp=4944221942&partnerID=8YFLogxK

UR - http://mitpress.mit.edu/catalog/item/default.asp?ttype=2&tid=3990

M3 - Article

VL - 9

SP - 260

EP - 266

JO - Advances in Neural Information Processing Systems

JF - Advances in Neural Information Processing Systems

SN - 1049-5258

ER -