Learning in ultrametric committee machines

Research output: Contribution to journalArticle

Abstract

The problem of learning by examples in ultrametric committee machines (UCMs) is studied within the framework of statistical mechanics. Using the replica formalism we calculate the average generalization error in UCMs with L hidden layers and for a large enough number of units. In most of the regimes studied we find that the generalization error, as a function of the number of examples presented, develops a discontinuous drop at a critical value of the load parameter. We also find that when L>1 a number of teacher networks with the same number of hidden layers and different overlaps induce learning processes with the same critical points.
Original languageEnglish
Pages (from-to)887-897
Number of pages11
JournalJournal of Statistical Physics
Volume149
Issue number5
Early online date21 Nov 2012
DOIs
Publication statusPublished - Nov 2012

Fingerprint

learning
Generalization Error
instructors
replicas
statistical mechanics
critical point
formalism
Replica
Learning Process
Statistical Mechanics
Critical value
Overlap
Critical point
Calculate
Unit
Learning

Bibliographical note

The original publication is available at www.springerlink.com

Keywords

  • multilayered networks
  • learning by examples

Cite this

@article{b9da31ee05a34737bd45d59068af212e,
title = "Learning in ultrametric committee machines",
abstract = "The problem of learning by examples in ultrametric committee machines (UCMs) is studied within the framework of statistical mechanics. Using the replica formalism we calculate the average generalization error in UCMs with L hidden layers and for a large enough number of units. In most of the regimes studied we find that the generalization error, as a function of the number of examples presented, develops a discontinuous drop at a critical value of the load parameter. We also find that when L>1 a number of teacher networks with the same number of hidden layers and different overlaps induce learning processes with the same critical points.",
keywords = "multilayered networks, learning by examples",
author = "Juan Neirotti",
note = "The original publication is available at www.springerlink.com",
year = "2012",
month = "11",
doi = "10.1007/s10955-012-0636-1",
language = "English",
volume = "149",
pages = "887--897",
journal = "Journal of Statistical Physics",
issn = "0022-4715",
publisher = "Springer",
number = "5",

}

Learning in ultrametric committee machines. / Neirotti, Juan.

In: Journal of Statistical Physics, Vol. 149, No. 5, 11.2012, p. 887-897.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Learning in ultrametric committee machines

AU - Neirotti, Juan

N1 - The original publication is available at www.springerlink.com

PY - 2012/11

Y1 - 2012/11

N2 - The problem of learning by examples in ultrametric committee machines (UCMs) is studied within the framework of statistical mechanics. Using the replica formalism we calculate the average generalization error in UCMs with L hidden layers and for a large enough number of units. In most of the regimes studied we find that the generalization error, as a function of the number of examples presented, develops a discontinuous drop at a critical value of the load parameter. We also find that when L>1 a number of teacher networks with the same number of hidden layers and different overlaps induce learning processes with the same critical points.

AB - The problem of learning by examples in ultrametric committee machines (UCMs) is studied within the framework of statistical mechanics. Using the replica formalism we calculate the average generalization error in UCMs with L hidden layers and for a large enough number of units. In most of the regimes studied we find that the generalization error, as a function of the number of examples presented, develops a discontinuous drop at a critical value of the load parameter. We also find that when L>1 a number of teacher networks with the same number of hidden layers and different overlaps induce learning processes with the same critical points.

KW - multilayered networks

KW - learning by examples

UR - http://www.scopus.com/inward/record.url?scp=84870736543&partnerID=8YFLogxK

U2 - 10.1007/s10955-012-0636-1

DO - 10.1007/s10955-012-0636-1

M3 - Article

VL - 149

SP - 887

EP - 897

JO - Journal of Statistical Physics

JF - Journal of Statistical Physics

SN - 0022-4715

IS - 5

ER -