Storage capacity of ultrametric committee machines

Research output: Contribution to journalArticle

Abstract

The problem of computing the storage capacity of a feed-forward network, with L hidden layers, N inputs, and K units in the first hidden layer, is analyzed using techniques from statistical mechanics. We found that the storage capacity strongly depends on the network architecture αc ∼ (log K)1-1/2L and that the number of units K limits the number of possible hidden layers L through the relationship 2L - 1 < 2log K.

Original languageEnglish
Article number065002
Number of pages16
JournalJournal of Physics A: Mathematical and Theoretical
Volume47
Issue number6
Early online date27 Jan 2014
DOIs
Publication statusPublished - 14 Feb 2014

Fingerprint

Statistical mechanics
Storage Capacity
Feedforward Networks
Unit
statistical mechanics
Statistical Mechanics
Computing

Bibliographical note

© 2014 IOP Publishing

Keywords

  • 84.35.+i
  • adaptive systems PACS numbers: 89.70.
  • replica trick
  • storage capacity

Cite this

@article{39f0f8642f47493c9c2e19740034d4d7,
title = "Storage capacity of ultrametric committee machines",
abstract = "The problem of computing the storage capacity of a feed-forward network, with L hidden layers, N inputs, and K units in the first hidden layer, is analyzed using techniques from statistical mechanics. We found that the storage capacity strongly depends on the network architecture αc ∼ (log K)1-1/2L and that the number of units K limits the number of possible hidden layers L through the relationship 2L - 1 < 2log K.",
keywords = "84.35.+i, adaptive systems PACS numbers: 89.70., replica trick, storage capacity",
author = "J.P. Neirotti",
note = "{\circledC} 2014 IOP Publishing",
year = "2014",
month = "2",
day = "14",
doi = "10.1088/1751-8113/47/6/065002",
language = "English",
volume = "47",
journal = "Journal of Physics A: Mathematical and Theoretical",
issn = "1751-8113",
publisher = "IOP Publishing Ltd.",
number = "6",

}

Storage capacity of ultrametric committee machines. / Neirotti, J.P.

In: Journal of Physics A: Mathematical and Theoretical, Vol. 47, No. 6, 065002, 14.02.2014.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Storage capacity of ultrametric committee machines

AU - Neirotti, J.P.

N1 - © 2014 IOP Publishing

PY - 2014/2/14

Y1 - 2014/2/14

N2 - The problem of computing the storage capacity of a feed-forward network, with L hidden layers, N inputs, and K units in the first hidden layer, is analyzed using techniques from statistical mechanics. We found that the storage capacity strongly depends on the network architecture αc ∼ (log K)1-1/2L and that the number of units K limits the number of possible hidden layers L through the relationship 2L - 1 < 2log K.

AB - The problem of computing the storage capacity of a feed-forward network, with L hidden layers, N inputs, and K units in the first hidden layer, is analyzed using techniques from statistical mechanics. We found that the storage capacity strongly depends on the network architecture αc ∼ (log K)1-1/2L and that the number of units K limits the number of possible hidden layers L through the relationship 2L - 1 < 2log K.

KW - 84.35.+i

KW - adaptive systems PACS numbers: 89.70.

KW - replica trick

KW - storage capacity

UR - http://www.scopus.com/inward/record.url?scp=84893428494&partnerID=8YFLogxK

U2 - 10.1088/1751-8113/47/6/065002

DO - 10.1088/1751-8113/47/6/065002

M3 - Article

VL - 47

JO - Journal of Physics A: Mathematical and Theoretical

JF - Journal of Physics A: Mathematical and Theoretical

SN - 1751-8113

IS - 6

M1 - 065002

ER -