The problem of computing the storage capacity of a feed-forward network, with L hidden layers, N inputs, and K units in the first hidden layer, is analyzed using techniques from statistical mechanics. We found that the storage capacity strongly depends on the network architecture αc ∼ (log K)1-1/2L and that the number of units K limits the number of possible hidden layers L through the relationship 2L - 1 < 2log K.
|Number of pages||16|
|Journal||Journal of Physics A: Mathematical and Theoretical|
|Early online date||27 Jan 2014|
|Publication status||Published - 14 Feb 2014|
Bibliographical note© 2014 IOP Publishing
- adaptive systems PACS numbers: 89.70.
- replica trick
- storage capacity