Statistical mechanics of mutual information maximization

R. Urbanczik

doi:10.1209/epl/i2000-00205-7

Statistical mechanics of mutual information maximization

R. Urbanczik^*

^*Corresponding author for this work

Aston University

Research output: Contribution to journal › Article › peer-review

Abstract

An unsupervised learning procedure based on maximizing the mutual information between the outputs of two networks receiving different but statistically dependent inputs is analyzed (Becker S. and Hinton G., Nature, 355 (1992) 161). By exploiting a formal analogy to supervised learning in parity machines, the theory of zero-temperature Gibbs learning for the unsupervised procedure is presented for the case that the networks are perceptrons and for the case of fully connected committees.

Original language	English
Pages (from-to)	685-691
Number of pages	7
Journal	Europhysics Letters
Volume	49
Issue number	5
DOIs	https://doi.org/10.1209/epl/i2000-00205-7
Publication status	Published - Mar 2000

Bibliographical note

Copyright of EDP Sciences

Keywords

unsupervised learning procedure
networks
supervised learning

Access to Document

10.1209/epl/i2000-00205-7

Europhysics_Letters_49(5).pdf
Copyright of EDP Sciences
Final published version, 123 KB

Cite this

@article{642cb61a978d4487b49ef2886766a84d,

title = "Statistical mechanics of mutual information maximization",

abstract = "An unsupervised learning procedure based on maximizing the mutual information between the outputs of two networks receiving different but statistically dependent inputs is analyzed (Becker S. and Hinton G., Nature, 355 (1992) 161). By exploiting a formal analogy to supervised learning in parity machines, the theory of zero-temperature Gibbs learning for the unsupervised procedure is presented for the case that the networks are perceptrons and for the case of fully connected committees.",

keywords = "unsupervised learning procedure, networks, supervised learning",

author = "R. Urbanczik",

note = "Copyright of EDP Sciences",

year = "2000",

month = mar,

doi = "10.1209/epl/i2000-00205-7",

language = "English",

volume = "49",

pages = "685--691",

journal = "Europhysics Letters",

issn = "0295-5075",

publisher = "IOP Publishing Ltd.",

number = "5",

}

TY - JOUR

T1 - Statistical mechanics of mutual information maximization

AU - Urbanczik, R.

N1 - Copyright of EDP Sciences

PY - 2000/3

Y1 - 2000/3

N2 - An unsupervised learning procedure based on maximizing the mutual information between the outputs of two networks receiving different but statistically dependent inputs is analyzed (Becker S. and Hinton G., Nature, 355 (1992) 161). By exploiting a formal analogy to supervised learning in parity machines, the theory of zero-temperature Gibbs learning for the unsupervised procedure is presented for the case that the networks are perceptrons and for the case of fully connected committees.

AB - An unsupervised learning procedure based on maximizing the mutual information between the outputs of two networks receiving different but statistically dependent inputs is analyzed (Becker S. and Hinton G., Nature, 355 (1992) 161). By exploiting a formal analogy to supervised learning in parity machines, the theory of zero-temperature Gibbs learning for the unsupervised procedure is presented for the case that the networks are perceptrons and for the case of fully connected committees.

KW - unsupervised learning procedure

KW - networks

KW - supervised learning

UR - http://www.scopus.com/inward/record.url?scp=0034144825&partnerID=8YFLogxK

UR - http://iopscience.iop.org/0295-5075/49/5/685/pdf?ejredirect=.iopscience

U2 - 10.1209/epl/i2000-00205-7

DO - 10.1209/epl/i2000-00205-7

M3 - Article

SN - 0295-5075

VL - 49

SP - 685

EP - 691

JO - Europhysics Letters

JF - Europhysics Letters

IS - 5

ER -

Statistical mechanics of mutual information maximization

Abstract

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this