Exploring the Function Space of Deep-Learning Machines

Bo Li, David Saad

Research output: Contribution to journalArticle

Abstract

The function space of deep-learning machines is investigated by studying growth in the entropy of functions of a given error with respect to a reference function, realized by a deep-learning machine. Using physics-inspired methods we study both sparsely and densely connected architectures to discover a layerwise convergence of candidate functions, marked by a corresponding reduction in entropy when approaching the reference function, gain insight into the importance of having a large number of layers, and observe phase transitions as the error increases.

Original languageEnglish
Article number248301
JournalPhysical Review Letters
Volume120
Issue number24
Early online date12 Jun 2018
DOIs
Publication statusPublished - 12 Jun 2018

Bibliographical note

© 2018 The American Physical Society. Exploring the Function Space of Deep-Learning Machines. Bo Li and David Saad. Phys. Rev. Lett. 120, 248301 – Published 12 June 2018

Keywords

  • deep-learning, statistical physics, machine learning

Fingerprint Dive into the research topics of 'Exploring the Function Space of Deep-Learning Machines'. Together they form a unique fingerprint.

  • Cite this