Exploring the Function Space of Deep-Learning Machines

Bo Li, David Saad

    Research output: Contribution to journalArticlepeer-review


    The function space of deep-learning machines is investigated by studying growth in the entropy of functions of a given error with respect to a reference function, realized by a deep-learning machine. Using physics-inspired methods we study both sparsely and densely connected architectures to discover a layerwise convergence of candidate functions, marked by a corresponding reduction in entropy when approaching the reference function, gain insight into the importance of having a large number of layers, and observe phase transitions as the error increases.

    Original languageEnglish
    Article number248301
    JournalPhysical Review Letters
    Issue number24
    Early online date12 Jun 2018
    Publication statusPublished - 12 Jun 2018

    Bibliographical note

    © 2018 The American Physical Society. Exploring the Function Space of Deep-Learning Machines. Bo Li and David Saad. Phys. Rev. Lett. 120, 248301 – Published 12 June 2018


    • deep-learning, statistical physics, machine learning


    Dive into the research topics of 'Exploring the Function Space of Deep-Learning Machines'. Together they form a unique fingerprint.

    Cite this