Robust neural networks using stochastic resonance neurons

Egor Manuylovich; Diego Argüello Ron; Morteza Kamalian-Kopae; Sergei K Turitsyn

doi:10.1038/s44172-024-00314-0

Robust neural networks using stochastic resonance neurons

Egor Manuylovich, Diego Argüello Ron, Morteza Kamalian-Kopae, Sergei K Turitsyn

Aston Institute of Photonic Technologies (AiPT)

Research output: Contribution to journal › Article › peer-review

Abstract

Various successful applications of deep artificial neural networks are effectively facilitated by the possibility to increase the number of layers and neurons in the network at the expense of the growing computational complexity. Increasing computational complexity to improve performance makes hardware implementation more difficult and directly affects both power consumption and the accumulation of signal processing latency, which are critical issues in many applications. Power consumption can be potentially reduced using analog neural networks, the performance of which, however, is limited by noise aggregation. Following the idea of physics-inspired machine learning, we propose here a type of neural network using stochastic resonances as a dynamic nonlinear node and demonstrate the possibility of considerably reducing the number of neurons required for a given prediction accuracy. We also observe that the performance of such neural networks is more robust against the impact of noise in the training data compared to conventional networks.

Original language	English
Article number	169
Number of pages	7
Journal	Communications Engineering
Volume	3
Issue number	1
Early online date	13 Nov 2024
DOIs	https://doi.org/10.1038/s44172-024-00314-0
Publication status	Published - 13 Nov 2024

Bibliographical note

Copyright © The Author(s). This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to
obtain permission directly from the copyright holder. To view a copy of this licence, visit https://creativecommons.org/licenses/by/4.0/.

Data Access Statement

The data that supports the findings of this study is available from the corresponding author upon reasonable request.

Access to Document

10.1038/s44172-024-00314-0Licence: CC BY 4.0

Manuylovich-2024-Robust-neural-networks-using-stocha
Copyright © The Author(s). This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit https://creativecommons.org/licenses/by/4.0/.
Final published version, 910 KBLicence: CC BY 4.0

Cite this

@article{c864f7b53d5943748997f7ab6ddd7908,

title = "Robust neural networks using stochastic resonance neurons",

abstract = "Various successful applications of deep artificial neural networks are effectively facilitated by the possibility to increase the number of layers and neurons in the network at the expense of the growing computational complexity. Increasing computational complexity to improve performance makes hardware implementation more difficult and directly affects both power consumption and the accumulation of signal processing latency, which are critical issues in many applications. Power consumption can be potentially reduced using analog neural networks, the performance of which, however, is limited by noise aggregation. Following the idea of physics-inspired machine learning, we propose here a type of neural network using stochastic resonances as a dynamic nonlinear node and demonstrate the possibility of considerably reducing the number of neurons required for a given prediction accuracy. We also observe that the performance of such neural networks is more robust against the impact of noise in the training data compared to conventional networks.",

author = "Egor Manuylovich and {Arg{\"u}ello Ron}, Diego and Morteza Kamalian-Kopae and Turitsyn, {Sergei K}",

note = "Copyright {\textcopyright} The Author(s). This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article{\textquoteright}s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article{\textquoteright}s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit https://creativecommons.org/licenses/by/4.0/.",

year = "2024",

month = nov,

day = "13",

doi = "10.1038/s44172-024-00314-0",

language = "English",

volume = "3",

number = "1",

}

TY - JOUR

T1 - Robust neural networks using stochastic resonance neurons

AU - Manuylovich, Egor

AU - Argüello Ron, Diego

AU - Kamalian-Kopae, Morteza

AU - Turitsyn, Sergei K

N1 - Copyright © The Author(s). This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit https://creativecommons.org/licenses/by/4.0/.

PY - 2024/11/13

Y1 - 2024/11/13

N2 - Various successful applications of deep artificial neural networks are effectively facilitated by the possibility to increase the number of layers and neurons in the network at the expense of the growing computational complexity. Increasing computational complexity to improve performance makes hardware implementation more difficult and directly affects both power consumption and the accumulation of signal processing latency, which are critical issues in many applications. Power consumption can be potentially reduced using analog neural networks, the performance of which, however, is limited by noise aggregation. Following the idea of physics-inspired machine learning, we propose here a type of neural network using stochastic resonances as a dynamic nonlinear node and demonstrate the possibility of considerably reducing the number of neurons required for a given prediction accuracy. We also observe that the performance of such neural networks is more robust against the impact of noise in the training data compared to conventional networks.

AB - Various successful applications of deep artificial neural networks are effectively facilitated by the possibility to increase the number of layers and neurons in the network at the expense of the growing computational complexity. Increasing computational complexity to improve performance makes hardware implementation more difficult and directly affects both power consumption and the accumulation of signal processing latency, which are critical issues in many applications. Power consumption can be potentially reduced using analog neural networks, the performance of which, however, is limited by noise aggregation. Following the idea of physics-inspired machine learning, we propose here a type of neural network using stochastic resonances as a dynamic nonlinear node and demonstrate the possibility of considerably reducing the number of neurons required for a given prediction accuracy. We also observe that the performance of such neural networks is more robust against the impact of noise in the training data compared to conventional networks.

UR - https://www.nature.com/articles/s44172-024-00314-0

UR - http://www.scopus.com/inward/record.url?scp=85209185538&partnerID=8YFLogxK

U2 - 10.1038/s44172-024-00314-0

DO - 10.1038/s44172-024-00314-0

M3 - Article

C2 - 39537964

SN - 2731-3395

VL - 3

JO - Communications Engineering

JF - Communications Engineering

IS - 1

M1 - 169

ER -

Robust neural networks using stochastic resonance neurons

Abstract

Bibliographical note

Data Access Statement

Access to Document

Other files and links

Fingerprint

Cite this