Complex-Valued Neural Network Design for Mitigation of Signal Distortions in Optical Links

Pedro J. Freire, Vladislav Neskorniuk, Antonio Napoli, Bernhard Spinnler, Nelson Costa, Ginni Khanna, Emilio Riccardi, Jaroslaw E. Prilepsky, Sergei K. Turitsyn

Research output: Contribution to journalArticlepeer-review


Nonlinearity compensation is considered as a key enabler to increase channel transmission rates in the installed optical communication systems. Recently, data-driven approaches - motivated by modern machine learning techniques - have been proposed for optical communications in place of traditional model-based counterparts. In particular, the application of neural networks (NN) allows improving the performance of complex modern fiber-optic systems without relying on any a priori knowledge of their specific parameters. In this work, we introduce a novel design of complex-valued NN for optical systems and examine its performance in standard single mode fiber (SSMF) and large effective-area fiber (LEAF) links operating in relatively high nonlinear regime. First, we present a methodology to design a new type of NN based on the assumption that the channel model is more accurate in the nonlinear regime. Second, we implement a Bayesian optimizer to jointly adapt the size of the NN and its number of input taps depending on the different fiber properties and total length. Finally, the proposed NN is numerically and experimentally validated showing an improvement of 1.7 dB in the linear regime, 2.04 dB at the optimal optical power and 2.61 at the max available power on Q-factor when transmitting a WDM 30 × 200G DP-16QAM signal over a 612 km SSMF legacy link. The results highlight that the NN is able to mitigate not only part of the nonlinear impairments caused by optical fiber propagation but also imperfections resulting from using low-cost legacy transceiver components, such as digital-to-analog converter (DAC) and Mach-Zehnder modulator.

Original languageEnglish
Article number9280329
Pages (from-to)1696 - 1705
Number of pages10
JournalJournal of Lightwave Technology
Issue number6
Early online date3 Dec 2020
Publication statusPublished - 15 Mar 2021

Bibliographical note

© 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

Funding: This paper was supported by the EU Horizon 2020 program under the
Marie Skodowska-Curie grant agreements No.766115 (FONTE) and 813144


  • Bayesian optimizer
  • Neural network
  • channel model
  • coherent detection
  • metropolitan links
  • nonlinear equalizer


Dive into the research topics of 'Complex-Valued Neural Network Design for Mitigation of Signal Distortions in Optical Links'. Together they form a unique fingerprint.

Cite this