Attention-aided partial bidirectional RNN-based nonlinear equalizer in coherent optical systems

Yifan Liu, Victor Sanchez, Pedro J. Freire, Jaroslaw E. Prilepsky, Mahyar J. Koshkouei, Matthew D. Higgins

Research output: Contribution to journalArticlepeer-review


We leverage the attention mechanism to investigate and comprehend the contribution of each input symbol of the input sequence and their hidden representations for predicting the received symbol in the bidirectional recurrent neural network (BRNN)-based nonlinear equalizer. In this paper, we propose an attention-aided novel design of a partial BRNN-based nonlinear equalizer, and evaluate with both LSTM and GRU units in a single-channel DP-64QAM 30Gbaud coherent optical communication systems of 20 × 50 km standard single-mode fiber (SSMF) spans. Our approach maintains the Q-factor performance of the baseline equalizer with a significant complexity reduction of ∼56.16% in the number of real multiplications required to equalize per symbol (RMpS). In comparison of the performance under similar complexity, our approach outperforms the baseline by ∼0.2dB to ∼0.25dB at the optimal transmit power, and ∼0.3dB to ∼0.45dB towards the more nonlinear region.
Original languageEnglish
Article number32908
Pages (from-to)32908-32923
Number of pages16
JournalOptics Express
Issue number18
Early online date24 Aug 2022
Publication statusPublished - 29 Aug 2022

Bibliographical note

Published by Optica Publishing Group under the terms of the Creative Commons Attribution 4.0 License. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.

Funding Information:
H2020 Marie Skłodowska-Curie Actions (813144); Leverhulme Trust (RP-2018-063); Engineering and Physical Sciences Research Council (EP/N509796/1, EP/R513374/1).


  • Atomic and Molecular Physics, and Optics


Dive into the research topics of 'Attention-aided partial bidirectional RNN-based nonlinear equalizer in coherent optical systems'. Together they form a unique fingerprint.

Cite this