Towards Transformer-Based Flow Volume Prediction in Network Traffic

Accurately predicting network flow volume using early packet-level features is critical for real-time applications such as resource allocation, bottleneck prediction, anomaly detection and more. In this paper, we investigate the suitability of Transformer-based architectures for the flow volume regression task. Specifically, we fine-tune two foundation models, NetFound and YaTC, originally pre-trained to learn traffic representations for classification tasks, and adapt them for flow volume regression. Additionally, we train a lightweight Transformer model, and compare the predictive performance and complexity of all Transformer-based models to a simple Multi-Layer Perceptron (MLP) model. Fine-tuning and training are conducted on two public datasets: MAWI and CIC-IDS-2017. Results demonstrate that Transformer-based models significantly outperform the MLP model. The lightweight Transformer achieves the best $\mathrm{R}^{\mathrm{2}}$ score of 0.963 on CIC-IDS-2017, while NetFound achieves the best performance on the real-world traffic dataset MAWI with an $\mathrm{R}^{\mathrm{2}}$ of 0.812. These findings highlight the effectiveness of Transformers in capturing complex relationships in early packet-level features of network traffic for flow volume regression task.