Lightweight Spectrum Prediction Based on Knowledge Distillation
Loading...
Date
2023-12
Authors
Cheng, R.
Zhang, J.
Deng, J.
Zhu, Y.
ORCID
Advisor
Referee
Mark
Journal Title
Journal ISSN
Volume Title
Publisher
Společnost pro radioelektronické inženýrství
Altmetrics
Abstract
To address the challenges of increasing complexity and larger number of training samples required for high-accuracy spectrum prediction, we propose a novel lightweight model, leveraging a temporal convolutional network (TCN) and knowledge distillation. First, the prediction accuracy of TCN is enhanced via a self-transfer method. Then, we design a two-branch network which can extract the spectrum features efficiently. By employing knowledge distillation, we transfer the knowledge from TCN to the two-branch network, resulting in improved accuracy for spectrum prediction of the lightweight network. Experimental results show that the proposed model can improve accuracy by 19.5% compared to the widely-used LSTM model with sufficient historical data and reduces 71.1% parameters to be trained. Furthermore, the prediction accuracy is improved by 17.9% compared to Gated Recurrent Units (GRU) in the scenarios with scarce historical data.
Description
Citation
Radioengineering. 2023 vol. 32, č. 4, s. 469-478. ISSN 1210-2512
https://www.radioeng.cz/fulltexts/2023/23_04_0469_0478.pdf
https://www.radioeng.cz/fulltexts/2023/23_04_0469_0478.pdf
Document type
Peer-reviewed
Document version
Published version
Date of access to the full text
Language of document
en