دورية أكاديمية
DTCM: Deep Transformer Capsule Mutual Distillation for Multivariate Time Series Classification
| العنوان: | DTCM: Deep Transformer Capsule Mutual Distillation for Multivariate Time Series Classification |
|---|---|
| المؤلفون: | Xiao, Zhiwen, Xu, Xin, Xing, Huanlai, Zhao, Bowen, Wang, Xinhan, Song, Fuhong, Qu, Rong, Feng, Li |
| بيانات النشر: | Institute of Electrical and Electronics Engineers |
| سنة النشر: | 2024 |
| المجموعة: | University of Nottingham: Repository@Nottingham |
| مصطلحات موضوعية: | Feature extraction, Classification algorithms, Time series analysis, Data mining, Transformers, Routing, Knowledge transfer, Capsule Network, Deep Learning, Knowledge Distillation, Multivariate Time Series Classification, Mutual L |
| الوصف: | This paper proposes a dual-network-based feature extractor, perceptive capsule network (PCapN), for multivariate time series classification (MTSC), including a local feature network (LFN) and a global relation network (GRN). The LFN has two heads (i.e., Head_A and Head_B), each containing two squash CNN blocks and one dynamic routing block to extract the local features from the data and mine the connections among them. The GRN consists of two capsule-based transformer blocks and one dynamic routing block to capture the global patterns of each variable and correlate the useful information of multiple variables. Unfortunately, it is difficult to directly deploy PCapN on mobile devices due to its strict requirement for computing resources. So, this paper designs a lightweight capsule network (LCapN) to mimic the cumbersome PCapN. To promote knowledge transfer from PCapN to LCapN, this paper proposes a deep transformer capsule mutual (DTCM) distillation method. It is targeted and offline, using one- and two-way operations to supervise the knowledge distillation process for the dual-network-based student and teacher models. Experimental results show that the proposed PCapN and DTCM achieve excellent performance on UEA2018 datasets regarding top-1 accuracy. |
| نوع الوثيقة: | article in journal/newspaper |
| اللغة: | unknown |
| تدمد: | 2379-8920 |
| العلاقة: | https://nottingham-repository.worktribe.com/output/32450554Test; IEEE Transactions on Cognitive and Developmental Systems; https://nottingham-repository.worktribe.com/file/32450554/1/TCDS24-HuanlaiTest |
| DOI: | 10.1109/tcds.2024.3370219 |
| الإتاحة: | https://doi.org/10.1109/tcds.2024.3370219Test https://nottingham-repository.worktribe.com/file/32450554/1/TCDS24-HuanlaiTest https://nottingham-repository.worktribe.com/output/32450554Test |
| حقوق: | openAccess |
| رقم الانضمام: | edsbas.31715B52 |
| قاعدة البيانات: | BASE |
| تدمد: | 23798920 |
|---|---|
| DOI: | 10.1109/tcds.2024.3370219 |