المؤلفون: Zhao, Bowen, Xing, Huanlai, Wang, Xinhan, Song, Fuhong, Xiao, Zhiwen

مصطلحات موضوعية: Computer Science - Machine Learning, Computer Science - Artificial Intelligence

الوصف: Attention-based models have been widely used in many areas, such as computer vision and natural language processing. However, relevant applications in time series classification (TSC) have not been explored deeply yet, causing a significant number of TSC algorithms still suffer from general problems of attention mechanism, like quadratic complexity. In this paper, we promote the efficiency and performance of the attention mechanism by proposing our flexible multi-head linear attention (FMLA), which enhances locality awareness by layer-wise interactions with deformable convolutional blocks and online knowledge distillation. What's more, we propose a simple but effective mask mechanism that helps reduce the noise influence in time series and decrease the redundancy of the proposed FMLA by masking some positions of each given series proportionally. To stabilize this mechanism, samples are forwarded through the model with random mask layers several times and their outputs are aggregated to teach the same model with regular mask layers. We conduct extensive experiments on 85 UCR2018 datasets to compare our algorithm with 11 well-known ones and the results show that our algorithm has comparable performance in terms of top-1 accuracy. We also compare our model with three Transformer-based models with respect to the floating-point operations per second and number of parameters and find that our algorithm achieves significantly better efficiency with lower complexity.

الوصول الحر: http://arxiv.org/abs/2207.07564Test

المؤلفون: Song, Fuhong, Xing, Huanlai, Wang, Xinhan, Luo, Shouxi, Dai, Penglin, Xiao, Zhiwen, Zhao, Bowen

مصطلحات موضوعية: Electrical Engineering and Systems Science - Signal Processing, Computer Science - Machine Learning

الوصف: This paper studies the trajectory control and task offloading (TCTO) problem in an unmanned aerial vehicle (UAV)-assisted mobile edge computing system, where a UAV flies along a planned trajectory to collect computation tasks from smart devices (SDs). We consider a scenario that SDs are not directly connected by the base station (BS) and the UAV has two roles to play: MEC server or wireless relay. The UAV makes task offloading decisions online, in which the collected tasks can be executed locally on the UAV or offloaded to the BS for remote processing. The TCTO problem involves multi-objective optimization as its objectives are to minimize the task delay and the UAV's energy consumption, and maximize the number of tasks collected by the UAV, simultaneously. This problem is challenging because the three objectives conflict with each other. The existing reinforcement learning (RL) algorithms, either single-objective RLs or single-policy multi-objective RLs, cannot well address the problem since they cannot output multiple policies for various preferences (i.e. weights) across objectives in a single run. This paper adapts the evolutionary multi-objective RL (EMORL), a multi-policy multi-objective RL, to the TCTO problem. This algorithm can output multiple optimal policies in just one run, each optimizing a certain preference. The simulation results demonstrate that the proposed algorithm can obtain more excellent nondominated policies by striking a balance between the three objectives regarding policy quality, compared with two evolutionary and two multi-policy RL algorithms.

الوصول الحر: http://arxiv.org/abs/2202.12028Test

المؤلفون: Xing, Huanlai, Xiao, Zhiwen, Qu, Rong, Zhu, Zonghai, Zhao, Bowen

مصطلحات موضوعية: Computer Science - Machine Learning, Computer Science - Artificial Intelligence

الوصف: This paper proposes an efficient federated distillation learning system (EFDLS) for multi-task time series classification (TSC). EFDLS consists of a central server and multiple mobile users, where different users may run different TSC tasks. EFDLS has two novel components, namely a feature-based student-teacher (FBST) framework and a distance-based weights matching (DBWM) scheme. Within each user, the FBST framework transfers knowledge from its teacher's hidden layers to its student's hidden layers via knowledge distillation, with the teacher and student having identical network structure. For each connected user, its student model's hidden layers' weights are uploaded to the EFDLS server periodically. The DBWM scheme is deployed on the server, with the least square distance used to measure the similarity between the weights of two given models. This scheme finds a partner for each connected user such that the user's and its partner's weights are the closest among all the weights uploaded. The server exchanges and sends back the user's and its partner's weights to these two users which then load the received weights to their teachers' hidden layers. Experimental results show that the proposed EFDLS achieves excellent performance on a set of selected UCR2018 datasets regarding top-1 accuracy.
Comment: 11 pages

الوصول الحر: http://arxiv.org/abs/2201.00011Test

المؤلفون: Xiao, Zhiwen, Xing, Huanlai, Qu, Rong, Feng, Li, Luo, Shouxi, Dai, Penglin, Zhao, Bowen, Dai, Yuanshun

مصطلحات موضوعية: Electrical and Electronic Engineering, Computer Science Applications, Human-Computer Interaction, Control and Systems Engineering, Software

الوصف: Multivariate time series classification (MTSC) based on deep learning (DL) has attracted increasingly more research attention. The performance of a DL-based MTSC algorithm is heavily dependent on the quality of the learned representations providing semantic information for downstream tasks, e.g., classification. Hence, a model’s representation learning ability is critical for enhancing its performance. This article proposes a densely knowledge-aware network (DKN) for MTSC. The DKN’s feature extractor consists of a residual multihead convolutional network (ResMulti) and a transformer-based network (Trans), called ResMulti-Trans. ResMulti has five residual multihead blocks for capturing the local patterns of data while Trans has three transformer blocks for extracting the global patterns of data. Besides, to enable dense mutual supervision between lower- and higher-level semantic information, this article adapts densely dual self-distillation (DDSD) for mining rich regularizations and relationships hidden in the data. Experimental results show that compared with 5 state-of-the-art self-distillation variants, the proposed DDSD obtains 13/4/13 in terms of “win”/“tie”/“lose” and gains the lowest-AVG_rank score. In particular, compared with pure ResMulti-Trans, DKN results in 20/1/9 regarding win/tie/lose. Last but not least, DKN overweighs 18 existing MTSC algorithms on 10 UEA2018 datasets and achieves the lowest-AVG_rank score.

العلاقة: https://nottingham-repository.worktribe.com/output/30106019Test; IEEE Transactions on Systems, Man, and Cybernetics: Systems; Volume 54; Issue 4; Pagination 2192-2204; https://nottingham-repository.worktribe.com/file/30106019/1/SMCA23Test

الإتاحة: https://doi.org/10.1109/tsmc.2023.3342640Test
https://nottingham-repository.worktribe.com/file/30106019/1/SMCA23Test
https://nottingham-repository.worktribe.com/output/30106019Test

المؤلفون: Xiao, Zhiwen, Xu, Xin, Xing, Huanlai, Zhao, Bowen, Wang, Xinhan, Song, Fuhong, Qu, Rong, Feng, Li

مصطلحات موضوعية: Feature extraction, Classification algorithms, Time series analysis, Data mining, Transformers, Routing, Knowledge transfer, Capsule Network, Deep Learning, Knowledge Distillation, Multivariate Time Series Classification, Mutual L

الوصف: This paper proposes a dual-network-based feature extractor, perceptive capsule network (PCapN), for multivariate time series classification (MTSC), including a local feature network (LFN) and a global relation network (GRN). The LFN has two heads (i.e., Head_A and Head_B), each containing two squash CNN blocks and one dynamic routing block to extract the local features from the data and mine the connections among them. The GRN consists of two capsule-based transformer blocks and one dynamic routing block to capture the global patterns of each variable and correlate the useful information of multiple variables. Unfortunately, it is difficult to directly deploy PCapN on mobile devices due to its strict requirement for computing resources. So, this paper designs a lightweight capsule network (LCapN) to mimic the cumbersome PCapN. To promote knowledge transfer from PCapN to LCapN, this paper proposes a deep transformer capsule mutual (DTCM) distillation method. It is targeted and offline, using one- and two-way operations to supervise the knowledge distillation process for the dual-network-based student and teacher models. Experimental results show that the proposed PCapN and DTCM achieve excellent performance on UEA2018 datasets regarding top-1 accuracy.

العلاقة: https://nottingham-repository.worktribe.com/output/32450554Test; IEEE Transactions on Cognitive and Developmental Systems; https://nottingham-repository.worktribe.com/file/32450554/1/TCDS24-HuanlaiTest

الإتاحة: https://doi.org/10.1109/tcds.2024.3370219Test
https://nottingham-repository.worktribe.com/file/32450554/1/TCDS24-HuanlaiTest
https://nottingham-repository.worktribe.com/output/32450554Test

المؤلفون: Xiao, Zhiwen, Xing, Huanlai, Qu, Rong, Li, Hui, Feng, Li, Zhao, Bowen, Yang, Jiayi

مصطلحات موضوعية: Feature extraction, Semantics, Time series analysis, Data mining, Brain modeling, Deep learning, Heuristic algorithms, Convolutional Neural Network, Knowledge Distillation, Time Series Classification

الوصف: Over the years, many deep learning algorithms have been developed for time series classification (TSC). A learning model’s performance usually depends on the quality of the semantic information extracted from lower and higher levels within the representation hierarchy. Efficiently promoting mutual learning between higher and lower levels is vital to enhance the model’s performance during model learning. To this end, we propose a self-bidirectional decoupled distillation (Self-BiDecKD) method for TSC. Unlike most self-distillation algorithms that usually transfer the target-class knowledge from higher to lower levels, Self-BiDecKD encourages the output of the output layer and the output of each lower-level block to form a bidirectional decoupled knowledge distillation (KD) pair. The bidirectional decoupled KD promotes mutual learning between lower- and higher-level semantic information and extracts the knowledge hidden in the target and non-target classes, helping Self-BiDecKD capture rich representations from the data. Experimental results show that compared with a number of self-distillation algorithms, Self-BiDecKD wins 35 out of 85 UCR2018 datasets and achieves the smallest AVG_rank score, namely 3.2882. In particular, compared with a non-self-distillation Baseline, Self-BiDecKD results in 58/8/19 regarding ‘win’/‘tie’/‘lose’.

العلاقة: https://nottingham-repository.worktribe.com/output/31435533Test; IEEE Transactions on Artificial Intelligence; https://nottingham-repository.worktribe.com/file/31435533/1/TAI2024Test

الإتاحة: https://doi.org/10.1109/tai.2024.3360180Test
https://nottingham-repository.worktribe.com/file/31435533/1/TAI2024Test
https://nottingham-repository.worktribe.com/output/31435533Test

المؤلفون: Han, Yuan, Wei, Xiaomin, Chen, Guojun, Shao, Enge, Zhou, Yilin, Li, Yuqing, Xiao, Zhiwen, Shi, Xiaoran, Zheng, Hao, Huang, Senlin, Chen, Yanmei, Wang, Yanbing, Zhang, Yeshen, Liao, Yulin, Liao, Wangjun, Bin, Jianping, Wang, Yuegang, Li, Xinzhong

المصدر: International Journal of Biological Sciences ; volume 20, issue 6, page 2072-2091 ; ISSN 1449-2288

مصطلحات موضوعية: Cell Biology, Developmental Biology, Molecular Biology, Applied Microbiology and Biotechnology, Ecology, Evolution, Behavior and Systematics

الإتاحة: https://doi.org/10.7150/ijbs.89883Test
https://www.ijbs.com/v20p2072.htmTest

المؤلفون: Shang, Jing, Xiao, Zhiwen, Tao, Tao, Wang, Jibin, Wu, Zhihui

المساهمون: National Key Research and Development Program of China

المصدر: Journal of King Saud University - Computer and Information Sciences ; volume 36, issue 2, page 101962 ; ISSN 1319-1578

مصطلحات موضوعية: General Computer Science

الإتاحة: https://doi.org/10.1016/j.jksuci.2024.101962Test
https://api.elsevier.com/content/article/PII:S131915782400051X?httpAccept=text/xmlTest
https://api.elsevier.com/content/article/PII:S131915782400051X?httpAccept=text/plainTest

المؤلفون: Ning, Yixi¹ (AUTHOR) ningy@uhv.edu, Yang, Sean² (AUTHOR), Xiao, Zhiwen³ (AUTHOR)

المصدر: Journal of Economics & Finance. Mar2024, Vol. 48 Issue 1, p238-261. 24p.

مصطلحات موضوعية: Rate of return on stocks, Abnormal returns, Information asymmetry, Organizational performance, Business enterprises, Sentiment analysis

مستخلص: This study examines the evolving information environment in a sample of post-IPO firms from 2010 to 2020 by exploring the relationship between firm-level manager sentiments and excess buy-and-hold stock returns. We confirm that there is a negative relationship between the full-text manager sentiment and the long-term excess buy-and-hold returns from 1 to 12 months after corporate filings. However, there is no positive association between manager sentiment and contemporaneous (e.g., 4-day) event-window announcement returns as documented in prior studies. The mispricing effect captured by the firm-level manager sentiment is less severe in the later years post IPOs, indicating the degree of information asymmetry is a dynamic phenomenon as the firms become more established over time. We also find the manager sentiment from the section of Management Discussion and Analysis (MD&A) is less optimistic about the firm performance than the full-text manager sentiment, and there is no mispricing effect of the MD&A manager sentiment on stock returns. These findings shed valuable insights on the dynamic information environment in the post-IPO firms from a unique perspective. [ABSTRACT FROM AUTHOR]

المؤلفون: Xiao, Zhiwen, Xu, Xin, Xing, Huanlai, Luo, Shouxi, Dai, Penglin, Zhan, Dawei

مصطلحات موضوعية: Computer Science - Machine Learning

الوصف: Time series data usually contains local and global patterns. Most of the existing feature networks pay more attention to local features rather than the relationships among them. The latter is, however, also important yet more difficult to explore. To obtain sufficient representations by a feature network is still challenging. To this end, we propose a novel robust temporal feature network (RTFN) for feature extraction in time series classification, containing a temporal feature network (TFN) and an LSTM-based attention network (LSTMaN). TFN is a residual structure with multiple convolutional layers. It functions as a local-feature extraction network to mine sufficient local features from data. LSTMaN is composed of two identical layers, where attention and long short-term memory (LSTM) networks are hybridized. This network acts as a relation extraction network to discover the intrinsic relationships among the extracted features at different positions in sequential data. In experiments, we embed RTFN into a supervised structure as a feature extractor and into an unsupervised structure as an encoder, respectively. The results show that the RTFN-based structures achieve excellent supervised and unsupervised performance on a large number of UCR2018 and UEA2018 datasets.
Comment: 41pages, 7figures, Revised Paper

الوصول الحر: http://arxiv.org/abs/2011.11829Test