المؤلفون: Zhu, Yutao, Zhou, Kun, Mao, Kelong, Chen, Wentong, Sun, Yiding, Chen, Zhipeng, Cao, Qian, Wu, Yihan, Chen, Yushuo, Wang, Feng, Zhang, Lei, Li, Junyi, Wang, Xiaolei, Wang, Lei, Zhang, Beichen, Dong, Zican, Cheng, Xiaoxue, Chen, Yuhan, Tang, Xinyu, Hou, Yupeng, Ren, Qiangqiang, Pang, Xincheng, Xie, Shufang, Zhao, Wayne Xin, Dou, Zhicheng, Mao, Jiaxin, Lin, Yankai, Song, Ruihua, Xu, Jun, Chen, Xu, Yan, Rui, Wei, Zhewei, Hu, Di, Huang, Wenbing, Gao, Ze-Feng, Chen, Yueguo, Lu, Weizheng, Wen, Ji-Rong

مصطلحات موضوعية: Computer Science - Computation and Language, Computer Science - Artificial Intelligence

الوصف: Large language models (LLMs) have become the foundation of many applications, leveraging their extensive capabilities in processing and understanding natural language. While many open-source LLMs have been released with technical reports, the lack of training details hinders further research and development. This paper presents the development of YuLan, a series of open-source LLMs with $12$ billion parameters. The base model of YuLan is pre-trained on approximately $1.7$T tokens derived from a diverse corpus, including massive English, Chinese, and multilingual texts. We design a three-stage pre-training method to enhance YuLan's overall capabilities. Subsequent phases of training incorporate instruction-tuning and human alignment, employing a substantial volume of high-quality synthesized data. To facilitate the learning of complex and long-tail knowledge, we devise a curriculum-learning framework throughout across these stages, which helps LLMs learn knowledge in an easy-to-hard manner. YuLan's training is finished on Jan, 2024 and has achieved performance on par with state-of-the-art LLMs across various English and Chinese benchmarks. This paper outlines a comprehensive technical roadmap for developing LLMs from scratch. Our model and codes are available at https://github.com/RUC-GSAI/YuLan-ChatTest.

الوصول الحر: http://arxiv.org/abs/2406.19853Test

المؤلفون: Lu, Weizheng, Zhang, Jiaming, Zhang, Jing, Chen, Yueguo

مصطلحات موضوعية: Computer Science - Artificial Intelligence, Computer Science - Computation and Language

الوصف: Tables, typically two-dimensional and structured to store large amounts of data, are essential in daily activities like database queries, spreadsheet calculations, and generating reports from web tables. Automating these table-centric tasks with Large Language Models (LLMs) offers significant public benefits, garnering interest from academia and industry. This survey provides an extensive overview of table tasks, encompassing not only the traditional areas like table question answering (Table QA) and fact verification, but also newly emphasized aspects such as table manipulation and advanced table data analysis. Additionally, it goes beyond the early strategies of pre-training and fine-tuning small language models, to include recent paradigms in LLM usage. The focus here is particularly on instruction-tuning, prompting, and agent-based approaches within the realm of LLMs. Finally, we highlight several challenges, ranging from private deployment and efficient inference to the development of extensive benchmarks for table manipulation and advanced data analysis.

الوصول الحر: http://arxiv.org/abs/2402.05121Test

المؤلفون: Lu, Weizheng, He, Kaisheng, Qin, Xuye, Li, Chengjie, Wang, Zhong, Yuan, Tao, Liao, Xia, Zhang, Feng, Chen, Yueguo, Du, Xiaoyong

مصطلحات موضوعية: Computer Science - Distributed, Parallel, and Cluster Computing

الوصف: Data science pipelines commonly utilize dataframe and array operations for tasks such as data preprocessing, analysis, and machine learning. The most popular tools for these tasks are pandas and NumPy. However, these tools are limited to executing on a single node, making them unsuitable for processing large-scale data. Several systems have attempted to distribute data science applications to clusters while maintaining interfaces similar to single-node libraries, enabling data scientists to scale their workloads without significant effort. However, existing systems often struggle with processing large datasets due to Out-of-Memory (OOM) problems caused by poor data partitioning. To overcome these challenges, we develop Xorbits, a high-performance, scalable data science framework specifically designed to distribute data science workloads across clusters while retaining familiar APIs. The key differentiator of Xorbits is its ability to dynamically switch between graph construction and graph execution. Xorbits has been successfully deployed in production environments with up to 5k CPU cores. Its applications span various domains, including user behavior analysis and recommendation systems in the e-commerce sector, as well as credit assessment and risk management in the finance industry. Users can easily scale their data science workloads by simply changing the import line of their pandas and NumPy code. Our experiments demonstrate that Xorbits can effectively process very large datasets without encountering OOM or data-skewing problems. Over the fastest state-of-the-art solutions, Xorbits achieves an impressive 2.66* speedup on average. In terms of API coverage, Xorbits attains a compatibility rate of 96.7%, surpassing the fastest framework by an impressive margin of 60 percentage points. Xorbits is available at https://github.com/xorbitsai/xorbitsTest.
Comment: ICDE 2024 Industrial and Application Track

الوصول الحر: http://arxiv.org/abs/2401.00865Test

المؤلفون: Lu, Weizheng¹ (AUTHOR), Chen, Yang¹ (AUTHOR) yangchen0507@hnu.edu.cn, Li, Jia¹ (AUTHOR) lijia123@hnu.edu.cn, Liaw, Peter K.² (AUTHOR), Fang, Qihong² (AUTHOR)

المصدر: Acta Mechanica Sinica. Jun2024, Vol. 40 Issue 6, p1-9. 9p.

مستخلص: High/medium entropy alloys (H/MEAs) are generally possible to exhibit chemical short-range order (SRO). However, the complex role of SRO on mechanical properties from nano-scale to meso-scale is still challenging so far. Here, we study the strengthening mechanism and deformation behavior in a model body-centered-cubic HfNbTa MEA by using atomic-scale molecular dynamics, micro-scale dislocation dynamics, and meso-scale crystal plasticity finite element. The SRO inhibits dislocation nucleation at the atomic scale, improving the flow stress. The SRO-induced ultrastrong local stress fluctuation greatly improves the micro-scale dislocation-based strength by the significant dislocation forest strengthening. Moreover, the Ta-rich locally ordered structure leads to an obvious heterogeneous strain and stress partitioning, which forms a strong strain gradient in the adjacent grain interiors and contributes to the strong back-stress-induced strain hardening. [ABSTRACT FROM AUTHOR]

المؤلفون: Lu, Weizheng, Chen, Yang, Zhang, Wei, Tan, Fusheng, Li, Jia, Liu, Bin, Liaw, Peter K, Fang, Qihong

المساهمون: National Natural Science Foundation of China, Hunan Provincial Innovation Foundation for Postgraduate, National Science Foundation, China Postdoctoral Science Foundation, Natural Science Foundation of Hunan Province

المصدر: International Journal of Mechanical Sciences ; volume 271, page 109140 ; ISSN 0020-7403

مصطلحات موضوعية: Mechanical Engineering, Mechanics of Materials, Condensed Matter Physics, General Materials Science, Civil and Structural Engineering, Ocean Engineering, Aerospace Engineering, Applied Mathematics

الإتاحة: https://doi.org/10.1016/j.ijmecsci.2024.109140Test
https://api.elsevier.com/content/article/PII:S0020740324001838?httpAccept=text/xmlTest
https://api.elsevier.com/content/article/PII:S0020740324001838?httpAccept=text/plainTest

المؤلفون: Li, Jia, Chen, Yang, Xie, Baobin, Lu, Weizheng, Fang, Qihong

المصدر: International Conference on Computational & Experimental Engineering and Sciences ; volume 25, issue 3, page 1-2 ; ISSN 1933-2815

الإتاحة: https://doi.org/10.32604/icces.2023.010064Test
https://www.techscience.com/icces/v25n3/53846/pdfTest

المؤلفون: Fang, Qihong, Lu, Weizheng, Chen, Yang, Feng, Hui, Liaw, Peter K, Li, Jia

المصدر: Journal of the Mechanics and Physics of Solids ; volume 169, page 105067 ; ISSN 0022-5096

مصطلحات موضوعية: Mechanical Engineering, Mechanics of Materials, Condensed Matter Physics

الإتاحة: https://doi.org/10.1016/j.jmps.2022.105067Test
https://api.elsevier.com/content/article/PII:S0022509622002447?httpAccept=text/xmlTest
https://api.elsevier.com/content/article/PII:S0022509622002447?httpAccept=text/plainTest

المؤلفون: Tian, Qiong, Liu, Qiuhong, Song, Kedong, Mei, Yufan, Lu, Weizheng, Peng, Jinfeng, Zhou, Ji, Ding, Yanhuai

المصدر: Fibers and Polymers ; volume 22, issue 1, page 30-35 ; ISSN 1229-9197 1875-0052

مصطلحات موضوعية: Polymers and Plastics, General Chemical Engineering, General Chemistry

الإتاحة: https://doi.org/10.1007/s12221-021-0103-1Test

المؤلفون: Liu, Qiuhong, Jiang, Wenwu, Lu, Weizheng, Mei, Yufan, He, Fakang, Zhang, Minhui, Liu, Yin, Chen, Yunqing, Peng, Jinfeng, Ding, Yanhuai

المساهمون: Natural Science Foundation of Hunan Province, Graduate Innovation Foundation of Hunan Province, National Natural Science Foundation of China, Scientific Research Foundation of Hunan Provincial Education Department, Postgraduate Education Reform Project of Hunan Province

المصدر: Nanotechnology ; volume 31, issue 43, page 435701 ; ISSN 0957-4484 1361-6528

الإتاحة: https://doi.org/10.1088/1361-6528/aba303Test

المؤلفون: Lu, Weizheng, Chen, Yang, Zhang, Wei, Tan, Fusheng, Li, Jia, Liu, Bin, Fang, Qihong, Liaw, Peter K.

الإتاحة: https://doi.org/10.2139/ssrn.4620611Test