دورية
Overview of the Transformer-based Models for NLP Tasks
| العنوان: | Overview of the Transformer-based Models for NLP Tasks |
|---|---|
| المؤلفون: | Gillioz, Anthony, Casas, Jacky, Mugellini, Elena, Abou Khaled, Omar |
| مصطلحات موضوعية: | natural language processing, neural net architecture, regression analysis, przetwarzanie języka naturalnego, architektura sieci neuronowej, analiza regresji |
| الوصف: | In 2017, Vaswani et al. proposed a new neural network architecture named Transformer. That modern architecture quickly revolutionized the natural language processing world. Models like GPT and BERT relying on this Transformer architecture have fully outperformed the previous state-of-the-art networks. It surpassed the earlier approaches by such a wide margin that all the recent cutting edge models seem to rely on these Transformer-based architectures. In this paper, we provide an overview and explanations of the latest models. We cover the auto-regressive models such as GPT, GPT-2 and XLNET, as well as the auto-encoder architecture such as BERT and a lot of post-BERT models like RoBERTa, ALBERT, ERNIE 1.0/2.0. 1. Track 1: Artificial Intelligence 2. Technical Session: 5th International Workshop on Language Technologies and Applications 3. Opracowanie rekordu ze środków MNiSW, umowa Nr 461252 w ramach programu "Społeczna odpowiedzialność nauki" - moduł: Popularyzacja nauki i promocja sportu (2021). |
| نوع الوثيقة: | Article |
| اللغة: | eng |
| الوصول الحر: | http://yadda.icm.edu.pl/baztech/element/bwmeta1.element.baztech-22d70702-e206-442d-9f11-6befc9884150Test |
| رقم الانضمام: | edsbzt.bwmeta1.element.baztech.22d70702.e206.442d.9f11.6befc9884150 |
| قاعدة البيانات: | BazTech |
| الوصف غير متاح. |