التفاصيل البيبلوغرافية
| العنوان: |
Two reproductions of a human-assessed comparative evaluation of a semantic error detection system |
| المؤلفون: |
Huidrom, Rudali, Dušek, Ondřej, Kasner, Zdeněk, Castro Ferrera, Thiago, Belz, Anya |
| المساهمون: |
Shaikh, Samira, Ferreira, Thiago, Stent, Amanda |
| بيانات النشر: |
Association for Computational Linguistics (ACL) |
| سنة النشر: |
2022 |
| المجموعة: |
Dublin City University: DCU Online Research Access Service (DORAS) |
| مصطلحات موضوعية: |
Computational linguistics |
| الوصف: |
In this paper, we present the results of two re- production studies for the human evaluation originally reported by Dušek and Kasner (2020) in which the authors comparatively evaluated outputs produced by a semantic error detection system for data-to-text generation against ref- erence outputs. In the first reproduction, the original evaluators repeat the evaluation, in a test of the repeatability of the original evaluation. In the second study, two new evaluators carry out the evaluation task, in a test of the reproducibility of the original evaluation under otherwise identical conditions. We describe our approach to reproduction, and present and analyse results, finding different degrees of re- producibility depending on result type, data and labelling task. Our resources are available and open-sourced. |
| نوع الوثيقة: |
conference object |
| وصف الملف: |
application/pdf |
| اللغة: |
English |
| العلاقة: |
https://doras.dcu.ie/28650Test/; https://aclanthology.org/2022.inlg-genchal.9Test; https://doras.dcu.ie/28650/1/2022.inlg-genchal.9.pdfTest; Huidrom, Rudali orcid logoorcid:0000-0003-0630-3603 , Dušek, Ondřej orcid logoorcid:0000-0002-1415-1702 , Kasner, Zdeněk orcid logoorcid:0000-0002-5753-5538 , Castro Ferrera, Thiago orcid logoorcid:0000-0003-0200-3646 and Belz, Anya orcid logoorcid:0000-0002-0552-8096 (2022) Two reproductions of a human-assessed comparative evaluation of a semantic error detection system. In: International Natural Language Generation Conference, 18-22 July 2022, Waterville, Maine, USA + Online. |
| الإتاحة: |
https://doras.dcu.ie/28650/1/2022.inlg-genchal.9.pdfTest |
| حقوق: |
cc_by_4 |
| رقم الانضمام: |
edsbas.941A02D8 |
| قاعدة البيانات: |
BASE |
