دورية أكاديمية
The Effectiveness of Machine Score-Ability Ratings in Predicting Automated Scoring Performance
| العنوان: | The Effectiveness of Machine Score-Ability Ratings in Predicting Automated Scoring Performance |
|---|---|
| اللغة: | English |
| المؤلفون: | Lottridge, Susan, Wood, Scott, Shaw, Dan |
| المصدر: | Applied Measurement in Education. 2018 31(3):215-232. |
| الإتاحة: | Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journalsTest |
| تمت مراجعته من قبل الزملاء: | Y |
| Page Count: | 18 |
| تاريخ النشر: | 2018 |
| نوع الوثيقة: | Journal Articles Reports - Research |
| Education Level: | Secondary Education |
| الواصفات: | Program Effectiveness, Computer Assisted Testing, Test Scoring Machines, Scoring, Automation, Rating Scales, Prediction, Scores, Correlation, Evaluation Criteria, Reading Tests, Science Tests, Mathematics Tests, Summative Evaluation, Secondary School Students, Outcome Measures |
| DOI: | 10.1080/08957347.2018.1464452 |
| تدمد: | 0895-7347 |
| مستخلص: | This study sought to provide a framework for evaluating machine score-ability of items using a new score-ability rating scale, and to determine the extent to which ratings were predictive of observed automated scoring performance. The study listed and described a set of factors that are thought to influence machine score-ability; these factors informed the score-ability rating applied by expert raters. Five Reading items, six Science items, and 10 Math items were examined. Experts in automated scoring served as reviewers, providing independent ratings of score-ability before engine calibration. Following the rating, engines were calibrated and their performances were evaluated using common industry criteria. Three derived criteria from the engine evaluations were computed: the score-ability value in the rating scale based on the empirical results, the number of industry evaluation criteria met by the engine, the approval status of the engine based on the number of criteria met. The results indicated that the score-ability ratings were moderately correlated with Science score-ability, the ratings were weakly correlated with Math score-ability, and were not correlated with Reading score-ability. |
| Abstractor: | As Provided |
| Number of References: | 12 |
| Entry Date: | 2018 |
| رقم الانضمام: | EJ1179718 |
| قاعدة البيانات: | ERIC |
| تدمد: | 0895-7347 |
|---|---|
| DOI: | 10.1080/08957347.2018.1464452 |