التفاصيل البيبلوغرافية
| العنوان: |
Pitfalls in Research on Ecological Validity of Novel Executive Function Tests: A Systematic Review and a Call to Action. |
| المؤلفون: |
Suchy, Yana1 yana.suchy@psych.utah.edu, Mora, Michelle Gereau1, DesRuisseaux, Libby A.1, Niermeyer, Madison A.2, Brothers, Stacey Lipio1 |
| المصدر: |
Psychological Assessment. Apr2024, Vol. 36 Issue 4, p243-261. 19p. |
| مصطلحات موضوعية: |
*MEDICAL quality control, *EXECUTIVE function, *FUNCTIONAL assessment, *SYSTEMATIC reviews, *MEDLINE, *NEUROPSYCHOLOGICAL tests, *MEDICAL research, *PSYCHOLOGY information storage & retrieval systems, RESEARCH evaluation |
| مستخلص: |
The term "ecological validity" (EV) has traditionally referred to test scores' ability to predict real-world functioning. However, a test's similarity to real-world tasks is sometimes mistaken for evidence of its ability to predict daily life, sometimes bypassing rigorous validation research. The goal of this systematic review was to examine the type and quality of evidence used to support claims of EV of novel face-valid tests of executive functions (EF). MEDLINE and PsychINFO databases were searched using the following terms: ecologic* AND neuropsychol* AND (executive function* OR executive dysfunction OR executive abilit*). Thirty-two articles that explicitly stated that their results demonstrated EV of a novel face-valid test of EF were identified. Results showed that only 60% of studies based their claims about EV on test scores' ability to predict functional outcomes, with the remaining 40% relying on other evidence (e.g., correlations with other measures, participant feedback, group differences). Among the studies that did base their conclusions on test scores' ability to predict outcomes (n = 19), an overwhelming majority relied on behavioral rating scales, utilized small sample sizes and participant-to-variable ratios, and failed to control for covariates and multiple comparisons. Poor scientific rigor was particularly pronounced in studies of "naturalistic" tests. The present systematic review reveals significant conceptual, methodological, and statistical flaws among an overwhelming majority of studies that claim to have found support for the EV of a novel face-valid test of EF. We call upon authors, reviewers, and editors to safeguard the scientific rigor of research in this area. Public Significance Statement: This literature review reveals considerable inconsistency in the quality of evidence among neuropsychological studies that claim to have demonstrated "ecological validity" of tests designed to resemble real-world tasks and intended for use as predictors of patients' daily functioning. We call upon neuropsychologists to use clearer terminology and to avoid common pitfalls in this type of test development research to improve our field's ability to interpret results of neuropsychological evaluations. [ABSTRACT FROM AUTHOR] |
| قاعدة البيانات: |
Academic Search Index |
