| الملخص: |
Our ability to test and falsify theories rests on valid and reliable measurement. Minimizing noise, or measurement error, is a hurdle that researchers must overcome. Researchers use state-of-the-art tools to investigate the cognitive underpinnings of human communicative behavior and measurement error is likely to differ by instrument. It is important to determine if a given instrument is sufficiently accurate to extract signal from noise. Reaction time is commonly used to measure cognitive processes during media use. A reaction time measurement tool has been recently implemented in Asteroid Impact, an open source, naturalistic, experimental video game stimulus. We tested if reaction time latency differed across computer operating system (macOS, Windows 10, Ubuntu 18.04), software (Python 2, Python 3), and trial modality (audio, visual) in Asteroid Impact.We used an Arduino microcontroller to generate a ground truth value of average response time latency. Specifically, we measured the latency between the onset of visual and auditory trials and a response from the Arduino. The Arduino was programmed to look for changes in luminance (visual trial) or sound amplitude (audio trial) every millisecond. When a change was detected, the Arduino was programmed to issue a keypress response. Any observed latency between trial onset and the microcontroller’s response reflects measurement error in the computer hardware and software. Ideally, this latency will have a small mean and variance, which would indicate that Asteroid Impact is accurate in measuring reaction times. All code to fully reproduce and replicate this study is available on GitHub (https://anonymous.4open.science/r/8b618f1e-6cf4-4bdd-9e3b-4058a37bfb53Test/).ANOVA results showed a significant three-way interaction F (2, 1188) = 199.95, p < .001, η2 = .25. Auditory trials using Python 3 on macOS had the lowest latency (M = 44.6, SD = 8.4). Auditory trials using Python 2 on Ubuntu 18.04 had the longest latency (M = 121, SD = 18.4). Raincloud plots of the main and two-way interaction (all p’s < .001) effects are included to further characterize the distribution of results.Which configuration of Asteroid Impact is most accurate when measuring reaction times? It depends. Auditory trials are most accurate on macOS (regardless of Python version). However, visual trials are most accurate using Python 2 on Ubuntu 18.04. Practically speaking, Windows 10 may be the best solution as it has acceptable accuracy for both visual and auditory trials (regardless of Python version). We are encouraged to see that reaction time measurements collected using Asteroid Impact are comparable to other response-latency measuring software. Moreover, these values are well below the speed at which the human visual system can detect and respond to complex visual perception tasks. Together, these results demonstrate that Asteroid Impact is a valid and reliable stimulus for measuring reaction time data. |
