This paper presents a new online single-trial EEG-based brain-computer interface (BCI) for controlling hand holding and sequence of hand grasping and opening in an interactive virtual reality environment. The goal of this research is to develop an interaction technique that will allow the BCI to be effective in real-world scenarios for hand grasp control. One of the major challenges in the BCI research is the subject training. Currently, in most online BCI systems, the classifier was trained offline using the data obtained during the experiments without feedback, and used in the next sessions in which the subjects receive feedback. We investigated whether the subject could achieve satisfactory online performance without offline training while the subjects receive feedback from the beginning of the experiments during hand movement imagination. Another important issue in designing an online BCI system is the machine learning to classify the brain signal which is characterized by significant day-to-day and subject-to-subject variations and time-varying probability distributions. Due to these variabilities, we introduce the use of an adaptive probabilistic neural network (APNN) working in a time-varying environment for classification of EEG signals. The experimental evaluation on ten naïve subjects demonstrated that an average classification accuracy of 75.4% was obtained during the first experiment session (day) after about 3 min of online training without offline training, and 81.4% during the second session (day). The average rates during third and eighth sessions are 79.0% and 84.0%, respectively, using previously calculated classifier during the first sessions, without online training and without the need to calibrate. The results obtained from more than 5000 trials on ten subjects showed that the method could provide a robust performance over different experiment sessions and different subjects.
