Controlling an avatar by thought using fMRI

Cohen O., 1, 3 

Mendelsohn A., 2
Drai D., 1
Malach R., 2
Friedman D. 1
1 The Interdisciplinary Center Herzliya,
2 Weizmann Institute of Science
3 Bar Ilan University.

We are carrying out a number of studies whereby subjects control the movement of an avatar by thought alone, using real-time fMRI. At the first stage of our study five three subjects were able to control right- and left-hand movements of an avatar using motor imagery without difficulty and with high accuracy. Therefore, we are running a study involving three classes: right-hand imagery to turn right, left-hand imagery to turn left, and feet imagery to move forward.
In the first part of the experiment we instruct the subject to imagine either left-hand, right-hand, or walking movements upon a predefined cue, and manually define regions of interest (ROIs), using a GLM analysis, by contrasting hand-motor and primary leg-motor regions. In the second phase (baseline) we compute the mean and standard deviation for each ROI. Finally, the subjects are instructed to move the avatar, according to auditory cues, by using motor imagery. At this preliminary stage we use a simple classification scheme; at each TR the system calculates the z-score value of each ROI relative to the mean values obtained during the baseline, and chooses the highest z-score value among the ROIs. The classification result is fed into the virtual reality system to move the avatar accordingly.
Our preliminary results indicate that subjects can learn to control an avatar using motor imagery in better-than-chance levels with very little training. Future work will include support vector machine (SVM) classifiers, with methods for feature reduction, in real time. Eventually we aim at allowing subjects to perform simple tasks in a virtual environment.