PITTSBURGH—A team of researchers from Carnegie Mellon University, in collaboration with the University of Minnesota, has made a breakthrough in the field of noninvasive robotic device control. Using a noninvasive brain-computer interface (BCI), researchers have developed the first successful mind-controlled robotic arm, which exhibits the ability to continuously track and follow a computer cursor.
BCIs have been shown to achieve good performance for controlling robotic devices, using only signals picked up by brain implants. When robotic devices can be controlled with high precision, they can be used to complete a variety of daily tasks. Until now, BCIs successful in controlling robotic arms have used invasive brain implants. These implants require a substantial amount of medical and surgical expertise to correctly install and operate, and they involve potential risks to subjects as well as high costs. As such, their use has been limited to a few clinical cases.
A grand challenge in BCI research is developing less invasive or even completely noninvasive technology that would allow patients to control their environment or robotic limbs, using their own thoughts. Noninvasive BCI technology using only thoughts, if successful, will have broad applications — in particular benefiting the lives of paralyzed patients and those with movement disorders.
But BCIs that use noninvasive external sensing, rather than brain implants, receive “dirtier” signals — leading to current lower resolution and less precise control. When using only the brain to control a robotic arm, noninvasive BCI hasn’t compared to implanted devices. Despite this BCI researchers forged ahead, their eyes on the noninvasive prize.
Bin He, Trustee Professor and department head of Biomedical Engineering at Carnegie Mellon University, is working on achieving the goal of noninvasive BCI.
“There have been major advances in mind controlled robotic devices using brain implants. It’s excellent science,” said He. “But noninvasive is the ultimate goal. Advances in neural decoding and the practical utility of noninvasive robotic arm control will have major implications on the eventual development of noninvasive neurorobotics.”
Using novel sensing and machine learning techniques, He and his lab have been able to access signals deep within the brain, achieving a high resolution of control over a robotic arm. With noninvasive neuroimaging and a novel continuous pursuit paradigm, He is overcoming the noisy EEG signals. This leads to significantly improve EEG-based neural decoding and facilitating real-time continuous 2D robotic device control.
Using a noninvasive BCI to control a robotic arm that tracks a cursor on a computer screen, He has shown in human subjects that a robotic arm can now follow the cursor continuously. Whereas noninvasive robotic arms controlled by humans had previously followed a moving cursor in jerky, discrete motions — as though the robotic arm was trying to catch up to the brain’s commands — now the arm follows the cursor in a smooth, continuous path.
In a paper published in Science Robotics, He’s team established a new framework that addresses and improves upon the brain and computer components of BCI by increasing user engagement and training, as well as spatial resolution of noninvasive neural data through EEG source imaging. The paper, entitled “Noninvasive neuroimaging enhances continuous neural tracking for robotic device control,” shows that the team’s unique approach to solving this problem not only enhanced BCI learning by nearly 60% for traditional center-out tasks, it also enhanced continuous tracking of a computer cursor by over 500%.
“Users were able to smoothly transition between virtual cursor and robotic arm control with minimal changes in performance, indicating the potential ease of integrating such a noninvasive assistive tool into clinical applications for autonomous use in daily life,” says the paper.
“Invasive systems have already demonstrated a level of control similar to such a noninvasive hypothetical; however, although such invasive approaches may offer much-needed help to a restricted number of patients with severe physical dysfunctions, most impaired persons will likely not qualify for participation due to both medical and financial limitations,” the paper continues. “Additionally, previous work has suggested that accessing sufficiently large patient populations for concrete and statistically significant conclusions may be difficult to obtain. Therefore, there is a strong need to further develop noninvasive BCI technology so that it can benefit most patients and even the general population in the future.”
The technology has applications that could help a variety of people by offering safe, noninvasive mind control of devices that could allow interact with and control of their environments. The technology has been tested in 68 able-bodied human subjects in up to 10 sessions for each subject, including virtual device control and controlling of a robotic arm for continuous pursuit. The technology is directly applicable to patients, and the team plans to conduct clinical trials in the near future.
“The effective training paradigm and additional ESI-based performance improvement demonstrated here, as well as the integration of such targeted enhancements toward robotic arm control, offer increasing confidence that noninvasive BCIs may be able to expand to widespread clinical investigation. We observed that, for robotic arm control, generic head models, rather than those derived from user-specific magnetic resonance imaging (MRI), were sufficient for high-quality performance,” the paper concludes. “Therefore, in all, the work presented in this paper is necessary for current EEG-based BCI paradigms to achieve useful and effective noninvasive robotic device control, and its results are pertinent in directing both ongoing and future studies.”
“Despite technical challenges using noninvasive signals, we are fully committed to bringing this safe and economic technology to people who can benefit from it,” noted He. “This work represents an important step in noninvasive brain-computer interfaces, a technology which someday may become a pervasive assistive technology aiding everyone, like smartphones.”
