Silvestro Micera, a neuroengineer, creates advanced technological solutions to assist people in regaining sensory and motor functions that have been lost due to traumatic events or neurological disorders. He had never previously worked on improving the human body and cognition using technology.
Micera and his colleagues report in Science Robotics on how diaphragm movement can be monitored for successful control of an extra arm, essentially augmenting a healthy individual with a third – robotic – arm.
“This study opens up new and exciting opportunities, showing that extra arms can be extensively controlled and that simultaneous control with both natural arms is possible,” says Micera, Bertarelli Foundation Chair in Translational Neuroengineering at EPFL and professor of Bioelectronics at Scuola Superiore Sant’Anna.
The research is part of the Third-Arm project, which was previously funded by the Swiss National Science Foundation (NCCR Robotics) and aims to provide a wearable robotic arm to aid in daily tasks or in search and rescue. Micera believes that investigating the cognitive limitations of third-arm control could lead to a better understanding of the human brain.
We want to understand if our brains are hardwired to control what nature has given us, and we’ve shown that the human brain can adapt to coordinate new limbs in tandem with our biological ones.
Solaiman Shokur
Micera continues, “The main motivation of this third arm control is to understand the nervous system. If you challenge the brain to do something that is completely new, you can learn if the brain has the capacity to do it and if it’s possible to facilitate this learning. We can then transfer this knowledge to develop, for example, assistive devices for people with disabilities, or rehabilitation protocols after stroke.”
“We want to understand if our brains are hardwired to control what nature has given us, and we’ve shown that the human brain can adapt to coordinate new limbs in tandem with our biological ones,” explains Solaiman Shokur, co-PI of the study and EPFL Senior Scientist at the Neuro-X Institute. “It’s about acquiring new motor functions, enhancement beyond the existing functions of a given user, be it a healthy individual or a disabled one. From a nervous system perspective, it’s a continuum between rehabilitation and augmentation.”
To explore the cognitive constraints of augmentation, the researchers first built a virtual environment to test a healthy user’s capacity to control a virtual arm using movement of his or her diaphragm. They found that diaphragm control does not interfere with actions like controlling one’s physiological arms, one’s speech or gaze.
In this virtual reality setup, the user is equipped with a belt that measures diaphragm movement. Wearing a virtual reality headset, the user sees three arms: the right arm and hand, the left arm and hand, and a third arm between the two with a symmetric, six-fingered hand.
“We made this hand symmetric to avoid any bias towards either the left or right hand,” says Giulia Dominijanni, a PhD student at EPFL’s Neuro-X Institute.
In the virtual environment, the user is then asked to reach out with either the left or right hand, or with the symmetric hand in the middle. In the real world, the user grips an exoskeleton with both arms, allowing control of the virtual left and right arms. The virtual middle, symmetric arm is controlled by movement detected by the belt around the diaphragm. Over 150 sessions, the setup was tested on 61 healthy subjects.
“Diaphragm control of the third arm is actually very intuitive, with participants learning to control the extra limb very quickly,” explains Dominijanni. “Moreover, our control strategy is inherently independent from the biological limbs and we show that diaphragm control does not impact a user’s ability to speak coherently.”
The researchers also tested diaphragm control with a real robotic arm, albeit a simplified one consisting of a rod that can be extended out and back in. The rod is extended out when the user contracts the diaphragm. In a VR-like experiment, the user is asked to reach for and hover over target circles with her left or right hand, or with the robotic rod.
In addition to the diaphragm, but not reported in the study, vestigial ear muscles were tested for their feasibility in performing new tasks. In this method, a user is outfitted with ear sensors and trained to control the movement of a computer mouse with fine ear muscle movement.
“Users could potentially use these ear muscles to control an extra limb,” says Shokur, emphasizing that these alternative control strategies could aid in the development of rehabilitation protocols for people with motor deficiencies one day.
Previous studies on the control of robotic arms, which were part of the third arm project, were aimed at assisting amputees. The most recent Science Robotics study goes beyond repairing the human body and toward augmentation.
“Our next step is to investigate the use of more complex robotic devices to perform real-life tasks both inside and outside of the laboratory, using our various control strategies.” Only then will we be able to see the true potential of this approach,” Micera concludes.
