Fundamentally, robotics is about helping people. Robots help us manufacture things, help us build things, and help make our lives easier and more convenient. As robotic systems increase in sophistication and capability, they’re starting to help people more directly, in elder care, rehabilitation centers, and hospitals. In the near future, robotics will become even more tightly integrated with humanity, to the point where cybernetics will be able to restore function to people with disabilities. In particular, amputee military personnel are the focus of one such program.
In 2014, DARPA announced its Hand Proprioception and Touch Interfaces (HAPTIX) program, which “seeks to create a prosthetic hand system that moves and provides sensation like a natural hand.”
According to Doug Weber, DARPA program manager of HAPTIX: “We believe that HAPTIX will create a sensory experience so rich and vibrant that the user will want to wear his or her prosthesis full-time and accept it as a natural extension of the body. If we can achieve that, DARPA is even closer to fulfilling its commitment to help restore full and natural functionality to wounded service members.”
Three different teams are involved in the HAPTIX project, and its success will depend on a carefully optimized mix of hardware, user interfaces, and control algorithms. OSRF is proud to be providing a customized version of the Gazebo simulator to the HAPTIX teams, allowing them to run tests on their software without being constrained by hardware availability: essentially, a kind of virtual playground for software engineers.
“The goal of HAPTIX is for OSRF to provide a realistic prosthetic simulation environment for biomechanical engineers to develop controllers for advanced prosthetics with high degrees of freedom,” explains John Hsu, co-founder and Chief Scientist at OSRF. The advanced prosthesis that DARPA is using in the HAPTIX program is DEKA’s “Luke” robotic arm, a 14 DoF cybernetic total arm replacement system. However, the arm is currently controlled by simple user interfaces designed for testing, and part of what HAPTIX hopes to deliver are interfaces that utilize control signals from muscles and nerves, while simultaneously delivering sensory feedback.
After nearly ten years of work and $40 million from DARPA, DEKA’s robotic arm is an amazing piece of hardware, but that’s just the beginning. “The hardware, in my opinion, needs to come before the software,” says Hsu. “They can be designed at the same time, but the hardware has a longer iteration cycle. Once you develop a nice hardware platform that’s stable, then you give it to the software team, and they take off, working really fast on the software while in the meantime trying not to break the hardware.”
This illustrates two reasons why having a good simulation environment is important: first, it lets you start working on the software before the hardware is fully complete, and second, it to some extent insulates software development from the hardware itself, meaning that you can have lots of engineers developing software in parallel, even if you only have one piece of hardware that may be fragile, expensive, and quite often inoperable for one reason or another.
For OSRF, creating and supporting a version of Gazebo for the HAPTIX program involves many different areas. Besides the customized simulation environment, OSRF has also provided teams with an OptiTrack motion capture device, NVIDIA stereo glasses and a 3D monitor, a 3D joystick, and the documentation required to get it all working together flawlessly. This custom version of Gazebo also includes support for a variety of teleoperation hardware, and for the first time, users can interact programmatically with the simulation using both Windows and MATLAB. HAPTIX developers can leverage these 3D sensors and teleoperation systems to translate the motions of physical arms and hands into virtual environments, allowing them to run common hand function tests in the real world and in simulation at the same time. This also lays the foundation for a framework that could provide amputees a powerful and affordable way to learn how to use their new prosthesis.
Once the HAPTIX teams receive their DEKA arms, OSRF’s job becomes even more important, according to Hsu, because they’ll get a chance to see how well the simulation is actually working and then refine it to bring it as close to reality as possible. “I’m really looking forward to the validation part,” Hsu says. “I think that’s one of the big missing pieces for many simulation platforms: good validation data. When we were working on Gazebo for the DARPA Robotics Challenge, we never had an ATLAS robot. Getting the DEKA hand to do validation is huge.”
Validation is the process of making sure that commands sent to the simulated DEKA arm result in the same movements as identical commands sent to the real DEKA arm. “We send commands to the real hand and the simulated hand to see if they behave differently,” explains Hsu. “If they do, we update our model to make them match.” The closer the simulation matches, the more useful it will be to the HAPTIX teams. The end goal is, of course, to get everything working on the real hardware, but an accurate and detailed simulator is critical to the development of effective software.
The first generation of the DEKA arm recently arrived at OSRF for validation testing, and the complete hardware is expected before the end of the year. OSRF has been steadily releasing a series of stable versions of the HAPTIX simulator, and as the fidelity of simulated position holding, force control and response, and other dynamics are verified on the arm over the next few months, OSRF will continue upgrading the simulation software to make sure that the HAPTIX teams have all of the tools that they need to progress as quickly and efficiently as possible.
By early 2017, Phase 1 of HAPTIX will be complete, and the software and hardware components that prove to be the most successful will continue into Phase 2, the end goal of which is a complete, functional HAPTIX system. DARPA is hoping that take-home trials of such a system will happen by 2019, and that soon after, any amputee who needs one will be able to benefit from a prosthetic hand that acts (and feels) just like the real thing.