Abstract:
Gait generation for soft robots is challenging due to the nonlinear dynamics and high dimensional input spaces of soft actuators. Limitations in soft robotic control and perception force researchers to hand-craft open loop controllers for gait sequences, which is a non-trivial process. Moreover, short soft actuator lifespans and natural variations in actuator behavior limit machine learning techniques to settings that can be learned on the same time scales as robot deployment. Lastly, simulation is not always possible, due to heterogeneity and nonlinearity in soft robotic materials and their dynamics change due to wear. We present a sample-efficient, simulation free, method for self-generating soft robot gaits, using very minimal computation. This technique is demonstrated on a motorized soft robotic quadruped that walks using four legs constructed from 16 "handed shearing auxetic" (HSA) actuators. To manage the dimension of the search space, gaits are composed of two sequential sets of leg motions selected from 7 possible primitives. Pairs of primitives are executed on one leg at a time; we then select the best-performing pair to execute while moving on to subsequent legs. This method -- which uses no simulation, sophisticated computation, or user input -- consistently generates good translation and rotation gaits in as low as 4 minutes of hardware experimentation, outperforming hand-crafted gaits. This is the first demonstration of completely autonomous gait generation in a soft robot.
Fig. 1. The “tortoisebot” hardware platform used for this work. Each of the four legs is composed of four Handed Shearing Auxetic (HSA) actuators which allow them to extend, bend, twist, and contract based on the rotation of four servos on the main chassis plate. (Why yes, we did have several interchangeable heads including a snail and at least one dinosaur.)
Discussion:
Machine learning has been used extensively in soft robotics. Our goal with this project was to understand how to use machine learning for soft systems without relying on extensive simulation. This is an important problem, because the same properties (non-linear dynamics, time and temperature varying behavior, complex compliant contacts) that make soft robots such an attractive application for machine learning, also make soft robots a real challenge to simulate accurately. For many systems, that is true as-built, but certainly it is true for almost all soft robots after a dozen operating hours and a few minor breakages.
In my view the real success of this project is that it does what it says on the tin: Our method can generate usable gaits for the soft-quadruped in less than 4 minutes without using any external simulation of any kind. It’s also reliable, we ran well over 100 trials and in-person demos without a single algorithmic failure, though we did have a few cases where legs became so badly degraded that the robot was simply no longer capable of moving. The paper goes into more detail about the gait-search method we used, but it’s a primitives-based combinatorial search, which could be applied to a wide range of soft systems with very little modification.
We have a few fun results in the paper, including showing that our learned gaits can be combined to produce accurate tracing behavior, but my personal favorite is definitely figure 2 below. That shows a “race” between the robot running the SOTA hand-tuned forward gait for the soft-quadruped, and a fresh forward gait found by our method, which is over 2.5 times faster. What I love about that result is how little tuning it required, we just trained a new gait and off it went. We had another fun moment much earlier in the research process where the gait-search method — on its very first run —found a working rotation gait: something that had evaded several hours of hand tuning and experimentation.
I think the two big weaknesses in the work are the lack of dynamism from the soft robot, and the relatively narrow behavior range of the gait search. While the robot really is soft, and it really does have non-linear dynamics, its movements are also pretty subtle. That helps provide stability “for free” since the robot can be trivially tuned to avoid ever risking a fall. That is very helpful, because the gait search method — while very fast — lacks some of the expressive power of more general models like a neural network. As a result the robot, while it can learn to walk in 4 minutes, will never learn state-dependent behaviors like fall recovery. In future work, I hope to apply some of these same ideas to more dynamic robots, and to a wider range of robotic tasks including mobility, recovery, manipulation, and adaptation.
Fig. 2. A “race” showing the comparison between (A) the fastest hand-tuned gait for the platform and (B) a freshly trained gait from our hardware learning. The steering bar we used to make this figure nice and linear limits the learned gait to around 2x the speed of the hand tuned gait, but unencumbered we found a speed difference of more than 2.5x.
What is an HSA:
Vid. 1. Note that the HSAs shown in this video are TPU actuators, which have a much larger range of motion than the photopolymer HSAs used in our Automated Gait Generation work.
A Handed Shearing Auxetic (HSA) actuator is a cylindrical soft-mechanism that acts as a compliant rotary-to-linear converter. When one side of the cylinder is rotated (with the other fixed) the structure will either extend or contract depending on the direction of rotation. HSAs are used as the vertical struts on the the platform (see figure 1). The HSAs used in this project were designed and made by our collaborators in Ryan Truby’s Lab at Northwestern.
The included video shows this behavior in action on a different (more recent) platform in the lab.