Coevolution of Morphology and Policy Implicit Neural Functions

Figure 1. Directly evolving weights of deep neural networks as genotypes unlocks a large design space, leading to a diverse set of fast running soft robots.

Simulataneously searching valid connected body shapes and muscle actuations which effectively leverage morphological structures can only be done efficiently for highly constrained design spaces. In this work, I explore the co-evolution of morphology and policy in the large design space of all possible functions by using deep neural networks as genotypes, with an application to fast running soft robots. By exploiting neural networks' low frequency bias, the 3D shapes of the robots generated from level sets of their morphology network's outputs lead to highly connected geometries with smoothly changing protruding structures. Additionally, by letting the policy network parameterize robots' sinosoidal muscle's actuation, I'm able to express a prior over the frequencies, amplitudes, and phases of muscle activity to guide the algorithm's exploration. Finally, I demonstrate that a higher resolution sample of the robot's morphology and policy can be performed after evolution, demonstrating the power of using implicit functions as genotypes. Robots evolved with these neural network genoptypes developed complex hopping gaits, achieving the fastest running speed of 1.10 m/s.

Hopping Gaits

Figure 2. All robots evolved its own complex variant of a hopping gait customized for the geometry of its "feet" while discovering a front-heavy "head" optimized for speed. Here, robots are shown at 0.40x real time speed.

Resampling at high resolutions

Figure 3. Representing morphologies and policies implicitly allows faster evolution at lower resolutions, while retaining the ability to resample at higher resolutions for more nuanced geometry and gait details. Here, the same robots is evolved in an 8x8 grid (left), but resampled to 14x14 (center) and 50x50 (right) grids.

Local exploration with weight mutations

Figure 4. Even with small perturbations to the neural network weights encoding the robots' morphologies and policies, the resulting slight change in its gait could be the difference between falling flat 5 steps in and winning the race.

Robot Parade


Code for the evolutionary algorithm and the soft robot simulator: Github


If you have any questions, please feel free to contact me.

Huy Ha
Columbia University