I recently joined ICLR 2022 to receive an award for our 3rd place at the ManiSkill Challenge 2021, and present our related paper Improving performance on the ManiSkill Challenge via Super-convergence and Multi-Task Learning.

Here is a list of works presented at ICLR 2022 main conference or workshops that I found interesting.

Robot morphology

Structure-Aware Transformer Policy for Inhomogeneous Multi-Task Reinforcement Learning Uses a graph model of the robot morphology as input, allowing generalization to different robots

Transform2Act: Learning a Transform-and-Control Policy for Efficient Agent Design Co-evolution of design and control strategy

Learning Transferable Policies By Inferring Agent Morphology Learns to generalize accross morphologies by learning morphology representation

Multi-agent robot

Reinforcement Learning under a Multi-agent Predictive State Representation Model: Method and Theory Uses an agent interaction graph for optimal policy learning.

Collective control of modular soft robots via embodied Spiking Neural Cellular Automata Presents a novel form of collective control of a voxel based soft robot based on spiking neural network.

Robot Learning

VAT-Mart: Learning Visual Action Trajectory Proposals for Manipulating 3D ARTiculated Objects Interesting generation of push/pull trajectory from affordance maps

Learning Vision-Guided Quadrupedal Locomotion End-to-End with Cross-Modal Transformers End to end learning of locomotion of quadruped robot from rgbd input. Transfer from simulation ro reality apparently working well.

Reinforcement Learning with Sparse Rewards using Guidance from Offline Demonstration A step from Imitation Learning where example trajectories are used for guidance, but learning is still based on reward.


Accelerated Policy Learning with Parallel Differentiable Simulation Differentiable simulation have helped with simple low level control, but not yet with more complex task necessiting many simulation steps. The proposed algorithms, SHAC, only propagates gradient through short horizons, thus obtaining benefits from RL and gradient based methods. They also used a GPU based diffentiable physics simulator allowing for massive parallelism.

RISP: Rendering-Invariant State Predictor with Differentiable Simulation and Rendering for Cross-Domain Parameter Estimation Solves the inverse problem of solving physical system dynamic parameters from video observations via a differentiable simulator.

Understanding Domain Randomization for Sim-to-real Transfer Theoretical fraemeword showing that domain randomization works, especially if memory-augmented policy models are used.


Evolution based strategies

A Unified Substrate for Body-Brain Co-evolution Evolves a Neural Cellular Automota to control the growth of a robot as well as the robot itself.

Efficiently Evolving Swarm Behaviors Using Grammatical Evolution With PPA-style Behavior Trees Evolves behavior trees for swarm behaviors.


A little taxonomy of open-endedness Potentially interesting when discussing robots in "open-ended" environments.

Fixing Malfunctional Objects With Learned Physical Simulation and Functional Prediction Potential for fixing scanned objects, or as part of an assisted design workflow.