Dec 14, 2022 How much can you accomplish with only a few minutes of data to learn from? Quite a bit! We use 4 minutes of experiential data to learn a model for robust real-time manipulation of a previously unknown object. Work led by Wanxin Jin, and supported by the Toyota Research Institute. Dexterous manipulation, making and breaking frictional contact, is inherently hybrid, with thousands of possible modes. Fortunately, most of these are unnecessary for control. Here, we’re learning a task-relevant reduced-order hybrid model, limiting the number of hybrid modes. This builds on a bunch of our recent work on (1) data-efficient learning of multi-contact models (ContactNets and related papers) and (2) real-time MPC through contact. In this paper, we bridge these two by imbuing the model-learning process with task relevancy. Check out the project website and the paper draft https://arxiv.org/abs/2211.16657. The last decade has seen tremendous progress in legged robots, driven by (among other things) optimization-based control. With Patrick Wensing, Yue Hu, Adrien Escande, Nicolas Mansard, and Andrea del Prete, we survey the field with an eye on what’s next Within the breadth of work in this area, we identify four main points of distinction. The choice of contact model, with implications on discontinuity or differentiability, and whether/how an algorithm must sequence the schedule of environmental contacts. The role of simplified models, which efficiently capture essential dynamic properties, in enabling real-time control and planning. The choice between numerical methods for optimal control (e.g. iLQR/DDP/collocation). Optimization-based (often QP) strategies for realizing motion planes via real-time feedback. We hope this will serve as a useful introduction for both new and experienced roboticists, particularly those with new ideas in control and learning! Check out the preprint on arxiv–feedback is welcome! https://arxiv.org/abs/2211.11644 The center of mass (CoM) position is the weight sum of each body’s position in a system, but is there an angular counterpart? Can we average each body’s orientation to get the angular center of mass” (ACoM)? Over the summer, Yu-Ming Chen worked with Boardwalk Robotics and IHMC, and has a new preprint out! In it, he introduces the ACoM in layman’s terms and shows an application to natural walking with the humanoid Nadia. https://arxiv.org/abs/2210.08111 It was awesome hosting the robotics world last week! DAIR Lab students gave demos at the GRASP Tour and at the Convention Center, with some press from the Philadelphia Inquirer, Alp was an award finalist, Bibit and William presented recent results at workshops, and Michael served as Local Arrangements Chair. ICRA was the first in-person conference for most of the lab, and also the first in-person conference with lab alumni! Tianze, Yuhan, and Mihir were all spotted representing their current Ph.D. and career institutions. We’re excited to see everyone at ICRA this week! Alp’s paper, Real-Time Multi-Contact Model Predictive Control via ADMM, which was named a Finalist for Oustanding Dynamics and Control Paper, will be presented twice: Tuesday, at 3:45 in Room 123 (TuB17, Optimization and Optimal Control II Session) Wednesday, at 3:40 in Room 121 (WeAw2, Awards Session) Updating an older post, this paper was accepted to RA-L/IROS, congrats to Brian and Will! How well do modern robotics simulators reproduce impact dynamics? How important are appropriately tuned contact parameters for physical realism? We compared simulated trajectories against real impact data from a cube toss and Penn Cassie jumping (or more accurately, landing). Simulators faithfully capture near rigid impacts while struggling with elasticity. While accuracy in reproducing cube toss data is largely insensitive to contact parameters if the parameters are stiff enough, correct stiffness and damping are necessary for accurately reproducing Cassie trajectories. https://arxiv.org/abs/2110.00541 We had one paper accepted to ICRA 2022 Real-Time Multi-Contact Model Predictive Control via ADMM, and two accepted to L4DC 2022 Generalization Bounded Implicit Learning of Nearly Discontinuous Functions and Learning Linear Complementarity Systems. Congrats to Alp, Bibit, Wanxin and Matt! Michael will be giving a couple of talks soon, at UC Santa Barbara and the University of Toronto. The talk at Toronto will be live streamed on YouTube, check it out to learn some of the details on our latest work. Implicit learning has shown a lot of promise, particularly for representing (near) discontinuous functions. For example, our recent work on ContactNets used implicit representations of geometry. Similarly, we’ve seen how unstructured, explicit approaches struggle with learning stiff or discontinuous functions. We’ve set out to better understand why (and when) implicit representations are useful. Most obviously, an implicit parameterization can better represent non-smoothness. Other authors have exploited this, for instance via embedding differentiable optimization into the learning process (Belbute-Peres et al., “End-to-end differentiable physics for learning and control”). However, this is only part of the story. If the underlying function to be learned is stiff or discontinuous, this stiffness ultimately manifests in the loss function. Instead, we’ve investigated ContactNets-inspired implicit losses which balance optimality of the embedded problem against prediction error, with better performance on near-discontinuous learning problems. In a new preprint, “Generalization Bounded Implicit Learning of Nearly Discontinuous Functions” by Bibit Bianchini et al., we show how this violation-implicit loss provably generalizes well to unseen data. The resulting loss landscape is well-conditioned (with low Lipschitz constants, despite the stiff underlying function). We also provably connect this loss to graph distance, a natural metric for evaluating steep or discontinuous functions. DAIR Ph.D. student Matt Halm gave the Penn GRASP SFI seminar a few weeks ago. Check out the talk, “Physics-inspired learning for discontinuous contact dynamics.” We’re excited that the paper “Stabilization of Complementarity Systems via Contact-Aware Controllers,” led by Alp Aydinoglu, has been accepted for publication in IEEE Transactions on Robotics (TRO). In it, we solve bilinear matrix inequalities to synthesize control policies that explicitly use measured state and force for feedback. Check out the video where the controller stabilizies a cart-pole that slams into nearby walls! https://arxiv.org/abs/2008.02104 Michael gave the Robotics Seminar at MIT not too long ago. The talk was recorded and is publically available. How well do modern robotics simulators reproduce impact dynamics? How important are appropriately tuned contact parameters for physical realism? We compared simulated trajectories against real impact data from a cube toss and Penn Cassie jumping (or more accurately, landing). Simulators faithfully capture near rigid impacts while struggling with elasticity. While accuracy in reproducing cube toss data is largely insensitive to contact parameters if the parameters are stiff enough, correct stiffness and damping are necessary for accurately reproducing Cassie trajectories. https://arxiv.org/abs/2110.00541 The lab has a new website. We’ve copied over some of our more recent news posts, but are largely starting fresh. Check it out! When a robot interacts with the world, inevitably it will touch the wrong thing or slip instead of sticking. How should feedback work when the contact mode is changing? Linearization is not useful and hybrid (MIQP) problems cannot be solved in real-time. I’ve been thinking about this problem since the start of my Ph.D., and we’ve finally made some real progress! An ADMM algorithm, which we call Consensus Complementarity Control (C3), jointly optimizes over trajectory and contact mode for real-time MPC. https://arxiv.org/abs/2109.07076 When objects collide, small changes in initial conditions can lead to dramatically different outcomes (imagine a pool break). Rigid models capture this via non-uniqueness. Typically, model-based controllers optimistically ignore these possibilities, sometimes leading to poor behavior around impact events. Using differential inclusions and complementarity problems, we describe and compute the set of possible outcomes for multiple, frictional impacts and provide guarantees of existence and completeness. https://arxiv.org/abs/2103.15714 When a robot impacts its environment, it undergoes a large and rapid (though not quite instantaneous) change in velocity. Mode detection and state estimation in these brief periods are incredibly difficult, so it makes very little sense to apply feedback on these varying and imprecise velocity estimates. However, this uncertainty only applies to a subspace of velocities. In a new preprint, we project velocities onto an impact invariant subspace, preserving control authority in this subspace without spuriously reacting to impact-driven uncertainty. http://arxiv.org/abs/2103.06907 Differentiable physics models enable learning contact dynamics for robotic systems, but at what cost? The underlying stiffness of contact poses a fundamental challenge to deep learning methods. Via numerical experiments learning ODEs for contact dynamics, we find that stiffness severely impacts (1) training error, (2) generalization error, and (3) data efficiency. The theoretical underpinnings of these results are perhaps well known, arising from the high Lipschitz constants due to contact stiffness. However, given the rise of deep learning applied to differentiable physics models of contact, it’s important to keep in mind the limitations of these approximations. There’s a resulting fundamental trade-off between physical accuracy (for stiff robotic contact) and amenability to learning methods. Learning on artificially soft contact models may not transfer to stiffer, real systems! https://arxiv.org/abs/2103.15406