All News
| Feb 27, 2024 |
Recently published in IEEE Transactions on Robotics!
How much can you accomplish with only a few minutes of data to learn from? Quite a bit! We use 4 minutes of experiential data to learn a model for robust real-time manipulation of a previously unknown object. Work led by Wanxin Jin, and supported by the Toyota Research Institute. Dexterous manipulation, making and breaking frictional contact, is inherently hybrid, with thousands of possible modes. Fortunately, most of these are unnecessary for control. Here, we’re learning a task-relevant reduced-order hybrid model, limiting the number of hybrid modes. This builds on a bunch of our recent work on (1) data-efficient learning of multi-contact models (ContactNets and related papers) and (2) real-time MPC through contact. In this paper, we bridge these two by imbuing the model-learning process with task relevancy. Check out the project website, paper, and freely available arXiv version |
|---|---|
| Feb 27, 2024 | We had three papers accepted to ICRA 2024. Congratulations to Hien, Yu-Ming, Wei-Cheng, Alp and Wanxin! |
| Oct 14, 2023 | For the upcoming 2023-2024 application cycle, we will be looking to recruit multiple incoming Ph.D. students across all relevant departments (MEAM, ESE, or CIS). We are dedicated to assembling a dynamic and diverse team of researchers, and actively seek individuals with diverse cultural, ethnic, socioeconomic, and academic backgrounds. Get more information and apply here. |
| Oct 9, 2023 | It has been a busy stretch for the lab! Two weeks ago, Alp Aydinoglu his Ph.D. thesis, titled “Control of Multi-Contact Systems via Local Hybrid Models.” Alp developed a class of algorithms for the control of multi-contact robotic systems, including MPC strategies that, in real time, are able to plan novel contact sequences. Check out the talk! |
| Sep 25, 2023 | Last week, Yu-Ming Chen defended his Ph.D. thesis! Yu-Ming’s thesis, titled “Toward High-performance Simple Models of Legged Locomotion”, explored the use of optimization and machine learning to computationally discover new and improved simple models that enable performant locomotion while remaining low-dimensional and easy to plan with. Check out the talk! |
| Aug 20, 2023 | We’re excited to be presenting two papers at CoRL this year! “Simultaneous Learning of Contact and Continuous Dynamics,” by Bibit Bianchini and Mathew Halm, builds upon our prior work in data-efficient learning of contact dynamics to jointly learn (and disambiguate) the effects of contact and those of continuous forces, solely by observing an object’s motion. “Vision-Based Contact Localization Without Touch or Force Sensing”, by Leon Kim and Yunshuang Li, with Dinesh Jayaraman, learns to identify and locate extrinsic contact between a held object and the environment. We use a single depth camera view, trained in simulation, and show surprising generality across different domains and the ability to make quality predictions despite occlusions. |
| Jul 28, 2023 | Following his defense, Mathew Halm’s Ph.D. thesis, “Addressing Stiffness-induced Challenges In Modeling and Identification of Rigid Body Systems with Frictional Impact,” is now available here. |
| Jul 13, 2023 | This week, Michael gave an Early Career Spotlight talk at Robotics: Science and Systems in Daegu, South Korea. |
| Jun 23, 2023 | Earlier this week, Mathew Halm was the first DAIR lab member to defend his Ph.D. thesis! Matt’s thesis, titled “Addressing Stiffness-induced Challenges In Modeling and Identification of Rigid Body Systems with Frictional Impact,” made fundamental advances in data-efficient learning of dynamical models for when robots touch the world. Check out the talk! |
| Jun 2, 2023 | Another honor for Alp! Our paper Stabilization of Complementarity Systems via Contact-Aware Controllers , recieved an Honorable Mention for the 2022 IEEE Transactions on Robotics King-Sun Fu Memorial Best Paper Award. Congratulations to Alp and Phil! |
| May 26, 2023 | We are very excited to congratulate our very own Wanxin Jin who has accepted an offer to join Arizona State University as a tenure-track assistant professor in the coming fall. Wanxin has been a postdoc in the lab since 2021 and has done fantastic work on task-driven modeling for manipulation, through our collaboration with TRI. Congratulations Wanxin! |
| May 16, 2023 | Over the last two years, we’ve been thrilled to have an ongoing collaboration with the Toyota Research Institute (TRI), particularly with the developers of Drake As this project enters its third year, Michael recently answered some questions about the research results and philosophy. Check out the Penn blog post, When Robots Touch the World or the related Penn news article. |
| Apr 13, 2023 | We are looking to hire a postdoc in the area of data-efficient learning and control for dexterous manipulation! Get more information and apply here. |
| Apr 12, 2023 | We received the NSF CAREER award for manipulating novel objects. We’re excited to deepen the connection between non-smooth contact dynamics and low-data learning for dexterous manipulation. This award builds on the work from many students in the lab, and is a credit to their hard work and ingenuity! |
| Dec 14, 2022 |
How much can you accomplish with only a few minutes of data to learn from? Quite a bit! We use 4 minutes of experiential data to learn a model for robust real-time manipulation of a previously unknown object. Work led by Wanxin Jin, and supported by the Toyota Research Institute. Dexterous manipulation, making and breaking frictional contact, is inherently hybrid, with thousands of possible modes. Fortunately, most of these are unnecessary for control. Here, we’re learning a task-relevant reduced-order hybrid model, limiting the number of hybrid modes. This builds on a bunch of our recent work on (1) data-efficient learning of multi-contact models (ContactNets and related papers) and (2) real-time MPC through contact. In this paper, we bridge these two by imbuing the model-learning process with task relevancy. Check out the project website and the paper draft https://arxiv.org/abs/2211.16657. |
| Nov 22, 2022 |
The last decade has seen tremendous progress in legged robots, driven by (among other things) optimization-based control. With Patrick Wensing, Yue Hu, Adrien Escande, Nicolas Mansard, and Andrea del Prete, we survey the field with an eye on what’s next
Within the breadth of work in this area, we identify four main points of distinction.
https://arxiv.org/abs/2211.11644 |
| Oct 25, 2022 |
The center of mass (CoM) position is the weight sum of each body’s position in a system, but is there an angular counterpart? Can we average each body’s orientation to get the ``angular center of mass” (ACoM)? Over the summer, Yu-Ming Chen worked with Boardwalk Robotics and IHMC, and has a new preprint out! In it, he introduces the ACoM in layman’s terms and shows an application to natural walking with the humanoid Nadia.
https://arxiv.org/abs/2210.08111 |
| May 30, 2022 | It was awesome hosting the robotics world last week! DAIR Lab students gave demos at the GRASP Tour and at the Convention Center, with some press from the Philadelphia Inquirer, Alp was an award finalist, Bibit and William presented recent results at workshops, and Michael served as Local Arrangements Chair. ICRA was the first in-person conference for most of the lab, and also the first in-person conference with lab alumni! Tianze, Yuhan, and Mihir were all spotted representing their current Ph.D. and career institutions. |
| May 22, 2022 |
We’re excited to see everyone at ICRA this week! Alp’s paper, Real-Time Multi-Contact Model Predictive Control via ADMM, which was named a Finalist for Oustanding Dynamics and Control Paper, will be presented twice:
|
| Apr 29, 2022 | Updating an older post, this paper was accepted to RA-L/IROS, congrats to Brian and Will! How well do modern robotics simulators reproduce impact dynamics? How important are appropriately tuned contact parameters for physical realism? We compared simulated trajectories against real impact data from a cube toss and Penn Cassie jumping (or more accurately, landing). Simulators faithfully capture near rigid impacts while struggling with elasticity. While accuracy in reproducing cube toss data is largely insensitive to contact parameters if the parameters are stiff enough, correct stiffness and damping are necessary for accurately reproducing Cassie trajectories. https://arxiv.org/abs/2110.00541 |
| Mar 1, 2022 | We had one paper accepted to ICRA 2022 Real-Time Multi-Contact Model Predictive Control via ADMM, and two accepted to L4DC 2022 Generalization Bounded Implicit Learning of Nearly Discontinuous Functions and Learning Linear Complementarity Systems. Congrats to Alp, Bibit, Wanxin and Matt! |
| Feb 28, 2022 | Michael will be giving a couple of talks soon, at UC Santa Barbara and the University of Toronto. The talk at Toronto will be live streamed on YouTube, check it out to learn some of the details on our latest work. |
| Dec 15, 2021 |
Implicit learning has shown a lot of promise, particularly for representing (near) discontinuous functions. For example, our recent work on ContactNets used implicit representations of geometry. Similarly, we’ve seen how unstructured, explicit approaches struggle with learning stiff or discontinuous functions. We’ve set out to better understand why (and when) implicit representations are useful. Most obviously, an implicit parameterization can better represent non-smoothness. Other authors have exploited this, for instance via embedding differentiable optimization into the learning process (Belbute-Peres et al., “End-to-end differentiable physics for learning and control”). However, this is only part of the story. If the underlying function to be learned is stiff or discontinuous, this stiffness ultimately manifests in the loss function. Instead, we’ve investigated ContactNets-inspired implicit losses which balance optimality of the embedded problem against prediction error, with better performance on near-discontinuous learning problems. In a new preprint, “Generalization Bounded Implicit Learning of Nearly Discontinuous Functions” by Bibit Bianchini et al., we show how this violation-implicit loss provably generalizes well to unseen data. The resulting loss landscape is well-conditioned (with low Lipschitz constants, despite the stiff underlying function). We also provably connect this loss to graph distance, a natural metric for evaluating steep or discontinuous functions. |
| Nov 19, 2021 |
DAIR Ph.D. student Matt Halm gave the Penn GRASP SFI seminar a few weeks ago. Check out the talk, “Physics-inspired learning for discontinuous contact dynamics.”
|
| Oct 21, 2021 |
We’re excited that the paper “Stabilization of Complementarity Systems via Contact-Aware Controllers,” led by Alp Aydinoglu, has been accepted for publication in IEEE Transactions on Robotics (TRO). In it, we solve bilinear matrix inequalities to synthesize control policies that explicitly use measured state and force for feedback. Check out the video where the controller stabilizies a cart-pole that slams into nearby walls!
https://arxiv.org/abs/2008.02104 |
| Oct 20, 2021 |
Michael gave the Robotics Seminar at MIT not too long ago. The talk was recorded and is publically available.
|
| Oct 14, 2021 | How well do modern robotics simulators reproduce impact dynamics? How important are appropriately tuned contact parameters for physical realism? We compared simulated trajectories against real impact data from a cube toss and Penn Cassie jumping (or more accurately, landing). Simulators faithfully capture near rigid impacts while struggling with elasticity. While accuracy in reproducing cube toss data is largely insensitive to contact parameters if the parameters are stiff enough, correct stiffness and damping are necessary for accurately reproducing Cassie trajectories. https://arxiv.org/abs/2110.00541 |
| Oct 13, 2021 | The lab has a new website. We’ve copied over some of our more recent news posts, but are largely starting fresh. Check it out! |
| Oct 1, 2021 | When a robot interacts with the world, inevitably it will touch the wrong thing or slip instead of sticking. How should feedback work when the contact mode is changing? Linearization is not useful and hybrid (MIQP) problems cannot be solved in real-time. I’ve been thinking about this problem since the start of my Ph.D., and we’ve finally made some real progress! An ADMM algorithm, which we call Consensus Complementarity Control (C3), jointly optimizes over trajectory and contact mode for real-time MPC. https://arxiv.org/abs/2109.07076 |
| Apr 21, 2021 | When objects collide, small changes in initial conditions can lead to dramatically different outcomes (imagine a pool break). Rigid models capture this via non-uniqueness. Typically, model-based controllers optimistically ignore these possibilities, sometimes leading to poor behavior around impact events. Using differential inclusions and complementarity problems, we describe and compute the set of possible outcomes for multiple, frictional impacts and provide guarantees of existence and completeness. https://arxiv.org/abs/2103.15714 |
| Mar 30, 2021 |
When a robot impacts its environment, it undergoes a large and rapid (though not quite instantaneous) change in velocity. Mode detection and state estimation in these brief periods are incredibly difficult, so it makes very little sense to apply feedback on these varying and imprecise velocity estimates. However, this uncertainty only applies to a subspace of velocities. In a new preprint, we project velocities onto an impact invariant subspace, preserving control authority in this subspace without spuriously reacting to impact-driven uncertainty.
http://arxiv.org/abs/2103.06907 |
| Mar 30, 2021 |
Differentiable physics models enable learning contact dynamics for robotic systems, but at what cost? The underlying stiffness of contact poses a fundamental challenge to deep learning methods. Via numerical experiments learning ODEs for contact dynamics, we find that stiffness severely impacts (1) training error, (2) generalization error, and (3) data efficiency. The theoretical underpinnings of these results are perhaps well known, arising from the high Lipschitz constants due to contact stiffness. However, given the rise of deep learning applied to differentiable physics models of contact, it’s important to keep in mind the limitations of these approximations. There’s a resulting fundamental trade-off between physical accuracy (for stiff robotic contact) and amenability to learning methods. Learning on artificially soft contact models may not transfer to stiffer, real systems! https://arxiv.org/abs/2103.15406 |