All News

Sep 5, 2024 Congratulations to William Yang, who successfully defended his thesis, titled “Controlling Contact Transitions for Dynamic Robots.” Will demonstrated remarkable new robot capabilities for controlling both high-speed impact events for bipedal running and jumping, and for dynamic, dexterous manipulation exploiting sliding friction and stick-slip transitions. The thesis document itself will be uploaded soon.
Aug 1, 2024 For the upcoming 2024-2025 application cycle, we will be looking to recruit multiple incoming Ph.D. students across all relevant departments (MEAM, ESE, or CIS). We are dedicated to assembling a dynamic and diverse team of researchers, and actively seek individuals with diverse cultural, ethnic, socioeconomic, and academic backgrounds. Get more information and apply here.
Jul 19, 2024 Congratulations to Will Yang! Will’s paper, “Dynamic On-Palm Manipulation via Controlled Sliding,” received the Outstanding Student Paper Award at Robotics: Science and Systems (RSS)!
May 22, 2024 How far can you get by simply specifying that an object move? With contact-implicit MPC, quite far! Will Yang’s paper “Dynamic On-Palm Manipulation via Controlled Sliding,” accepted to RSS 2024, pushes the limits of real-time dexterity. Check out the project website and arXiv preprint
May 2, 2024 We have another recently published paper in IEEE Transactions on Robotics! Simple models, like inverted pendulums and single rigid-body models, are ubiquitous in legged locomotion. They achieve good performance with only a low-dimensional representation, but can an algorithm do better? We explore the use of numerical optimization to synthesize new simple models, achieving higher performance across a wider range of bipedal walking tasks. This work led by recent grad Yu-Ming Chen, and supported by the Toyota Research Institute.

Check out the project website, paper, and freely available arXiv version
Feb 27, 2024 Recently published in IEEE Transactions on Robotics! How much can you accomplish with only a few minutes of data to learn from? Quite a bit! We use 4 minutes of experiential data to learn a model for robust real-time manipulation of a previously unknown object. Work led by Wanxin Jin, and supported by the Toyota Research Institute.

Dexterous manipulation, making and breaking frictional contact, is inherently hybrid, with thousands of possible modes. Fortunately, most of these are unnecessary for control. Here, we’re learning a task-relevant reduced-order hybrid model, limiting the number of hybrid modes. This builds on a bunch of our recent work on (1) data-efficient learning of multi-contact models (ContactNets and related papers) and (2) real-time MPC through contact. In this paper, we bridge these two by imbuing the model-learning process with task relevancy. Check out the project website, paper, and freely available arXiv version
Feb 27, 2024 We had three papers accepted to ICRA 2024.
  1. Adaptive Contact-Implicit Model Predictive Control with Online Residual Learning
  2. Enhancing Task Performance of Learned Simplified Models via Reinforcement Learning
  3. Reinforcement Learning for Reduced-order Models of Legged Robots
Congratulations to Hien, Yu-Ming, Wei-Cheng, Alp and Wanxin!
Oct 14, 2023 For the upcoming 2023-2024 application cycle, we will be looking to recruit multiple incoming Ph.D. students across all relevant departments (MEAM, ESE, or CIS). We are dedicated to assembling a dynamic and diverse team of researchers, and actively seek individuals with diverse cultural, ethnic, socioeconomic, and academic backgrounds. Get more information and apply here.
Oct 9, 2023 It has been a busy stretch for the lab! Two weeks ago, Alp Aydinoglu his Ph.D. thesis, titled “Control of Multi-Contact Systems via Local Hybrid Models.” Alp developed a class of algorithms for the control of multi-contact robotic systems, including MPC strategies that, in real time, are able to plan novel contact sequences. Check out the talk!
Sep 25, 2023 Last week, Yu-Ming Chen defended his Ph.D. thesis! Yu-Ming’s thesis, titled “Toward High-performance Simple Models of Legged Locomotion”, explored the use of optimization and machine learning to computationally discover new and improved simple models that enable performant locomotion while remaining low-dimensional and easy to plan with. Check out the talk!
Aug 20, 2023 We’re excited to be presenting two papers at CoRL this year! “Simultaneous Learning of Contact and Continuous Dynamics,” by Bibit Bianchini and Mathew Halm, builds upon our prior work in data-efficient learning of contact dynamics to jointly learn (and disambiguate) the effects of contact and those of continuous forces, solely by observing an object’s motion. “Vision-Based Contact Localization Without Touch or Force Sensing”, by Leon Kim and Yunshuang Li, with Dinesh Jayaraman, learns to identify and locate extrinsic contact between a held object and the environment. We use a single depth camera view, trained in simulation, and show surprising generality across different domains and the ability to make quality predictions despite occlusions.
Jul 28, 2023 Following his defense, Mathew Halm’s Ph.D. thesis, “Addressing Stiffness-induced Challenges In Modeling and Identification of Rigid Body Systems with Frictional Impact,” is now available here.
Jul 13, 2023 This week, Michael gave an Early Career Spotlight talk at Robotics: Science and Systems in Daegu, South Korea.
Jun 23, 2023 Earlier this week, Mathew Halm was the first DAIR lab member to defend his Ph.D. thesis! Matt’s thesis, titled “Addressing Stiffness-induced Challenges In Modeling and Identification of Rigid Body Systems with Frictional Impact,” made fundamental advances in data-efficient learning of dynamical models for when robots touch the world. Check out the talk!
Jun 2, 2023 Another honor for Alp! Our paper Stabilization of Complementarity Systems via Contact-Aware Controllers , recieved an Honorable Mention for the 2022 IEEE Transactions on Robotics King-Sun Fu Memorial Best Paper Award. Congratulations to Alp and Phil!
May 26, 2023 We are very excited to congratulate our very own Wanxin Jin who has accepted an offer to join Arizona State University as a tenure-track assistant professor in the coming fall. Wanxin has been a postdoc in the lab since 2021 and has done fantastic work on task-driven modeling for manipulation, through our collaboration with TRI. Congratulations Wanxin!
May 16, 2023 Over the last two years, we’ve been thrilled to have an ongoing collaboration with the Toyota Research Institute (TRI), particularly with the developers of Drake As this project enters its third year, Michael recently answered some questions about the research results and philosophy. Check out the Penn blog post, When Robots Touch the World or the related Penn news article.
Apr 13, 2023 We are looking to hire a postdoc in the area of data-efficient learning and control for dexterous manipulation! Get more information and apply here.
Apr 12, 2023 We received the NSF CAREER award for manipulating novel objects. We’re excited to deepen the connection between non-smooth contact dynamics and low-data learning for dexterous manipulation. This award builds on the work from many students in the lab, and is a credit to their hard work and ingenuity!
Dec 14, 2022 How much can you accomplish with only a few minutes of data to learn from? Quite a bit! We use 4 minutes of experiential data to learn a model for robust real-time manipulation of a previously unknown object. Work led by Wanxin Jin, and supported by the Toyota Research Institute.

Dexterous manipulation, making and breaking frictional contact, is inherently hybrid, with thousands of possible modes. Fortunately, most of these are unnecessary for control. Here, we’re learning a task-relevant reduced-order hybrid model, limiting the number of hybrid modes. This builds on a bunch of our recent work on (1) data-efficient learning of multi-contact models (ContactNets and related papers) and (2) real-time MPC through contact. In this paper, we bridge these two by imbuing the model-learning process with task relevancy. Check out the project website and the paper draft https://arxiv.org/abs/2211.16657.
Nov 22, 2022 The last decade has seen tremendous progress in legged robots, driven by (among other things) optimization-based control. With Patrick Wensing, Yue Hu, Adrien Escande, Nicolas Mansard, and Andrea del Prete, we survey the field with an eye on what’s next Within the breadth of work in this area, we identify four main points of distinction.
  1. The choice of contact model, with implications on discontinuity or differentiability, and whether/how an algorithm must sequence the schedule of environmental contacts.
  2. The role of simplified models, which efficiently capture essential dynamic properties, in enabling real-time control and planning.
  3. The choice between numerical methods for optimal control (e.g. iLQR/DDP/collocation).
  4. Optimization-based (often QP) strategies for realizing motion planes via real-time feedback.
We hope this will serve as a useful introduction for both new and experienced roboticists, particularly those with new ideas in control and learning! Check out the preprint on arxiv–feedback is welcome!

https://arxiv.org/abs/2211.11644
Oct 25, 2022 The center of mass (CoM) position is the weight sum of each body’s position in a system, but is there an angular counterpart? Can we average each body’s orientation to get the ``angular center of mass” (ACoM)? Over the summer, Yu-Ming Chen worked with Boardwalk Robotics and IHMC, and has a new preprint out! In it, he introduces the ACoM in layman’s terms and shows an application to natural walking with the humanoid Nadia.

https://arxiv.org/abs/2210.08111
May 30, 2022 It was awesome hosting the robotics world last week! DAIR Lab students gave demos at the GRASP Tour and at the Convention Center, with some press from the Philadelphia Inquirer, Alp was an award finalist, Bibit and William presented recent results at workshops, and Michael served as Local Arrangements Chair. ICRA was the first in-person conference for most of the lab, and also the first in-person conference with lab alumni! Tianze, Yuhan, and Mihir were all spotted representing their current Ph.D. and career institutions.
May 22, 2022 We’re excited to see everyone at ICRA this week! Alp’s paper, Real-Time Multi-Contact Model Predictive Control via ADMM, which was named a Finalist for Oustanding Dynamics and Control Paper, will be presented twice:
  • Tuesday, at 3:45 in Room 123 (TuB17, Optimization and Optimal Control II Session)
  • Wednesday, at 3:40 in Room 121 (WeAw2, Awards Session)
Apr 29, 2022 Updating an older post, this paper was accepted to RA-L/IROS, congrats to Brian and Will! How well do modern robotics simulators reproduce impact dynamics? How important are appropriately tuned contact parameters for physical realism? We compared simulated trajectories against real impact data from a cube toss and Penn Cassie jumping (or more accurately, landing). Simulators faithfully capture near rigid impacts while struggling with elasticity. While accuracy in reproducing cube toss data is largely insensitive to contact parameters if the parameters are stiff enough, correct stiffness and damping are necessary for accurately reproducing Cassie trajectories. https://arxiv.org/abs/2110.00541
Mar 1, 2022 We had one paper accepted to ICRA 2022 Real-Time Multi-Contact Model Predictive Control via ADMM, and two accepted to L4DC 2022 Generalization Bounded Implicit Learning of Nearly Discontinuous Functions and Learning Linear Complementarity Systems. Congrats to Alp, Bibit, Wanxin and Matt!
Feb 28, 2022 Michael will be giving a couple of talks soon, at UC Santa Barbara and the University of Toronto. The talk at Toronto will be live streamed on YouTube, check it out to learn some of the details on our latest work.
Dec 15, 2021 Implicit learning has shown a lot of promise, particularly for representing (near) discontinuous functions. For example, our recent work on ContactNets used implicit representations of geometry. Similarly, we’ve seen how unstructured, explicit approaches struggle with learning stiff or discontinuous functions. We’ve set out to better understand why (and when) implicit representations are useful.

Most obviously, an implicit parameterization can better represent non-smoothness. Other authors have exploited this, for instance via embedding differentiable optimization into the learning process (Belbute-Peres et al., “End-to-end differentiable physics for learning and control”). However, this is only part of the story. If the underlying function to be learned is stiff or discontinuous, this stiffness ultimately manifests in the loss function.

Instead, we’ve investigated ContactNets-inspired implicit losses which balance optimality of the embedded problem against prediction error, with better performance on near-discontinuous learning problems. In a new preprint, “Generalization Bounded Implicit Learning of Nearly Discontinuous Functions” by Bibit Bianchini et al., we show how this violation-implicit loss provably generalizes well to unseen data. The resulting loss landscape is well-conditioned (with low Lipschitz constants, despite the stiff underlying function). We also provably connect this loss to graph distance, a natural metric for evaluating steep or discontinuous functions.
Nov 19, 2021 DAIR Ph.D. student Matt Halm gave the Penn GRASP SFI seminar a few weeks ago. Check out the talk, “Physics-inspired learning for discontinuous contact dynamics.”
Oct 21, 2021 We’re excited that the paper “Stabilization of Complementarity Systems via Contact-Aware Controllers,” led by Alp Aydinoglu, has been accepted for publication in IEEE Transactions on Robotics (TRO). In it, we solve bilinear matrix inequalities to synthesize control policies that explicitly use measured state and force for feedback. Check out the video where the controller stabilizies a cart-pole that slams into nearby walls!
https://arxiv.org/abs/2008.02104
Oct 20, 2021 Michael gave the Robotics Seminar at MIT not too long ago. The talk was recorded and is publically available.
Oct 14, 2021 How well do modern robotics simulators reproduce impact dynamics? How important are appropriately tuned contact parameters for physical realism? We compared simulated trajectories against real impact data from a cube toss and Penn Cassie jumping (or more accurately, landing). Simulators faithfully capture near rigid impacts while struggling with elasticity. While accuracy in reproducing cube toss data is largely insensitive to contact parameters if the parameters are stiff enough, correct stiffness and damping are necessary for accurately reproducing Cassie trajectories. https://arxiv.org/abs/2110.00541
Oct 13, 2021 The lab has a new website. We’ve copied over some of our more recent news posts, but are largely starting fresh. Check it out!
Oct 1, 2021 When a robot interacts with the world, inevitably it will touch the wrong thing or slip instead of sticking. How should feedback work when the contact mode is changing? Linearization is not useful and hybrid (MIQP) problems cannot be solved in real-time. I’ve been thinking about this problem since the start of my Ph.D., and we’ve finally made some real progress! An ADMM algorithm, which we call Consensus Complementarity Control (C3), jointly optimizes over trajectory and contact mode for real-time MPC. https://arxiv.org/abs/2109.07076
Apr 21, 2021 When objects collide, small changes in initial conditions can lead to dramatically different outcomes (imagine a pool break). Rigid models capture this via non-uniqueness. Typically, model-based controllers optimistically ignore these possibilities, sometimes leading to poor behavior around impact events. Using differential inclusions and complementarity problems, we describe and compute the set of possible outcomes for multiple, frictional impacts and provide guarantees of existence and completeness. https://arxiv.org/abs/2103.15714
Mar 30, 2021 When a robot impacts its environment, it undergoes a large and rapid (though not quite instantaneous) change in velocity. Mode detection and state estimation in these brief periods are incredibly difficult, so it makes very little sense to apply feedback on these varying and imprecise velocity estimates. However, this uncertainty only applies to a subspace of velocities. In a new preprint, we project velocities onto an impact invariant subspace, preserving control authority in this subspace without spuriously reacting to impact-driven uncertainty.
http://arxiv.org/abs/2103.06907
Mar 30, 2021 Differentiable physics models enable learning contact dynamics for robotic systems, but at what cost? The underlying stiffness of contact poses a fundamental challenge to deep learning methods. Via numerical experiments learning ODEs for contact dynamics, we find that stiffness severely impacts (1) training error, (2) generalization error, and (3) data efficiency.

The theoretical underpinnings of these results are perhaps well known, arising from the high Lipschitz constants due to contact stiffness. However, given the rise of deep learning applied to differentiable physics models of contact, it’s important to keep in mind the limitations of these approximations. There’s a resulting fundamental trade-off between physical accuracy (for stiff robotic contact) and amenability to learning methods.

Learning on artificially soft contact models may not transfer to stiffer, real systems! https://arxiv.org/abs/2103.15406