How much can you accomplish with only a few minutes of data to learn from? Quite a bit! We use 4 minutes of experiential data to learn a model for robust real-time manipulation of a previously unknown object. Work led by Wanxin Jin, and supported by the Toyota Research Institute.

Dexterous manipulation, making and breaking frictional contact, is inherently hybrid, with thousands of possible modes. Fortunately, most of these are unnecessary for control. Here, we’re learning a task-relevant reduced-order hybrid model, limiting the number of hybrid modes. This builds on a bunch of our recent work on (1) data-efficient learning of multi-contact models (ContactNets and related papers) and (2) real-time MPC through contact. In this paper, we bridge these two by imbuing the model-learning process with task relevancy. Check out the project website and the paper draft