2nd Workshop on

Learning Effective Abstractions for Planning (LEAP)

CoRL 2024, Munich, Germany | November 9, 2024

Email: leap-workshop@googlegroups.com

Overview

Complex long-horizon and sparse reward robotics tasks are challenging for scaling end-to-end learning methods. By contrast, planning approaches have shown great potential to handle such complex tasks effectively. One of the major criticisms of planning-based approaches has been the lack of availability of accurate world models (aka abstractions) to utilize.

There has been a renewed interest in using learning-based approaches to learn symbolic representations that support planning. However, this research is often fragmented in disjoint sub-communities such as task and motion planning, reinforcement learning (hierarchical, model-based), planning with formal logic, planning with natural language (language models), and neuro-symbolic AI. This workshop aims to create a common forum to share insights, discuss key questions, and chart a path forward via abstraction.

We aim to facilitate this bridge-building in two ways: (1) a diverse selection of papers and invited speakers; and (2) a highly interactive workshop. Concretely, the workshop will highlight approaches that use different learning methods, mainly to learn symbolic and composable representations of world models. Key questions for discussion include:

  1. What is the right objective for abstraction learning for robotic planning? To what extent should we consider factors such as soundness, completeness, target planner and planning efficiency, and task distribution?
  2. What level of abstraction is needed for it to be effective? How general-purpose or specific do these abstractions have to be for long-term autonomy? Do learned abstractions need to be hierarchical or at a single level?
  3. To what extent should the abstractions used for robotic planning be interpretable or explainable to a human? How can this be achieved?
  4. How can existing pre-trained foundational models (large language models (LLMs) and vision-language models (VLMs)) be utilized for learning symbolic abstractions while ensuring guarantees about correctness?
  5. When, where, and from what data should abstractions be learned? Should they be learned as priors in the robot factory, using expert demonstrations, or in the “wild” from interaction with humans or the world?

Following the success of the previous offering of the workshop at CoRL 2023, we propose a second iteration of the workshop at CoRL 2024. Specifically, the previous iteration of the workshop received a total of 26 submissions highlighting important characteristics and challenges of learning abstractions. They highlighted how pre-trained foundational models, specifically LLMs or VLMs, enable learning various forms of abstractions for a diverse set of robotics tasks. This iteration of the workshop would have a stronger emphasis on learning provably correct symbolic abstractions using different techniques.

Areas of Interest

We solicit papers of the following topics:

Best Paper: ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulation

Important Dates

Paper Submission Deadline October 11, 2024 (AoE)
Paper Acceptance October 23, 2024 (AoE)
Camera-ready Version Due November 8, 2024 (AoE)
Workshop November 9, 2024

Schedule

Session 1

8:25 AM - 8:30 AM Welcome Remarks
8:30 AM - 9:00 AM Invited Talk:
Eric Rosen
9:00 AM - 9:30 AM Invited Talk:
Gregory Stein
9:30 AM - 10:00 AM Invited Talk:
Marc Toussaint
10:00 AM - 10:30 AM Contributed Talks (3 talks)
10:30 AM - 11:00 AM Coffee Break

Session 2

11:00 AM - 11:30 AM Invited Talk:
Georgia Chalvatzaki
11:30 AM - 12:00 PM Invited Talk:
Leslie Kaelbling
12:00 PM - 1:45 PM Lunch Break
(with Conference Industry Panel)

Session 3

1:45 PM - 2:15 PM Invited Talk:
Caelan Garrett
2:15 PM - 2:45 PM Invited Talk:
Siddharth Srivastava
2:45 PM - 3:30 PM Contributed Talks (3 talks)
3:30 PM - 4:00 PM Coffee Break

Session 4

4:00 PM - 4:50 PM Panel Discussion
Moderator: Leslie Kaelbling
Chelsea Finn
Animesh Garg
Siddharth Srivastava
Marc Toussaint
Vincent Vanhoucke
4:50 PM - 6:00 PM Closing Remarks & Poster Session

Invited Speakers

Georgia Chalvatzaki
Georgia Chalvatzaki
TU Darmstadt
Germany




Lesli Pack Kaelbling
Leslie Pack Kaelbling
Massachusetts Institute of Technology
USA


Eric Rosen
Eric Rosen
The AI Institute
Boston Dynamics
Siddharth Srivastava
Siddharth Srivastava
Arizona State University
USA


Gregory Stein
Gregory Stein
George Mason University
USA


Marc Toussaint
Marc Toussaint
TU Berlin
Germany


Organizing Committees


Naman Shah
Naman Shah
Brown University
USA


David Paulius
David Paulius
Brown University
USA


Nishanth Kumar
Nishanth Kumar
Massachusetts Institute of Technology
USA


Jiayun Mao
Jiayuan Mao
Massachusetts Institute of Technology
USA


Yiqing Xu
Yiqing Xu
Natitonal University of Singapore
Singapore


Nakul Gopalan
Nakul Gopalan
Arizona State University
USA


Rudolph Lioutikov
Rudolph Lioutikov
Karlsruhe Institute of Technology
Germany


George Konidaris
George Konidaris
Brown University
USA


Accepted Papers (Read them here)

Program Committee

* Outstanding Reviewers
Abhishek Paudel Ahmed Hendawy Ahmed Jaafar* Akhil Bagaria
Alper Ahmetoglu Anant Sah* Bartłomiej Cieślar Ben Eisner*
Benned Hedegaard Bingjie Tang Chengguang Xu Chongkai Gao
Cunjun Yu* Elie Aljalbout Eric Rosen* Hongyi Zhou
Hongze Wang Jason Xinyu Liu Jayesh Nagpal Jiafei Duan
Jiaxu Xing Jiyong Ahn Kevin Jatin Vora Linfeng Zhao
Max Merlin Mingxi Jia Minyoung Hwang Nils Blank*
Omkar Patil Pulkit Verma* Rashmeet Kaur Nayyar Roshan Dhakal*
Shivam Vats Shivansh Patel Shreyas Sundara Raman* Shu Wang
Snehal Jauhri Tabitha Edith Lee* Tianyang Pan* Tom Silver*
Utkarsh Aashu Mishra* Vignesh Prasad Viraj Parimi Weiwei Gu*
Weiyu Liu Xiaogang Jia Xiaolin Fang Xusheng Luo
Yaoyao Qian Yixuan Huang Yongchao Chen Yuchen Liu
Zirui Zhao* Ziyi Yang*