Human-Centered Robot Learning in the Era of Big Data and Large Models

May 19th, 2025 | Atlanta, Georgia, USA | ICRA 2025

Recent advances in training large-scale models on massive datasets have led to the development of foundational models, such as large language models (LLMs) and large multimodal models (LMMs), capable of interacting with humans to perform sophisticated tasks. These models also enable new paradigms for human-robot interaction, facilitating the deployment of robots in human-centered environments to operate alongside humans (e.g., transportation, household tasks, healthcare, and warehouses). For example, robots powered by LLMs and LMMs show promise in following complex human instructions, collaborating with humans, and understanding and adhering to social norms.

However, compared to deploying AI models that interact with humans virtually (e.g., chatbots), deploying those embodied AI agents to interact with humans in the physical world imposes various unique challenges, for example: 1) The models need to be grounded in the physical world, especially in human representations and understanding of the physical world, to facilitate efficient and trustworthy human interaction; 2) The risk of physical harm to humans imposes significantly higher requirements on AI safety and alignment methods; 3) Robot data, especially human-robot interaction data, is much less accessible, compared to the abundance of internet data used for training virtual AI models.

This workshop aims to bring together interdisciplinary experts from related fields such as robot learning, human-robot interaction, natural language processing, cognitive science, and trustworthy AI to gather and discuss these challenges. We envision that this cross-disciplinary exchange will build new pathways for developing and deploying large-scale models in human-centered embodied AI systems. We intend to guide our conversations with the following questions:

What are the ideal data types for learning embodied interactions, and how can we acquire them efficiently and safely?
How can we train large models for human-centered robots to ensure robustness across diverse human interaction behaviors?
What challenges do human-centered robotic applications pose for AI safety and alignment, and how can we align robot behaviors with stakeholder values?
What new interaction paradigms between humans and robots are enabled by big data and large models? Given these new paradigms, what are the best practices for allocating tasks to robots and people in a collaborative setting?

Speakers

Andrea Bajcsy

Carnegie Mellon University

Benjamin Kuipers

University of Michigan

Jens Kober

TU Delft

Harold Soh

National University of Singapore

Matthew Gombolay

Georgia Institute of Technology

Ruohan Zhang

Stanford University

Call For Papers

Submission deadline: ~~April 6th, 11:59pm AOE, 2025~~ April 7th, 11:59pm AOE, 2025

Review and decision: April 20th, 2025

Camera-ready deadline: May 11th, 2025

Openreview Submission Link

We invite submissions that address the fundamental challenges in enabling large-scale human-centered robot learning or demonstrate novel human-centered embodied AI systems enabled by large-scale data and models. The related topics include but are not limited to:

Foundation models for embodied AI
Data accessibility for embodied AI
AI Safety and alignment for embodied AI
Efficient fine-tuning of pre-trained large models for embodied AI
Applications in human-robot interaction and collaboration

Accepted papers will be presented as spotlight talks and/or posters. The accepted papers will be made available on the workshop website with the authors' consent. All papers will be presented on-site during the workshop.

Submission Guidelines

The review process will be single-blind. Submissions will be evaluated based on originality, technical quality, and relevance to the workshop themes. The review process will not be public; only the accepted papers will be available on the workshop website, given the authors' consent.

There is no strict page limit; however, we encourage submissions to be within 4-8 pages (excluding references) to facilitate thorough review. Papers should be submitted in PDF, adhering to the ICRA template and our submission guidelines. Also, we encourage authors to submit videos, code, or data in their supplementary material (zip file) or through external services like Github repos.

Additionally, to encourage discussion on the workshop topic, we ask authors to provide 1~2 paragraphs of statements on how their work relates to the workshop theme. This information will help us organize the workshop discussion and better feature the accepted works on the website.

Schedule

00 days

00 hours

00 minutes

Atlanta Time Local Time

	Opening Remarks Opening Remarks by Workshop Organizers	Organizers
	Keynote Talk 1 Models, Trust, and Unknown Unknowns	Benjamin Kuipers
	Keynote Talk 2 Data-Driven Approaches for Policy Learning in Household Robots	Ruohan Zhang
	Spotlight Talks of Contributed Papers Details Azizul Zahid: Toward Aligning Human and Robot Actions via Multi-Modal Demonstration Learning Haonan Chen: Tool-as-Interface: Learning Robot Tool Use from Human Play through Imitation Learning Kushal Kedia: One-Shot Imitation under Mismatched Execution Shuijing Liu: DRAGON: A Dialogue-Based Robot for Assistive Navigation with Visual Language Grounding Xuefei Sun: Spatial-LLaVA: Enhancing Large Language Models with Spatial Referring Expressions for Visual Understanding Noriaki Hirose: Learning to Drive Anywhere with Model-Based Reannotation Allan Wang: Memory-Maze: Scenario Driven Benchmark and Visual Language Navigation Model for Guiding Blind People Peiqi Liu: DynaMem: Online Dynamic Spatio-Semantic Memory for Open World Mobile Manipulation Xingeu Zhou: Curate, Connect, Inquire: A System for Findable Accessible Interoperable and Reusable (FAIR) Human-Robot Centered Datasets David Paulius: Lang2LTL-2: Grounding Spatiotemporal Navigation Commands Using Large Language and Vision-Language Models Alessio Galatolo (Remote): Look, Think, Understand: Multimodal Reasoning for Socially-Aware Robotics Wang Rundong (Remote/Video): RoboMAN: Human-like Compliance Manipulation for Electronics Assembly via an Online Memory-Augmented Network	Authors of contributions
	Coffee Break and Poster Session
	Keynote Talk 3 Learning from Intuitive Interactions with Human Teachers	Jens Kober
	Keynote Talk 4 (Tentative) Rethinking Social, Task, and Motion Planning with Foundation Models	Harold Soh
	Lunch Break
	Keynote Talk 5 What Can Robot Safety Learn from LLM Safety?	Andrea Bajcsy
	Keynote Talk 6 Value Alignment and Safety in Human-Robot Interactive and Explainable Learning with Models Big and Small	Matthew Gombolay
	Spotlight Talks of Contributed Papers Details Shenghui Chen: Learning Human Perception Dynamics for Informative Robot Communication Michael Munje: SocialNav-SUB: Benchmarking VLMs for Scene Understanding in Social Robot Navigation Jiageng Mao: Universal Humanoid Robot Pose Learning from Internet Human Videos Yaru Niu: Human2LocoMan: Learning Versatile Quadrupedal Manipulation with Human Pretraining Siddhant Haldar: Point Policy: Unifying Observations and Actions with Key Points for Robot Manipulation Bukeikhan Omarali: From Gaze to Action: Leveraging Affordance Grounding for Human Intention Understanding Weiwei Gu: Continual Robot Skill and Task Learning via Dialogue Maithili Patel: Robot Behavior Personalization from Sparse User Feedback Kartik Ramachandruni: PARSEC: Preference Adaptation for Robotic Object Rearrangement from Scene Context Yunfan Jiang (Remote): BEHAVIOR Robot Suite: Streamlining Real-World Whole-Body Manipulation for Everyday Household Activities Zichao Hu (Remote): ComposableNav: Instruction-Following Navigation in Dynamic Environments via Composable Diffusion Amanuel Ergogo (Remote): Towards Embodied Agent Intent Explanation in Human-Robot Collaboration: ACT Error Analysis and Solution Conceptualization	Authors of contributions
	Coffee Break and Poster Session
	Panel Discussion Panel Discussion with Keynote Speakers	All Speakers
	Closing Remarks Closing Remarks by Workshop Organizers	Organizers