Pretrain to Adapt: What Makes a Pretrained Policy Adaptable?

Overview

The strongest robotic manipulation systems today are typically built in two stages: first, pretrain a generalist policy on a large and diverse dataset, then adapt it to the specific setting where we actually want to deploy it. Adaptation can take many forms — supervised or RL fine-tuning, in-context learning, or test-time steering — and can adapt models not only to new tasks, but also to new embodiments and environments. Such adaptation is often critical for achieving effective performance in the target deployment domain.

These two stages optimize for different objectives, however: pretraining targets broad competence across many tasks, while adaptation targets the specific deployment setting of interest. Pretraining does not directly optimize for how readily a policy can be adapted — the final objective we often care about. The community has devoted significant effort to expanding pretrained capabilities by scaling the number of embodiments, tasks, compute, and data diversity considered in pretraining, on the assumption that a stronger pretrained generalist makes a better starting point for adaptation. But the most capable pretrained model is not necessarily the easiest to adapt, and we have no reliable way to tell, before adaptation, whether one pretrained policy is a better starting point than another. Meanwhile, approaches for adapting generalist policies to downstream settings typically assume the generalist policy is fixed, and do not consider how adaptation performance might be limited or enabled by the choice of pretrained policy.

This workshop seeks to bridge these two stages and ask what makes a pretrained policy a good foundation for adaptation? Our goal is to bring together researchers working on pretraining generalist policies with those developing approaches for adaptation, and to inspire discussion on how the two should be designed together: what properties of a pretrained robot policy make it an effective starting point for downstream adaptation, and how adaptation objectives should shape the way we evaluate and build pretrained models.

Core Research Questions

Capabilities and limits of adaptation

What can different forms of adaptation extract from a pretrained policy?

Supervised finetuning, reinforcement learning, in-context learning, test-time steering, and continual learning provide very different mechanisms for incorporating new information. When can adaptation efficiently recombine or unlock behaviors already in a pretrained model, and when does it require learning genuinely new capabilities with parameter updates? How do downstream data, feedback, and interaction budgets determine what is reachable from a given pretrained initialization?

Coupling pretraining and adaptation

How should the pretraining recipe depend on the adaptation method it will be paired with, and vice versa?

Different adaptation methods ask different things of a pretrained model: reinforcement learning needs broad behavioral coverage and the plasticity to keep improving; in-context learning needs the model to treat new task data as a prompt; steering requires that the pretrained model already contains distinct behavioral modes that can be selected through conditioning.

Measuring readiness to adapt

How can we measure readiness for adaptation?

Pretraining is usually judged by validation loss, but loss measures how well a model fits its data, not how well it adapts. Readiness for adaptation is instead a property of a model's learning dynamics: how cheaply it can be steered toward new behaviors. Can it be predicted from offline metrics alone, or estimated efficiently by probing adaptation on a small set of held-out tasks? Can two policies with similar zero-shot performance nevertheless have fundamentally different adaptation potential?

Pretraining for adaptation

If a pretrained policy will ultimately be adapted to new settings, how should pretraining prepare it for that process?

Beyond standard behavior cloning on expert demonstrations, alternative objectives and heterogeneous data—such as play, suboptimal and failure trajectories, or autonomous experience—may shape the behavioral coverage, steerability, and plasticity available at adaptation time. Should pretrained policies be explicitly trained to infer and exploit new task information from context? Which pretraining objectives, data mixtures, and training curricula best support efficient downstream adaptation?

Speakers

Karl Pertsch

Physical Intelligence

Sherry Yang

Asst. Professor, NYU

Jesse Zhang

Postdoc, University of Washington

Siddharth Karamcheti

Asst. Professor, Georgia Tech

In addition to our invited speakers, the panel discussion will also feature:

Felix Yanwei Wang

Generalist AI

Panelist

Mengdi Xu

Asst. Professor, Tsinghua University

Panelist

Accepted Papers

Twenty-nine papers were accepted: five orals and twenty-four posters. They cover:

What pretraining data and objectives make a policy easy to adapt
Reinforcement learning post-training and real-world policy improvement
Test-time steering and routing for vision-language-action models
World models for policy evaluation and planning
Memory, plasticity, and continual learning
Tactile and multisensory adaptation
Transfer across embodiments

The five orals are short talks from 2:55–3:30 PM. All twenty-nine papers are presented at the poster session from 3:30–4:15 PM.

See all accepted papers

Titles, authors, affiliations and poster board assignments.

Program

Tentative schedule for November 12, 2026 in Austin, TX — subject to change.

2:00 – 2:05 PM

Welcome

Opening Remarks

2:05 – 2:30 PM

Invited Talk 1

Karl Pertsch

Physical Intelligence

2:30 – 2:55 PM

Invited Talk 2

Sherry Yang

Asst. Professor, NYU

2:55 – 3:30 PM

Oral Talks

Contributed Oral Talks

Five papers, 7 minutes each — 6 min talk + 1 min Q&A.

2:55CrossBFM: Distilling a Shared Latent Behavior Space Across Humanoid Embodiments
3:02OmniTacTune: Policy-Agnostic Real-World RL for Tactile Residual Adaptation of Visual Policies
3:09Beyond Validation Loss: Coverage and Reliability Dynamics in Robot Policy Pretraining
3:16Multisensory Continual Learning: Adapting Pretrained Visuomotor Policies to Force
3:23Bi-Lipschitz-Regularized Pretraining for Efficient Policy Finetuning

3:30 – 4:15 PM

Break & Posters

Coffee Break + Poster Session

Coffee served 3:30 – 4:00 PM.

4:15 – 4:40 PM

Invited Talk 3

Siddharth Karamcheti

Asst. Professor, Georgia Tech

4:40 – 5:05 PM

Invited Talk 4

Jesse Zhang

Postdoc, University of Washington

5:05 – 5:55 PM

Panel

Panel Discussion

Open discussion with invited speakers and panelists.

5:55 – 6:00 PM

Closing

Closing Remarks

Interactive Activities

💬

Before the workshop

Slack Workspace

We have created a Slack workspace for authors and attendees interested in the workshop topics to discuss ideas and share papers prior to the workshop date.

🖼️

During the workshop

Interactive Poster Session

All accepted papers present posters during the 3:30 – 4:15 PM session, with coffee served alongside. Authors stay at their boards for the full slot so attendees can work through methods and results one-on-one with them.

❓

Before & during

Panel Q&A

We will collect questions for the panel discussion from the Slack workspace, an online poll, and in-person attendees. Submit a question for the panel.