CORE - Common Outcome Regularities for Robot Manipulation

Overview

Human videos contain useful evidence of a task's completed state.

CORE turns successful terminal observations into concrete, reusable visual conditions. The robot is not asked to imitate human actions; it learns to pursue the visual regularities that successful outcomes share.

Robot imitation learning often relies on expensive robot demonstrations. Meanwhile, abundant human or action-free videos are difficult to use directly because they lack robot-executable actions and may differ strongly in embodiment.

CORE addresses this gap by learning Common Outcome Regularities: stable object configurations, spatial relations, and contact constraints that define successful completion. These learned representations are distilled into visual goal prototypes and injected into robot policy backbones.

Outcome-guided policy learning with CORE compared with language-conditioned policies

Figure 1 - Why outcomes? Language instructions give high-level task intent but can omit geometry, pose, contact, and terminal-state constraints. CORE learns visual goal prototypes from successful terminal states and uses them to guide execution.

Key idea

Common outcomes transfer better than trajectories.

The method keeps what is essential to task success and sidesteps what cannot be transferred across bodies, cameras, and action spaces.

01

Use action-free demos

Successful videos carry task-level physical priors even without robot action labels or a matched embodiment.

02

Extract outcome regularities

CORE concentrates representation learning on terminal states, making shared object configurations and contact relations explicit.

03

Inject into policy backbones

A compact goal-conditioning module augments established policy families with concrete visual goal features.

Method

From raw video to an outcome-conditioned policy in three stages.

CORE converts successful terminal observations into task-level visual goal prototypes, then conditions the robot policy on both its current outcome embedding and the desired prototype.

CORE pipeline with terminal representation learning, goal prototype construction, and policy injection

Figure 2 - CORE pipeline. A terminal outcome encoder is trained first; successful embeddings are then aggregated into goal prototypes; finally, the policy uses the outcome feature and prototype together for action generation.

01

Terminal representation learning

Contrastive sampling pulls successful terminal outcomes together and separates non-terminal or task-negative observations.

02

Goal prototype construction

Successful terminal embeddings are aggregated into robust task-level prototypes with K-Means and Top-K averaging.

03

Backbone-agnostic injection

The policy receives a joint condition composed of the current outcome embedding and the static visual goal prototype.

Results

Higher success rates in simulation and on real robots.

Average success rate (%) reported in the paper. CORE consistently improves its matching backbone and outperforms frozen text-conditioning in the evaluated settings.

Benchmark	DP3	DP3 + Text	DP3 + CORE	MP1	MP1 + Text	MP1 + CORE
Meta-World	81.5	81.9	84.0 (+2.5)	81.7	81.4	85.6 (+3.9)
RoboTwin 2.0	77.5	77.4	82.3 (+4.8)	68.3	69.7	79.4 (+11.1)
Real World	65.0	68.0	82.0 (+17.0)	69.0	67.0	85.0 (+16.0)

Meta-World
39 tasks across Easy, Medium, Hard, and Very Hard levels.

RoboTwin 2.0
10 representative tasks spanning placement, handover, drawers, and stacking.

Real world
Five contact-rich tasks using an ARX R5 arm and Intel RealSense L515 observations.

Ablations

The outcome encoder and prototype design both matter.

Each component contributes to a representation of task completion that is both discriminative and useful for policy control.

ABLATION 01

Full CORE on hard tasks

On Meta-World Hard and Very Hard tasks, MP1 + CORE reaches 71.7 average success versus 64.2 for MP1.

ABLATION 02

Top-K prototype averaging

Top-K construction reaches 71.7 average success, outperforming random terminal selection and a single mean prototype.

ABLATION 03

Learned visual encoder

Across four tasks, CORE + DP3 averages 79.8 versus 59.2 for Uni3D + DP3, showing that terminal outcome learning is central.

Analysis

Evidence that the representation, data, and prototype all contribute.

The paper examines more than the final average: CORE improves optimization behavior, benefits from additional action-free visual data, and outperforms a frozen general-purpose visual encoder.

Training curves on four representative RoboTwin 2.0 tasks

TRAINING DYNAMICS

Higher and more stable learning curves

Across representative RoboTwin 2.0 tasks, CORE-integrated DP3 and MP1 policies sustain stronger success rates as training progresses.

Effect of action-free human demonstration quantity

DATA SCALING

Useful from limited video data

Performance rises as action-free human demonstrations grow from 5 to 100, with early stabilization on Pick Place Wall.

Comparison of the learned CORE visual encoder and a frozen Uni3D encoder

ENCODER STUDY

Learn the outcome representation

On four tasks, CORE + DP3 reaches 79.8% versus 59.2% for Uni3D + DP3; CORE + MP1 reaches 85.7% versus 67.4%.

Demos

From visual demonstration to CORE execution.

Each row shows the same task: a visual demonstration, a policy baseline, and the result conditioned on a learned CORE prototype.

Open DrawerVisual demo / baseline / CORE

Visual demonstration

Action-free evidence of the completed state.

MP1 baseline

Can stop short of the desired terminal state.

MP1 + CORE

Guided by the learned outcome prototype.

Close DrawerVisual demo / baseline / CORE

Visual demonstration

The terminal structure defines completion.

MP1 baseline

Final-state details remain under-specified.

MP1 + CORE

Goal prototypes guide successful completion.

Adjust BottleVisual demo / CORE execution

Visual demonstration

Action-free evidence of the target bottle arrangement.

CORE execution

The learned visual goal condition guides the final adjustment.

CookVisual demo / CORE execution

Visual demonstration

Action-free video captures the task's terminal structure.

CORE execution

CORE conditions the policy toward a successful contact-rich outcome.

Stack Three BowlsVisual demo / baseline / CORE

Visual demonstration

Shared evidence of a nested terminal configuration.

MP1 baseline

A contact-rich task with precise terminal constraints.

MP1 + CORE

Outcome conditioning guides the final arrangement.

Qualitative comparison of CORE and MP1 on drawer tasks

Figure 3 - Qualitative comparison. On Open Drawer and Close Drawer, CORE reaches target terminal states more reliably than MP1 in the shown cases.

Real-world setup with ARX robotic arm, Intel RealSense L515, and objects

Real-world setup. Five contact-rich tasks are evaluated using 100 robot demonstrations and 50 action-free human demonstrations per task.

Real-world tasks

Task-level success rates.

CORE improves policy outcomes across multiple contact-rich real-world manipulation tasks.

Method	Adj. Bottle	Cook	Close Drawer	Open Drawer	Stack3	Average
DP3	60	80	65	45	75	65.0
MP1	70	75	70	55	75	69.0
DP3 + Text	60	80	75	50	75	68.0
MP1 + Text	65	80	70	50	70	67.0
DP3 + CORE	75	95	85	70	85	82.0
MP1 + CORE	85	90	90	75	85	85.0

Citation

Cite CORE

If CORE is useful for your work, please cite the arXiv paper and explore the implementation.

arXiv LogSSim / CORE

@misc{sheng2026core,
  title  = {CORE: Common Outcome Regularities from Action-Free Visual
            Demonstrations for Robot Manipulation},
  author = {Sheng, Juyi and Tan, Mingxin and Li, Jincheng and Liu, Mengyuan},
  year   = {2026},
  note   = {arXiv:2606.29517},
  url    = {https://arxiv.org/abs/2606.29517}
}