Show HN: VLDA-learn compact Deltas as actionable transition interfaces for robot

Back to Projects

LoongForge-VLDA

Vision-Language-Delta-Action: learning compact Deltas as actionable transition interfaces for robotic manipulation.

1 Harbin Institute of Technology, Shenzhen · 2 Baidu, Inc. · 3 Shenzhen Loop Area Institute

• Equal contribution † Corresponding author

What does an action actually change? Manipulation is a chain of state transitions, and most of every frame stays the same. VLA models must infer that change implicitly; world-action models predict whole future frames and spend capacity on pixels that never move.

VLDA distills only what changes into a compact Delta and hands it to the action head as an explicit interface: understand the scene, express the transition, then execute.

A V-JEPA 2 EMA teacher provides the target transition. Two filters distill it into a Delta; one refinement module feeds it back into the action head.

Selects the top-64 regions with the largest transition magnitude and matches them to the predicted Delta by Sinkhorn optimal transport, so the Delta is not diluted by static background.

Anchors the local patches to a global summary of the transition, keeping the Delta consistent with the task as a whole.

DAMG grounds the Delta on the action manifold with an InfoNCE loss. DSAF seeds the action flow with a Delta-based noise bias and a prefix token.

Pre-train the Delta from the teacher, without actions.

Train the action head with the Delta as an interface, keeping the stage-1 loss as a regularizer.

Built on LoongForge. VLDA is trained with the LoongForge embodied stack: DDP, ZeRO-1, FSDP and HSDP, component-wise freezing and checkpointing. Training used 8 GPUs. Inference needs no future observations, and the EMA teacher can be dropped after training.

Three simulation benchmarks and a set of ablations. Our row is highlighted.

VLDA beats π0.5 by 1.6 points on average and edges out every other baseline, reaching 95.2% on the Long suite.

Ahead of π0.5 on all seven perturbation dimensions, and ahead of the strongest baseline, GuidedVLA, on six of them (+9.1 on average). The gap widens on the Long suite: +13.4 over π0.5 and +12.1 over GuidedVLA.

7 perturbation dimensions · 10,030 rollouts

Zero-shot success rate (%) on LIBERO-Plus

Best or tied-best on every selected task. Put Bottles Dustbin jumps +46, Stacking Three +30, Place Dual Shoes +27 and Open Microwave +27. Across all 50 tasks VLDA scores 50.86% against 43.84% for π0.5.

10 selected hard tasks · 50-task average 50.86% vs 43.84%

Removing any single piece costs accuracy. LT-Filter and GT-Anchor each help alone, and together they help most, notably on the real robot.

Basket Relocation, R1 Lite: 48% without both → 72% with both

Simulation columns are LIBERO success rate (%); real-world columns are Ring Stacking, Table Sweeping, Conveyor Sorting and Basket Relocation on R1 Lite. Each module helps alone; together they help most.

Four long-horizon tasks on a GALAXEA R1 Lite, 25 trials each, plus a second platform, AgileX Cobot Magic. Numbers are final-stage success rates (%).

Averaged over four tasks, VLDA reaches 73.0% against 56.0% for π0.5. On a second robot the lead holds. Basket Relocation uses an out-of-distribution mat position, and VLDA still reaches 72% at the final placement.

The Delta concentrates on the regions that move and interact, not on the static scene.

Training code, configs and recipes for VLDA live in LoongForge.

git clone https://github.com/baidu-baige/LoongForge.git

Citation

@misc{zhang2026visionlanguagedeltaactionlearningcompactdeltas, title={Vision-Language-Delta-Action: Learning Compact Deltas as Actionable Transition Interfaces for Robotic Manipulation}, author={Lingsen Zhang and Wei Li and Jie He and Henghua Zhang and Kai Mo and Shiyong Li and Yanpeng Wang and Rui Shao}, year={2026}, eprint={2610.25874}, archivePrefix={arXiv}, primaryClass={cs.RO}, url={https://arxiv.org/abs/2610.25874}, }

Read the preprint on arXiv ↗.