From Abstraction to Instantiation:
Learning Behavioral Representation for Vision-Language-Action Model

Bing Hu1, Zaijing Li1,2, Rui Shao†1,2,3, Junda Chen1, April Hua Liu4, Wei-Shi Zheng3,5, Liqiang Nie1
1Harbin Institute of Technology, Shenzhen 2PengCheng Laboratory 3Shenzhen Loop Area Institute 4Shanghai University of Finance and Economics 5Sun Yat-sen University
†Corresponding author
Accepted as an Oral at ICML 2026
arXiv PDF Code Coming Soon

Abstract

Vision-Language-Action (VLA) models often suffer from performance degradation under distribution shifts, as they struggle to learn generalized behavior representations across varying environments. While existing approaches attempt to construct behavior representations through action-centric latent variables, they are often limited by short-horizon temporal fragmentation and static execution-alignment, leading to inconsistent behaviors in complex scenarios. To address these limitations, we propose BehaviorVLA, a framework that facilitates robust manipulation through the learning of a temporally coherent behavioral representations. Our approach features two symmetric components: (1) the Visuomotor Behavior Encoder (VBE), which utilizes a causal Mamba-based architecture to aggregate long-horizon trajectory information into a unified behavior representation; and (2) the Phase-conditioned Behavior Decoder (PBD), which decodes this representation into precise actions by dynamically aligning task-level priors with real-time execution progress. Experiments on RoboTwin 2.0, LIBERO, and CALVIN demonstrate state-of-the-art success rates of 58%, 98%, and 4.36 (Avg. Len), respectively. Notably, in real-world sim-to-real transfer, BehaviorVLA matches the performance of OpenVLA-OFT using only 50% of the demonstration data, showcasing its superior data efficiency and generalization.

Method

Overview of the BehaviorVLA framework
Overview of BehaviorVLA. Given an instruction and observation, the vision-language backbone retrieves a global behavior prototype from the memory bank once at the beginning of an episode. During execution, the Visuomotor Behavior Encoder (VBE) models the current phase online with vision, action, and behavior streams. The Phase-Conditioned Behavior Decoder (PBD) fuses the stable prototype with the evolving phase state to guide the flow policy toward temporally coherent and precise actions.

Experiments

RoboTwin 2.0

BehaviorVLA performance comparison on RoboTwin 2.0
Results on RoboTwin 2.0. Under the Hard setting with domain randomization, BehaviorVLA reaches an average success rate of 58% across 20 bimanual manipulation tasks. The phase-conditioned predictor-corrector design improves both global coordination and local action precision in challenging scenes.

LIBERO

BehaviorVLA performance comparison on LIBERO
Results on LIBERO. BehaviorVLA achieves a 98.0% average success rate across the Spatial, Object, Goal, and Long suites. The strongest improvement appears on LIBERO-Long, where explicit behavior abstraction and phase-aware decoding help prevent temporal drift during extended tasks.

CALVIN

BehaviorVLA performance comparison on CALVIN
Results on CALVIN. In the CALVIN ABC-to-D setting, which tests transfer to an unseen environment, BehaviorVLA reaches an average sequence length of 4.36. The learned behavior manifold filters scene-specific variation while preserving transferable task structure.

Real-World Evaluation

BehaviorVLA results on real-world generalization and long-horizon tasks
Results on real-world manipulation. On the GALAXEA R1 Lite platform, BehaviorVLA is evaluated on four generalization tasks and four long-horizon tasks. It achieves average success rates of 70% and 55%, respectively, and remains competitive with reduced training data, demonstrating strong data efficiency and robustness to changes in lighting, scenes, object instances, and positions.

Conclusion

In this work, we introduce BehaviorVLA, a framework designed to enhance the robustness of VLA models by learning temporally coherent behavior representations. Through the synergistic design of the Visuomotor Behavior Encoder and Phase-conditioned Behavior Decoder, our approach effectively balances global behavior abstraction with precise, phase-aligned control. Extensive experiments demonstrate that BehaviorVLA achieves state-of-the-art performance across simulation benchmarks and significantly enhances sim-to-real transfer efficiency, matching leading baselines with only 50% of the fine-tuning data. These results suggest that explicitly modeling structured behavior representations is a scalable and data-efficient path toward robust robotic manipulation in Real-World.

Demo

Place bottles in basket

Stack bowl on plate