Video2DoorTraversal

Push Door Traversal via Simulated Door Twins

Xincheng Tang¹Yiji Chen¹Youhan Xie¹ Wanyu Li¹Zhengjie Shu¹Lai Jiang¹ Wenkang Hu¹Yitong Li¹Jinchuang Zhang³ Xibin Song²Ruigang Yang^1,3†

¹ Shanghai Jiao Tong University ² Shandong University ³ NeoWa Robotics

^† Corresponding author

E-mail: tangxincheng@sjtu.edu.cn, ryang2@sjtu.edu.cn

arXiv Paper Code Coming soon

Single RGB Video· One-Shot Real-to-Sim-to-Real

96.57%average real-world success

80.95%zero-shot success

13scomplete traversal

Explore the pipeline

Overview

From one RGB video to real-world door traversal.

Video2DoorTraversal reconstructs an instance-aligned articulated door twin, generates physically executable experience in simulation, and learns closed-loop whole-body control for the real robot.

System overviewSingle RGB Video → DoorTwin → Real-World Deployment

Abstract

Door opening and traversal is a long-horizon loco-manipulation task that requires precise handle interaction and coordinated base–arm control. We present Video2DoorTraversal, a single-video real-to-sim-to-real framework for wheel-legged mobile manipulators. Given one RGB video of a real door, DoorTwin reconstructs an instance-aligned, articulated, and simulation-ready door twin with realistic geometry and appearance. A simulation-in-the-loop agent converts the recovered articulation into a parameterized skill program and iteratively refines failed rollouts to generate physically executable demonstrations. These demonstrations are used to train ArticuACT, a dual-depth policy that predicts coordinated base, arm, and gripper commands using robot-centric camera conditioning and interaction-aware supervision. With all perception and policy inference running onboard, the system achieves a 96.57% average success rate across five real doors and an 80.95% zero-shot success rate on structurally similar unseen doors, while completing the full approach, opening, and traversal sequence in approximately 13 s on average.

Method

A compact real-to-sim-to-real pipeline.

One handheld RGB video grounds the door instance. Simulation supplies the robot experience; onboard sensing closes the loop at deployment.

DoorTwin

One video, many simulation-ready articulated doors.

DoorTwin recovers task-relevant geometry, articulation, handle placement, and appearance while preserving the observed door instance.

OverviewA gallery of generated articulated DoorTwin assets

Comparison with Articulated-Asset Baselines DoorTwin (Ours) is compared with Articulate-Anything, PhysX-Omni, and Articraft. The comparison focuses on instance-level fidelity in global door proportions, panel-relative handle placement, articulation, and visual appearance across real-world door instances.

Comparison of Geometry and Appearance Quality

Method	Geometry		Appearance			VLM Score ↑
Method	SS ↑	mIoU ↑	PSNR ↑	SSIM ↑	LPIPS ↓	VLM Score ↑
PhysX-Omni	65.75	0.635	17.51	0.726	0.484	11.54
Articraft	89.03	0.880	16.64	0.702	0.470	28.75
Articulate Anything	86.62	0.831	16.03	0.667	0.565	11.59
DoorTwin (Ours)	94.95	0.972	18.53	0.692	0.408	56.74

Sim Agentic Trajectory Generation

Generate, execute, diagnose, refine.

The simulation-in-the-loop expert turns the recovered articulation into executable traversal programs, then collects successful demonstrations under domain randomization.

Single rolloutComplete skill execution in one environment

Parallel collectionDomain-randomized expert trajectory generation

Real-World Experiments

Robust traversal across diverse door types.

The same system handles distinct geometry and mechanisms using only onboard observations during execution.

169 / 175successful trials across five real doors

96.57%average real-world success rate

13 saverage end-to-end traversal time

Diverse Door Types

Different handles. Different appearances. Complete traversal.

Lever handle

Doorknob

Vertical cylindrical handle

Vertical square handle

Repeated-Trial Reliability

10 / 10

Ten consecutive successful real-world traversals.

The full sequence is repeated from approach and handle interaction through door opening and doorway traversal, demonstrating stable closed-loop execution.

Success 10 / 10Repeated one-shot real-world rollouts

Zero-Shot Cross-Door Generalization

80.95%

Transfer to structurally similar unseen doors.

A policy trained from one reconstructed DoorTwin directly traverses unseen doors without additional trajectory generation or real-world fine-tuning.

Zero-shot transferNo retraining on the target door

Reference

BibTeX

@article{tang2026video2doortraversal,
  title   = {Video2DoorTraversal: Push Door Traversal via
             Simulated Door Twins},
  author  = {Tang, Xincheng and Chen, Yiji and Xie, Youhan and
             Li, Wanyu and Shu, Zhengjie and Jiang, Lai and
             Hu, Wenkang and Li, Yitong and Zhang, Jinchuang and
             Song, Xibin and Yang, Ruigang},
  journal = {arXiv preprint},
  year    = {2026}
}

The arXiv identifier and publication venue will be added when available.