Orbit-Planner

Latent world models · On-orbit autonomy

Orbit-Planner Towards Latent World Models for On-Orbit Obstacle Avoidance of Satellite Agents

Zhijian Li1,2 Chao Ren2,* Peijin Wang2 Xian Sun2

1University of Chinese Academy of Sciences 2Key Laboratory of Target Cognition and Application Technology (TCAT), Aerospace Information Research Institute, Chinese Academy of Sciences

A CubeSat agent that imagines action-conditioned futures in latent space, recovers physical state changes, and plans collision-free maneuvers online.

Paper Code Dataset
Overview of the CubeSat obstacle-avoidance task and Orbit-Planner pipeline
Explore

01 / Demo

Imagine first and move second.

Orbit-Planner continuously predicts future latent states, decodes physical states, and replans before executing the next control action.

Orbit-Planner CubeSat navigating through orbital obstacles in simulation
Closed-loop on-orbit obstacle avoidance in Isaac Sim. The side panels show the egocentric observation and predicted depth grid.

02 / Dataset

Behaviorally diverse trajectories

8,000 trajectories are collected with randomized obstacles and lighting: 7,200 for training and 800 for testing. Three behavioral modes expose the model to safe, unsafe, and exploratory motion.

Expert trajectory sample 01

Expert

Goal-directed

RRT* planning with PD tracking, wider safety margins, and low control noise.

Risky trajectory sample 02

Risky

Collision-prone

Direct goal-seeking trajectories without obstacle-aware planning.

Exploratory trajectory sample 03

Exploratory

High-diversity

Strong random action noise for broad, unstructured state-space exploration.

03 / Framework

A two-stage latent world model

Stage I learns a multimodal predictive representation from RGB, spacecraft state, and control actions. Stage II freezes the world model, probes imagined physical changes, and plans with recovered obstacle geometry.

Two-stage Orbit-Planner framework showing world-model pre-training and physics probing for planning
World-model pre-training → action-conditioned latent rollout → physics probing and online trajectory planning.

04 / Results

Robust across speed and density

Across six unseen settings with one, three, or five obstacles and two initial velocities, Orbit-Planner consistently outperforms the reactive imitation-learning baseline.

Orbit-Planner 91.7%
Diffusion Policy 55.0%
Success rates of Orbit-Planner and Diffusion Policy across six obstacle avoidance settings
Closed-loop navigation success in Isaac Sim.

05 / Citation

Build on Orbit-Planner

Orbit-Planner is available on arXiv as arXiv:2608.16651. 

@article{li2026orbitplanner,
  title   = {Orbit-Planner: Towards Latent World Models for On-Orbit Obstacle Avoidance of Satellite Agents},
  author  = {Li, Zhijian and Ren, Chao and Wang, Peijin and Sun, Xian},
  journal = {arXiv preprint arXiv:2608.16651},
  year    = {2026}
}