Robot Native Engine

Robots are not plugins. RNE is a Rust robot-native game engine for deterministic simulation, embodied AI, synthetic sensors, and policy evaluation.

RNE combines a headless, replayable simulation core with real wgpu rendering. Worlds hold robot, sensor, actuator, agent, and episode entities; simulation needs no renderer, and ROS 2 is an optional adapter, not a core dependency.

Real simulation showcase

Every frame below is rendered by wgpu from deterministic simulation or pinned camera state; gates and regeneration commands are in README showcase acceptance.

Real indoor 3DGS · mobile manipulation A real photo-derived interior (Voxel51 Dr Johnson 3DGS) bound to real cameras and landmarks by a fail-closed validation fixture. A SCARA arm on a lift mast closes its pads on the block, which is held where they caught it (the pads stay level with it, within 8 mm of its faces, measured every step), lifted 0.501 m, carried 1.596 m and placed within 0.061 m; live wrist RGB-D self-masks the robot and drives the final approach without payload truth. metadata · source
OpenArm v2 · bimanual control 18-axis typed feedback, an IK-solved pick / handoff / place cycle gated on real fingertip contact, and exact Rapier replay over 1,400 steps. metadata · source	Factory inspection Parts ride a belt conveyor, a kinematic belt carrying free dynamic parts by friction, and stop in front of the G1. It lowers its right hand onto each part until the simulation reports contact, rests it there and lifts away; a lamp turns green only on that contact. The fingertips meet each part within about 1 cm of its top-face centre without moving it. metadata
Go2 · turns a door knob The Go2 grips the door's round knob with both fingers, turns it until the latch lets go, pushes the 6 kg door open, walks through, and pushes it shut until the latch catches; the hand is the only part of the robot that touches the door. It steers only on its own Livox Mid-360 localization (0.06 m RMS), with the sensor model fitted to real Go2 recordings. metadata · details	PLATEAU UAV · RGB-D flight A detailed multirotor flies 56.0 m over imported PLATEAU LOD1 buildings with textured facades, 2.55 m minimum building clearance, zero collisions, and synchronized onboard RGB-D. metadata · source

Highlights

Area	Included	Docs
City simulation	PLATEAU import, traffic, LiDAR, RGB-D, OSM HUD	docs, ex. 46–47
Vehicle dynamics	Bicycle/Ackermann, tire saturation, suspension, road excitation	docs, ex. 49–51
Quadruped locomotion	Official Go2, model-based trot and heading control, torque control, disturbances	docs, ex. 52–65, 126
Humanoid locomotion	Official G1 23-DoF, balance, learned stride, CEM eval	docs, ex. 39, 63, 67, 68
Manipulation	PBR/3DGS mobile manipulator, friction grasp, Dex3 hands	docs, ex. 32, 40–42, 89
Deformables	XPBD cable and cloth, deterministic headless replay	ex. 43–45
More demos	Localization, native planning/dynamics/legged/WBC/OC, Go2 jump	docs

Independent validation wanted

RNE remains below 1.0 until outside projects reproduce tasks and pass the shipped conformance kits (native bundles include the tools; no source checkout needed).

Only v0.4.0 official assets qualify; if that page lacks the native archives and SHA256SUMS yet, prepare the checklist but do not open an evidence issue (v0.1.0 does not qualify).

• External project reproduction + Failure Capsule
• Installed flagship reproduction
• Third-party plugin conformance
• External physics/simulator/hardware/accelerator conformance

See the external evidence intake guide. Opening an issue is only the start of review: it does not imply acceptance; in-repo reference implementations do not count as independent evidence.

Vehicle dynamics at the grip limit

Same controller, two plants: the dynamic car's trail turns red once the front axle saturates. No-slip follows the line; the dynamic car runs wide past tire grip. Vehicle dynamics.

Four cars race three laps on the tire-limited dynamic bicycle model, from a grid in reverse order of pace. Each follows a minimum-curvature racing line at the speed its own grip and power allow; a faster car catches a slower one, takes its tow down the straight and passes beside it, and the car behind always leaves room. Four passes, the fastest car wins, the closest two cars came was 2.47 m centre to centre, and no car left the track. The cars are built from their parts (wings, sidepods, halo, steered and spinning wheels); the circuit uses CC0 Poly Haven asphalt, grass, tyres and barriers. source

Four racing drones fly three laps of an eight-gate course on the MultirotorFlight model (position loop, velocity loop, tilt-limited acceleration), each chasing a point on the course at the speed its own curvature profile allows, in its own lane of the gate opening. A pursuit start sends the slowest off first and the fastest last; all six pairs change places on the final lap and the fastest wins. All 99 gate crossings are inside the opening, the tightest with 0.56 m between props and frame, and the closest two drones came was 0.80 m. The quads are built from their parts (carbon X frame, motor bells, spinning three-blade props, tilted FPV camera, battery, antennas, LED strips). source

Navigation, SLAM, and multi-robot

Nothing but the walls and the desk is on the orange AGV's map. The second AGV, the pedestrian and the hand truck are bodies in the physics world, so its LiDAR hits them: red dots are the returns the map does not explain, and yellow rings are the tracks ObstacleTracker makes of them. The route (magenta) is plan_path over a costmap with each track written in, moving ones swept 2 s ahead, and replanned five times a second. Pure pursuit drives the wheels, and avoid_velocities gives way to the moving tracks. The AGV swings out past the second AGV (footprints at least 0.32 m apart), stops for 2.2 s while the pedestrian crosses (0.23 m clear), and docks 0.05 m from the goal. The second AGV keeps its lane and never needs to brake. It has no simulated sensors: it gets the orange AGV's pose over the fleet link and the pedestrian's true position.

Lifelong SLAM: the robot maps this warehouse on four days, starting somewhere new each time with 0.6 to 1.6 m of odometry drift over its loop, while pallets arrive, leave and move. Each day it recognizes where it is in the lifelong map, registers every keyframe against it, and the map on the board updates: red where a pallet left, green where one arrived. Every pallet that changed was detected on every day (9 of 9), and 798 of the 828 cells flagged as changed lie on a pallet that really changed. The map stays within 3 cm of the building after rigid alignment, its frame holds where the first day put it, and pruning keeps the pose graph at the first day plus the latest. The cyan rays, the red outline and the yellow returns are the scan the robot's LiDAR returns at that moment. The board and floor marks are drawn from the lifelong map itself. Lifelong mapping, source.

rne_nav/rne_slam: deterministic, ROS-free costmaps, a transform tree, A*/DWA/pure-pursuit, multi-robot avoidance, an EKF, 3D ICP, and online 2D SLAM with loop closure and AMCL (a ROS 2 adapter maps to Nav2). Details: Navigation, SLAM.

Logistics across floors

Goods-in to delivery. A forklift AGV takes a case off a stand, calls the lift, rides up with the load and sets it down on the floor above. The mast is a prismatic joint with a position servo and the case is an ordinary dynamic body throughout: it moves 0.038 m on the tines across the whole carry. source · more lift demos in Multi-floor navigation and More demos

Go2 locomotion

With an arm on its back, the Go2 opens a latched swing door by its knob, walks through, and shuts it behind itself. It closes a two-finger gripper on the round knob, holds it only while both fingers are measured on it, and turns it with its wrist until the latch lets go; then it pushes the 6 kg door open to 86° and back shut until the latch catches, and no part of the robot but the hand ever touches the door. Every walking command comes from its own Livox Mid-360 localization (0.058 m RMS), with the sensor model fitted to real Go2 recordings. docs/GO2_DOOR.md · source

The pieces underneath, each with its own GIF in the docs:

• Model-based trot on unitree_go2_jump, the Go2 with its feet attached: 500 Hz joint torques, stance tau = -J^T f, Raibert swing; held headings stay within 0.04 rad RMS over 8.5 m. docs/GO2_LOCOMOTION.md
• Livox Mid-360 fitted to real Go2 recordings: non-repetitive pattern to 0.13° on a held-out recording, measured rig occlusion and near-range blanking; walking, every floor-facing band's no-return fraction stays between the recordings'. docs/LIVOX_MID360.md
• Navigation with no map given: online SLAM on the Mid-360, A* through unexplored space, both rooms reached with 0.042 m RMS localization while leg odometry alone drifts 7.1 m. docs/LIVOX_MID360.md

G1 locomotion

A full backflip in native RoboSim/Rapier: 62.5 µs step, 21 convex body colliders with self-collision, bounded joint effort and gravity only — no imposed base trajectory, no root wrench, no RL. It lands on its feet and is still standing 15 s later. Peak joint speed is 1.039x the URDF rating, under the unchanged 1.05 gate.

The GIF replays a recorded native rollout — the renderer applies the recorded poses and takes zero physics ticks, and the model and recording hashes are checked before the first frame. The controller comes from a parameter search, not a learned policy. This is a simulator result; hardware is unvalidated. Details and the full evidence trail: docs/G1_CONTACT_BACKFLIP.md.

Walking is a separate and much weaker claim. Example 68 holds the v0.3 sustained envelope upright for 3000 ticks / 50 s, turning the commanded way the whole time (+1.6 / −2.2 rad) without holding its heading target, but that walk goes backwards: the knees bend toward the way the robot faces while the body travels the other way, because the search that found its torque overlay scored distance without a direction. Measured along the facing, its 8 s windows are -0.16 m and -0.22 m.

UnitreeG1TorqueOverlay::FORWARD_STRIDE walks forwards, straight and without turning: +0.14 to +0.16 m per window, travel within a mean 0.20 rad of the facing. It holds only under the exact conditions it was trained in. A constant 1e-6 N·m of extra hip-yaw torque tips it over, so it cannot yet be steered or stopped. This is a stability-and-direction claim, not a navigation one. Details: docs/G1_LOCOMOTION.md.

Quickstart

git clone https://github.com/rsasaki0109/RobotNativeEngine.git
cd RobotNativeEngine
cargo run -p hello_world --example 00_hello_world
cargo run -p falling_cube --example 01_falling_cube

For a complete local validation, run cargo run -p xtask -- ci (the long smoke gate splits into manipulator/locomotion/assets/media partitions, e.g. cargo run -p xtask -- ci-smoke media). The headless asset CLI, replay, and determinism-check commands, and the full example index, are in examples/README.md.

To poke at a robot by hand, the Robot workbench puts joint sliders, a floor/obstacle editor, and RGB/depth/LiDAR views for a URDF or MJCF model in one browser window: cargo run --release --locked -p robot_workbench.

Independent integrations

The native release archive includes a one-command installed product proof:

./bin/rne-flagship-proof flagship-proof --cross-backend \
  --measure-on "lab-workstation-a" --verify-installed-bundle .

It runs the same indoor TaskSpec through Rapier and bundled MuJoCo, verifies both replays plus the Failure Capsule against SHA256SUMS, and writes a SHA-256-bound report with no source checkout, renderer, or network needed. Details: flagship validation.

Third-party plugins, physics backends, adapters, and external task reproductions go through the fixed external evidence intake; submission never implies acceptance.

Architecture

The workspace is split by responsibility:

• rne_core/rne_math/rne_ecs/rne_world/rne_robot/rne_sensor/rne_ai/rne_data: schedules, ECS, spatial math, entity/robot control, sensors, learning interfaces, typed data streams.
• rne_planning/rne_dynamics/rne_legged/rne_wbc: backend-neutral joint-space planning, articulated dynamics, legged templates, whole-body control.
• rne_physics/rne_physics_rapier and rne_render/rne_render_wgpu: backend-neutral traits plus the Rapier and wgpu implementations.
• rne_asset/rne_plugin/rne_traffic: assets, plugin interfaces, backend-neutral traffic.
• adapters/ros2: ROS 2 integration; core crates remain ROS 2-free.

Determinism and testing

Simulation uses SimClock, explicit seeds, stable entity ordering, and replay digests; headless examples/tests never initialize a renderer; public APIs use explicit units (_m, _rad, _s, _hz); physics backends never leak engine-specific handles through core traits.

Standard checks:

cargo fmt --all
cargo clippy --workspace --all-targets -- -D warnings
cargo test --workspace
cargo run -p xtask -- ci-headless
cargo run --locked -p xtask -- flagship
cargo run -p xtask -- ci

Python and ROS 2 adapters

The Python adapter exposes native environments for policy experiments:

python3 -m venv .venv
.venv/bin/pip install maturin
.venv/bin/maturin develop -m crates/rne_py/Cargo.toml
.venv/bin/python examples/04_python_policy/run.py

ROS 2 is optional, isolated under adapters/ros2; see the bridge README for setup.

Documentation

• Architecture · Roadmap · OSS parity · Plugin SDK · Browser viewer
• Conformance/readiness: physics · hardware · simulator · OpenArm cross-sim · compat corpus · support · 1.0 readiness · flagship validation
• Indoor autonomy: multi-floor navigation
• Physics: height field terrain · collision bake · arm position control
• Locomotion: G1/workbench/splat bg · Go2 · frontier plan · sensors
• Case studies: Tsukuba/full/3DGS bg · SSL 2v2/adapter
• More demos · Examples · Changelog

License

Licensed under either the Apache License 2.0 or the MIT license, at your option.