Skip to content

feat(pi05): add native SFT support and optional LIBERO workflow - #352

Draft
zhangwt20011015 wants to merge 12 commits into
radixark:mainfrom
zhangwt20011015:codex/pi05-libero-sft
Draft

zhangwt20011015 wants to merge 12 commits into
radixark:mainfrom
zhangwt20011015:codex/pi05-libero-sft

Conversation

@zhangwt20011015

@zhangwt20011015 zhangwt20011015 commented Nov 3, 2026 •

Copy link
Copy Markdown

Summary

Add native pi05 full-parameter supervised fine-tuning to Miles, initialized from the official pi05_base checkpoint, with an optional LIBERO workflow for data preparation and simulator evaluation through SGLang.

  • Implement the model package through the shared MilesModelBackend, action flow-matching batch preparation, the existing SFT loss, FSDP2 and gradient checkpointing. Training uses unmodified Transformers; OpenPI model patches are not required.
  • Reuse DCP checkpoint saving and add full-model export with bundle metadata and normalization assets. Explicit checkpoint resume now fails on missing or invalid required state instead of silently starting fresh; this changes shared FSDP resume behavior.
  • Make serving imports lazy for train-only execution and let model families declare their SFT supervision streams.
  • Add an optional plugins/libero package for base conversion, pinned demonstration downloads, normalization, training launch, matching inference transforms and resumable simulator evaluation. Training/simulation and SGLang serving use separate environments.
  • Document installation, training, resume, export and evaluation. Dataset-specific behavior stays in the optional plugin.

Validation

  • Previously completed a training run through step 2, resumed through step 4, exported the checkpoint, loaded it in SGLang and completed a LIBERO episode. The episode did not succeed; this validates execution, not convergence.
  • Re-evaluated one reference episode with an existing step-1000 checkpoint: success at 84 simulator steps and 17 inference requests, matching the earlier record. This is a single-episode regression, not a suite success-rate claim.
  • Passed 10 targeted CPU regression cases for evaluation resume and data-source restore, including real DCP weight save/load. Evaluation cases use simulator/HTTP/video stand-ins to check control flow, not policy quality. Completed episode videos are preserved, interrupted episodes can be retried, and incompatible resume requests are rejected. Default cursor validation lives in the default data source; custom data sources retain their own save/load format. Python syntax, shell syntax and diff whitespace checks pass.

Scope and limitations

FP32 full-parameter action SFT only. LoRA, mixed precision, sequence parallelism and online serving weight updates are not supported by this integration. A complete 30,000-step training curve and multi-node validation remain outstanding; the documented recipe remains NV. Runtime artifacts, datasets, weights and validation logs are not included.

@Rockdu Rockdu self-assigned this Nov 3, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants