Skip to content

refactor(fsdp): select loss hooks by a (loss_type, role) table - #367

Merged
Rockdu merged 20 commits into
mainfrom
dmd2/refactor-a-hook-registry
Nov 11, 2026
Merged

Rockdu merged 20 commits into
mainfrom
dmd2/refactor-a-hook-registry

Conversation

@Rockdu

@Rockdu Rockdu commented Nov 10, 2026

Copy link
Copy Markdown
Collaborator

Stacked on #343 (DMD2 9/10): the diff here is the 3 commits above it. Follow-up to the DMD2 stack review: a trainer group's hooks should come from a table, not from --custom-*-path values filled in by the argument layer.

What

  • loss_hub.LOSS_HOOKS[(loss_type, role)] holds each built-in group's prepare / formula / train-input / train-output hooks (policy_loss, nft, sft_loss, dmd2 actor, dmd2 critic); get_loss_hooks(args, role) looks them up, and only custom_loss reads the --custom-*-path flags (an unset prepare or formula falls back to Flow-GRPO's). set_default_diffusion_args no longer fills hook paths for nft / dmd2, and the critic's compute_trainer_configs overrides shrink to the six values that differ (hf_checkpoint, lr, load, save, num_steps_per_rollout, global_batch_size).
  • --custom-*-path under any loss type other than custom_loss is rejected at parse time (it would otherwise be silently ignored). The three SFT recipe scripts drop their two now-redundant path flags.
  • The "teacher consumer" check in validate_reference_model_args is deleted: the argument layer cannot tell which hook will forward the teacher, so --teacher-load is accepted under every loss type (dmd2 still requires it).

Why

With hooks selected by (loss_type, role), --loss-type dmd2 --custom-loss-function-path x can no longer make loss_type lie, the trunk stops branching on path strings, and the stale custom_train_input_function_path class of bug (a setattr by name) cannot recur. This mirrors miles core's get_loss_function.

Validation

CPU, on this head:

check result
tests/fast (minus the Mac-only distributed / ray-socket cases) 630 passed, 3 skipped
tests/fast/utils/test_loss_type_recipes_parse.py one representative argv per loss type parses and wires the expected hooks; dmd2 actor / critic get their entries by role
tests/fast/backends/fsdp_utils/test_loss_hub_selection.py every table entry is the function the old path defaults named
pre-commit run --all-files passes

GPU (4x H200, sglang-miles head), before the GPU runs were paused:

run result
test_sd3_ocr_grpo_2xGPU (policy_loss) PASSED, 8 metric series bitwise
test_sd3_nft_pickscore_3xGPU (nft) PASSED, 10 metric series bitwise
test_h3_t2va_sft_2xGPU (sft_loss) PASSED, 2 metric series bitwise
SD3.5-Medium DMD2 smoke (K=5, batch 10, deterministic) vs the base #343 head 141 non-timing metrics, 0 differences

Behaviour ledger (intentional changes): --custom-*-path outside custom_loss is rejected instead of ignored; nft / sft_loss + --teacher-load is accepted instead of rejected. Everything else is bitwise.

Files

  • miles/backends/fsdp_utils/loss_hub/__init__.py: LossHooks, LOSS_HOOKS, get_loss_hooks
  • miles/backends/fsdp_utils/actor.py: the actor takes its hooks from the table
  • miles/utils/arguments.py: no hook-path filling; validate_loss_hook_paths; teacher consumer check removed
  • miles/utils/trainer_config.py: critic overrides shrink to the differing values
  • scripts/run_diffusion_sft_{wan22,h3_t2va,h3_ref2va}.py: drop the two SFT hook-path flags
  • docs/user-guide/{customization,cli-reference}.md, docs/fsdp_model_roles.md, docs/developer/contributor-guide.md, docs/advanced/sde-backend.md: hook selection and flag semantics
  • tests/fast/...: test_loss_hub_selection.py, test_loss_type_recipes_parse.py, test_loss_hook_path_args.py (new), test_reference_arguments.py, test_dmd2_arguments.py, test_trainer_config.py, test_train_sde_backend_args.py, test_sft_encoder_args.py updated

Checklist

  • pre-commit run --all-files passes
  • Added/updated tests for new behaviour
  • pytest -x is green (tests/fast on CPU; the GPU suites were not run after the last docs-only edits)
  • If launch flags changed, python3 train.py --help still parses (train_diffusion.py --help)
  • If a public flag was added, it appears in the CLI reference docs (no flag added; the changed semantics are in cli-reference.md)
  • If an example was added, it has a real walkthrough (no example added)

zhihengy and others added 13 commits November 11, 2026 06:53
`train` returned its train output directly, so the driver's ray.get on the
train refs pulled every rank's output into the controller before the next
group read it again from the object store. Return `Box(ray.put(output))`
like the rollout shards: the driver gets a small box, the payload stays in
the store until the paired rank of a later group unboxes it.

Also note the train-output hook as a `--teacher-load` consumer in
docs/fsdp_model_roles.md, which cli-reference.md already did.

Co-Authored-By: Claude Mythos 5.1 <noreply@anthropic.com>
train returns Box(ray.put(output)); the driver discarded its ray.get and
passed the task refs on, so the consumer's box.inner hit an ObjectRef.
Keep the boxes the way the rollout shards are kept.

Co-Authored-By: Claude Mythos 5.1 <noreply@anthropic.com>
…tprocess

Review on #338: `custom_train_input_postprocess_func` says what the hook
does, a post-processing step on the converted train pairs. The flag
follows the prepare hook's pattern: `--custom-train-input-postprocess-path`
sets `args.custom_train_input_postprocess_path`, loaded into
`self.custom_train_input_postprocess_func`. Docs and the hook test follow.

Co-Authored-By: Claude Mythos 5.1 <noreply@anthropic.com>
#338 renamed the hook to --custom-train-input-postprocess-path; the DMD2
defaults and the critic override still set custom_train_input_function_path,
so --loss-type dmd2 failed in parse_args.

Co-Authored-By: Claude Mythos 5.1 <noreply@anthropic.com>
miles/ray/rollout.py imports imageio_ffmpeg to mux the eval audio; the
GPU image had it only as a transitive dependency and the hosted CPU
suite not at all, so tests/fast/rollout/test_eval_media.py failed to
import there.

Co-Authored-By: Claude Mythos 5.1 <noreply@anthropic.com>
Whether a hook forwards the teacher is the hook's business; the argument
layer cannot tell from --loss-type and the hook paths who will read it,
so the check only ever guessed. --teacher-load is now accepted under every
loss type (dmd2 still requires it).
Stop filling the --custom-*-path arguments on behalf of nft / dmd2 and
stop overriding them per trainer role: a group's prepare, formula,
train-input and train-output hooks come from loss_hub.LOSS_HOOKS keyed by
(loss_type, role), the way miles core's get_loss_function does. Only
custom_loss reads the path flags. The critic overrides shrink to the six
values that really differ (checkpoint, lr, load/save dirs, steps, batch).
A path given under any other loss type would be silently ignored now that
the table selects the hooks, so fail at parse time instead.
@Rockdu
Rockdu force-pushed the dmd2/06-h3-recipe branch 6 times, most recently from cf79748 to 0f2b8e6 Compare November 11, 2026 15:45
Conflict: main's critic override "use_ema": False (from #333) sits where this
branch removed the four hook paths; keep both, and expect use_ema False in the
critic's args in test_trainer_config.
@Rockdu
Rockdu removed this pull request from stack #374 November 11, 2026 15:54
@Rockdu
Rockdu changed the base branch from dmd2/06-h3-recipe to main November 11, 2026 15:54
@Rockdu
Rockdu merged commit e71882b into main Nov 11, 2026
11 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants