Repository navigation
refactor(fsdp): select loss hooks by a (loss_type, role) table - #367
Merged
Merged
Conversation
6 tasks done
6 tasks done
`train` returned its train output directly, so the driver's ray.get on the train refs pulled every rank's output into the controller before the next group read it again from the object store. Return `Box(ray.put(output))` like the rollout shards: the driver gets a small box, the payload stays in the store until the paired rank of a later group unboxes it. Also note the train-output hook as a `--teacher-load` consumer in docs/fsdp_model_roles.md, which cli-reference.md already did. Co-Authored-By: Claude Mythos 5.1 <noreply@anthropic.com>
train returns Box(ray.put(output)); the driver discarded its ray.get and passed the task refs on, so the consumer's box.inner hit an ObjectRef. Keep the boxes the way the rollout shards are kept. Co-Authored-By: Claude Mythos 5.1 <noreply@anthropic.com>
…re micro-batching
…tprocess Review on #338: `custom_train_input_postprocess_func` says what the hook does, a post-processing step on the converted train pairs. The flag follows the prepare hook's pattern: `--custom-train-input-postprocess-path` sets `args.custom_train_input_postprocess_path`, loaded into `self.custom_train_input_postprocess_func`. Docs and the hook test follow. Co-Authored-By: Claude Mythos 5.1 <noreply@anthropic.com>
…e the student's samples
… from the critic group's scores
#338 renamed the hook to --custom-train-input-postprocess-path; the DMD2 defaults and the critic override still set custom_train_input_function_path, so --loss-type dmd2 failed in parse_args. Co-Authored-By: Claude Mythos 5.1 <noreply@anthropic.com>
…metrics to judge it
… reward reads the media
miles/ray/rollout.py imports imageio_ffmpeg to mux the eval audio; the GPU image had it only as a transitive dependency and the hosted CPU suite not at all, so tests/fast/rollout/test_eval_media.py failed to import there. Co-Authored-By: Claude Mythos 5.1 <noreply@anthropic.com>
zhihengy
force-pushed
the
dmd2/06-h3-recipe
branch
from
November 11, 2026 08:18
3117376 to
37695a1
Compare
Merged
5 of 6 tasks
Whether a hook forwards the teacher is the hook's business; the argument layer cannot tell from --loss-type and the hook paths who will read it, so the check only ever guessed. --teacher-load is now accepted under every loss type (dmd2 still requires it).
Stop filling the --custom-*-path arguments on behalf of nft / dmd2 and stop overriding them per trainer role: a group's prepare, formula, train-input and train-output hooks come from loss_hub.LOSS_HOOKS keyed by (loss_type, role), the way miles core's get_loss_function does. Only custom_loss reads the path flags. The critic overrides shrink to the six values that really differ (checkpoint, lr, load/save dirs, steps, batch).
A path given under any other loss type would be silently ignored now that the table selects the hooks, so fail at parse time instead.
Rockdu
force-pushed
the
dmd2/refactor-a-hook-registry
branch
from
November 11, 2026 13:03
fa6087f to
972e01e
Compare
4 tasks done
Rockdu
force-pushed
the
dmd2/06-h3-recipe
branch
6 times, most recently
from
November 11, 2026 15:45
cf79748 to
0f2b8e6
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #343 (DMD2 9/10): the diff here is the 3 commits above it. Follow-up to the DMD2 stack review: a trainer group's hooks should come from a table, not from
--custom-*-pathvalues filled in by the argument layer.What
loss_hub.LOSS_HOOKS[(loss_type, role)]holds each built-in group's prepare / formula / train-input / train-output hooks (policy_loss,nft,sft_loss,dmd2actor,dmd2critic);get_loss_hooks(args, role)looks them up, and onlycustom_lossreads the--custom-*-pathflags (an unset prepare or formula falls back to Flow-GRPO's).set_default_diffusion_argsno longer fills hook paths for nft / dmd2, and the critic'scompute_trainer_configsoverrides shrink to the six values that differ (hf_checkpoint,lr,load,save,num_steps_per_rollout,global_batch_size).--custom-*-pathunder any loss type other thancustom_lossis rejected at parse time (it would otherwise be silently ignored). The three SFT recipe scripts drop their two now-redundant path flags.validate_reference_model_argsis deleted: the argument layer cannot tell which hook will forward the teacher, so--teacher-loadis accepted under every loss type (dmd2still requires it).Why
With hooks selected by
(loss_type, role),--loss-type dmd2 --custom-loss-function-path xcan no longer makeloss_typelie, the trunk stops branching on path strings, and the stalecustom_train_input_function_pathclass of bug (a setattr by name) cannot recur. This mirrors miles core'sget_loss_function.Validation
CPU, on this head:
tests/fast(minus the Mac-only distributed / ray-socket cases)tests/fast/utils/test_loss_type_recipes_parse.pytests/fast/backends/fsdp_utils/test_loss_hub_selection.pypre-commit run --all-filesGPU (4x H200, sglang-miles head), before the GPU runs were paused:
test_sd3_ocr_grpo_2xGPU(policy_loss)test_sd3_nft_pickscore_3xGPU(nft)test_h3_t2va_sft_2xGPU(sft_loss)Behaviour ledger (intentional changes):
--custom-*-pathoutsidecustom_lossis rejected instead of ignored;nft/sft_loss+--teacher-loadis accepted instead of rejected. Everything else is bitwise.Files
miles/backends/fsdp_utils/loss_hub/__init__.py:LossHooks,LOSS_HOOKS,get_loss_hooksmiles/backends/fsdp_utils/actor.py: the actor takes its hooks from the tablemiles/utils/arguments.py: no hook-path filling;validate_loss_hook_paths; teacher consumer check removedmiles/utils/trainer_config.py: critic overrides shrink to the differing valuesscripts/run_diffusion_sft_{wan22,h3_t2va,h3_ref2va}.py: drop the two SFT hook-path flagsdocs/user-guide/{customization,cli-reference}.md,docs/fsdp_model_roles.md,docs/developer/contributor-guide.md,docs/advanced/sde-backend.md: hook selection and flag semanticstests/fast/...:test_loss_hub_selection.py,test_loss_type_recipes_parse.py,test_loss_hook_path_args.py(new),test_reference_arguments.py,test_dmd2_arguments.py,test_trainer_config.py,test_train_sde_backend_args.py,test_sft_encoder_args.pyupdatedChecklist
pre-commit run --all-filespassespytest -xis green (tests/faston CPU; the GPU suites were not run after the last docs-only edits)python3 train.py --helpstill parses (train_diffusion.py --help)cli-reference.md)