Repository navigation
[DMD2 4/10] feat(fsdp): add a train-input hook that preprocesses train pairs before micro-batching - #338
Merged
Conversation
This was referenced Oct 24, 2026
Merged
Merged
zhihengy
marked this pull request as ready for review
October 24, 2026 00:14
zhihengy
force-pushed
the
dmd2/02b-train-input-hook
branch
from
October 24, 2026 00:17
ff83350 to
d3244bf
Compare
5 of 6 tasks
Merged
5 of 6 tasks
6 tasks done
Rockdu
force-pushed
the
dmd2/02b-train-input-hook
branch
from
November 4, 2026 19:41
d3244bf to
09820eb
Compare
Rockdu
approved these changes
Nov 5, 2026
| external_data=external_data, | ||
| ) | ||
| if self.custom_train_input_func is not None: | ||
| train_pairs = self.custom_train_input_func(loss_ctx, train_pairs) |
Collaborator
There was a problem hiding this comment.
custom_train_input_postprocess_func here is clearer
zhihengy
force-pushed
the
dmd2/02b-train-input-hook
branch
from
November 9, 2026 22:16
09820eb to
17bd1ba
Compare
zhihengy
added a commit
that referenced
this pull request
Nov 9, 2026
…tprocess Review on #338: `custom_train_input_postprocess_func` says what the hook does, a post-processing step on the converted train pairs. The flag follows the prepare hook's pattern: `--custom-train-input-postprocess-path` sets `args.custom_train_input_postprocess_path`, loaded into `self.custom_train_input_postprocess_func`. Docs and the hook test follow. Co-Authored-By: Claude Mythos 5.1 <noreply@anthropic.com>
zhihengy
force-pushed
the
dmd2/02b-train-input-hook
branch
from
November 10, 2026 03:59
fc579c4 to
22e3377
Compare
niehen6174
pushed a commit
to niehen6174/miles_diffusion
that referenced
this pull request
Nov 10, 2026
…tprocess Review on radixark#338: `custom_train_input_postprocess_func` says what the hook does, a post-processing step on the converted train pairs. The flag follows the prepare hook's pattern: `--custom-train-input-postprocess-path` sets `args.custom_train_input_postprocess_path`, loaded into `self.custom_train_input_postprocess_func`. Docs and the hook test follow. Co-Authored-By: Claude Mythos 5.1 <noreply@anthropic.com>
zhihengy
added a commit
that referenced
this pull request
Nov 10, 2026
…tprocess Review on #338: `custom_train_input_postprocess_func` says what the hook does, a post-processing step on the converted train pairs. The flag follows the prepare hook's pattern: `--custom-train-input-postprocess-path` sets `args.custom_train_input_postprocess_path`, loaded into `self.custom_train_input_postprocess_func`. Docs and the hook test follow. Co-Authored-By: Claude Mythos 5.1 <noreply@anthropic.com>
zhihengy
force-pushed
the
dmd2/02b-train-input-hook
branch
from
November 10, 2026 11:44
22e3377 to
cd303eb
Compare
zhihengy
added a commit
that referenced
this pull request
Nov 10, 2026
#338 renamed the hook to --custom-train-input-postprocess-path; the DMD2 defaults and the critic override still set custom_train_input_function_path, so --loss-type dmd2 failed in parse_args. Co-Authored-By: Claude Mythos 5.1 <noreply@anthropic.com>
4 tasks done
Rockdu
reviewed
Nov 11, 2026
| external_data=external_data, | ||
| ) | ||
| if self.custom_train_input_postprocess_func is not None: | ||
| train_pairs = self.custom_train_input_postprocess_func(loss_ctx, train_pairs) |
Collaborator
There was a problem hiding this comment.
CC cmt
[P2] Still open from the last round: the empty check at L359 runs on the raw list, so a hook that returns [] passes 0 % num_steps_per_rollout and build_microbatch_schedule (n empty steps) and the loop then does optimizer.step / lr step / global_step / EMA step / train-output hook with no forward at all. Only fp16 and nft fail loudly; bf16 GRPO/DMD2 log a zero-loss step and move on.
select_scored_pairs can't return [] today, hence P2, but the hook is documented as a filter, so please re-check if not train_pairs: raise after the hook and add the [] case to the test.
zhihengy
added a commit
that referenced
this pull request
Nov 11, 2026
…tprocess Review on #338: `custom_train_input_postprocess_func` says what the hook does, a post-processing step on the converted train pairs. The flag follows the prepare hook's pattern: `--custom-train-input-postprocess-path` sets `args.custom_train_input_postprocess_path`, loaded into `self.custom_train_input_postprocess_func`. Docs and the hook test follow. Co-Authored-By: Claude Mythos 5.1 <noreply@anthropic.com>
zhihengy
force-pushed
the
dmd2/02b-train-input-hook
branch
from
November 11, 2026 08:18
cd303eb to
77dedd4
Compare
zhihengy
added a commit
that referenced
this pull request
Nov 11, 2026
#338 renamed the hook to --custom-train-input-postprocess-path; the DMD2 defaults and the critic override still set custom_train_input_function_path, so --loss-type dmd2 failed in parse_args. Co-Authored-By: Claude Mythos 5.1 <noreply@anthropic.com>
…re micro-batching
…tprocess Review on #338: `custom_train_input_postprocess_func` says what the hook does, a post-processing step on the converted train pairs. The flag follows the prepare hook's pattern: `--custom-train-input-postprocess-path` sets `args.custom_train_input_postprocess_path`, loaded into `self.custom_train_input_postprocess_func`. Docs and the hook test follow. Co-Authored-By: Claude Mythos 5.1 <noreply@anthropic.com>
Rockdu
force-pushed
the
dmd2/02b-train-input-hook
branch
from
November 11, 2026 15:43
77dedd4 to
d8e53e5
Compare
Rockdu
pushed a commit
that referenced
this pull request
Nov 11, 2026
… from the critic group's scores (#334) * feat(dmd2): train the student with the distribution matching gradient from the critic group's scores * fix(dmd2): follow the train-input hook rename in the DMD2 defaults #338 renamed the hook to --custom-train-input-postprocess-path; the DMD2 defaults and the critic override still set custom_train_input_function_path, so --loss-type dmd2 failed in parse_args. Co-Authored-By: Claude Mythos 5.1 <noreply@anthropic.com> --------- Co-authored-by: Claude Mythos 5.1 <noreply@anthropic.com>
Rockdu
added a commit
that referenced
this pull request
Nov 11, 2026
* feat(ray): hand earlier trainer groups' outputs to later groups as external_data * fix(fsdp): box the train output so the driver's ray.get only waits `train` returned its train output directly, so the driver's ray.get on the train refs pulled every rank's output into the controller before the next group read it again from the object store. Return `Box(ray.put(output))` like the rollout shards: the driver gets a small box, the payload stays in the store until the paired rank of a later group unboxes it. Also note the train-output hook as a `--teacher-load` consumer in docs/fsdp_model_roles.md, which cli-reference.md already did. Co-Authored-By: Claude Mythos 5.1 <noreply@anthropic.com> * fix(ray): hand later trainer groups the output boxes, not the raw refs train returns Box(ray.put(output)); the driver discarded its ray.get and passed the task refs on, so the consumer's box.inner hit an ObjectRef. Keep the boxes the way the rollout shards are kept. Co-Authored-By: Claude Mythos 5.1 <noreply@anthropic.com> * feat(fsdp): add a train-input hook that preprocesses train pairs before micro-batching * refactor(fsdp): rename the train-input hook to custom_train_input_postprocess Review on #338: `custom_train_input_postprocess_func` says what the hook does, a post-processing step on the converted train pairs. The flag follows the prepare hook's pattern: `--custom-train-input-postprocess-path` sets `args.custom_train_input_postprocess_path`, loaded into `self.custom_train_input_postprocess_func`. Docs and the hook test follow. Co-Authored-By: Claude Mythos 5.1 <noreply@anthropic.com> * feat(rollout): stop the rollout at a random step for DMD2 backward simulation * feat(dmd2): train a fake-score critic group from the teacher and score the student's samples * feat(dmd2): train the student with the distribution matching gradient from the critic group's scores * fix(dmd2): follow the train-input hook rename in the DMD2 defaults #338 renamed the hook to --custom-train-input-postprocess-path; the DMD2 defaults and the critic override still set custom_train_input_function_path, so --loss-type dmd2 failed in parse_args. Co-Authored-By: Claude Mythos 5.1 <noreply@anthropic.com> * perf(ray): sleep each trainer group right after its train * feat(dmd2): add the MiniMax H3 4-step DMD2 recipe with the media and metrics to judge it * feat(dmd2): skip the engine's VAE decode of training rollouts when no reward reads the media * build: declare imageio-ffmpeg, which the eval media muxing imports miles/ray/rollout.py imports imageio_ffmpeg to mux the eval audio; the GPU image had it only as a transitive dependency and the hosted CPU suite not at all, so tests/fast/rollout/test_eval_media.py failed to import there. Co-Authored-By: Claude Mythos 5.1 <noreply@anthropic.com> * docs(h3): say the DMD2 recipe downloads the VidProM prompt list from the Hub * docs(dmd2): count the VidProM prompts after dedup * fix(dmd2): load the FastH3 reference by its Hub id on every node * refactor(args): drop the teacher consumer check Whether a hook forwards the teacher is the hook's business; the argument layer cannot tell from --loss-type and the hook paths who will read it, so the check only ever guessed. --teacher-load is now accepted under every loss type (dmd2 still requires it). * refactor(fsdp): select loss hooks from a (loss_type, role) table Stop filling the --custom-*-path arguments on behalf of nft / dmd2 and stop overriding them per trainer role: a group's prepare, formula, train-input and train-output hooks come from loss_hub.LOSS_HOOKS keyed by (loss_type, role), the way miles core's get_loss_function does. Only custom_loss reads the path flags. The critic overrides shrink to the six values that really differ (checkpoint, lr, load/save dirs, steps, batch). * fix(args): reject --custom-*-path outside --loss-type custom_loss A path given under any other loss type would be silently ignored now that the table selects the hooks, so fail at parse time instead. --------- Co-authored-by: Andy Ye <89891424+zhihengy@users.noreply.github.com> Co-authored-by: Claude Mythos 5.1 <noreply@anthropic.com>
Rockdu
added a commit
that referenced
this pull request
Nov 11, 2026
* feat(ray): hand earlier trainer groups' outputs to later groups as external_data * fix(fsdp): box the train output so the driver's ray.get only waits `train` returned its train output directly, so the driver's ray.get on the train refs pulled every rank's output into the controller before the next group read it again from the object store. Return `Box(ray.put(output))` like the rollout shards: the driver gets a small box, the payload stays in the store until the paired rank of a later group unboxes it. Also note the train-output hook as a `--teacher-load` consumer in docs/fsdp_model_roles.md, which cli-reference.md already did. Co-Authored-By: Claude Mythos 5.1 <noreply@anthropic.com> * fix(ray): hand later trainer groups the output boxes, not the raw refs train returns Box(ray.put(output)); the driver discarded its ray.get and passed the task refs on, so the consumer's box.inner hit an ObjectRef. Keep the boxes the way the rollout shards are kept. Co-Authored-By: Claude Mythos 5.1 <noreply@anthropic.com> * feat(fsdp): add a train-input hook that preprocesses train pairs before micro-batching * refactor(fsdp): rename the train-input hook to custom_train_input_postprocess Review on #338: `custom_train_input_postprocess_func` says what the hook does, a post-processing step on the converted train pairs. The flag follows the prepare hook's pattern: `--custom-train-input-postprocess-path` sets `args.custom_train_input_postprocess_path`, loaded into `self.custom_train_input_postprocess_func`. Docs and the hook test follow. Co-Authored-By: Claude Mythos 5.1 <noreply@anthropic.com> * feat(rollout): stop the rollout at a random step for DMD2 backward simulation * feat(dmd2): train a fake-score critic group from the teacher and score the student's samples * feat(dmd2): train the student with the distribution matching gradient from the critic group's scores * fix(dmd2): follow the train-input hook rename in the DMD2 defaults #338 renamed the hook to --custom-train-input-postprocess-path; the DMD2 defaults and the critic override still set custom_train_input_function_path, so --loss-type dmd2 failed in parse_args. Co-Authored-By: Claude Mythos 5.1 <noreply@anthropic.com> * perf(ray): sleep each trainer group right after its train * feat(dmd2): add the MiniMax H3 4-step DMD2 recipe with the media and metrics to judge it * feat(dmd2): skip the engine's VAE decode of training rollouts when no reward reads the media * build: declare imageio-ffmpeg, which the eval media muxing imports miles/ray/rollout.py imports imageio_ffmpeg to mux the eval audio; the GPU image had it only as a transitive dependency and the hosted CPU suite not at all, so tests/fast/rollout/test_eval_media.py failed to import there. Co-Authored-By: Claude Mythos 5.1 <noreply@anthropic.com> * docs(h3): say the DMD2 recipe downloads the VidProM prompt list from the Hub * docs(dmd2): count the VidProM prompts after dedup * fix(dmd2): load the FastH3 reference by its Hub id on every node * refactor(args): drop the teacher consumer check Whether a hook forwards the teacher is the hook's business; the argument layer cannot tell from --loss-type and the hook paths who will read it, so the check only ever guessed. --teacher-load is now accepted under every loss type (dmd2 still requires it). * refactor(fsdp): select loss hooks from a (loss_type, role) table Stop filling the --custom-*-path arguments on behalf of nft / dmd2 and stop overriding them per trainer role: a group's prepare, formula, train-input and train-output hooks come from loss_hub.LOSS_HOOKS keyed by (loss_type, role), the way miles core's get_loss_function does. Only custom_loss reads the path flags. The critic overrides shrink to the six values that really differ (checkpoint, lr, load/save dirs, steps, batch). * fix(args): reject --custom-*-path outside --loss-type custom_loss A path given under any other loss type would be silently ignored now that the table selects the hooks, so fail at parse time instead. * fix(args): reject --diffusion-recompute-old-log-prob under dmd2 DMD2 has no log-prob ratio to recompute, and once the recompute loop passes its real step index the critic's last window would run its scoring prepare and formula a second time. sft_loss and nft already reject the flag. * refactor(fsdp): pass the loop counters to the hooks as arguments microbatch_id and optim_step_idx were written onto DiffusionLossContext before every forward and read back by the hooks; the planned micro-batch prefetch would race on them. They are now required keyword arguments of every prepare and formula hook, passed by the two trunk loops; the numbering is unchanged (training from 0, the old-log-prob recompute of windows 1.. reuses their training forwards' numbers). * refactor(fsdp): hand the train-output list to the hooks as an argument The DMD2 critic formula appended its scoring inputs to a list on the context and the train-output hook read them back: a hook-to-hook channel invisible in the call graph. The trunk now owns one list per train call and passes it to the formulas and to the train-output hook. * refactor(fsdp): freeze DiffusionLossContext Nothing writes the context after construction any more: it is the rollout-level, read-only view the hooks share. eq=False keeps identity comparison; a synthesized hash would fail on the dict fields. --------- Co-authored-by: Andy Ye <89891424+zhihengy@users.noreply.github.com> Co-authored-by: Claude Mythos 5.1 <noreply@anthropic.com>
Rockdu
added a commit
that referenced
this pull request
Nov 11, 2026
…by the critic's prepare hook (#372) * feat(ray): hand earlier trainer groups' outputs to later groups as external_data * fix(fsdp): box the train output so the driver's ray.get only waits `train` returned its train output directly, so the driver's ray.get on the train refs pulled every rank's output into the controller before the next group read it again from the object store. Return `Box(ray.put(output))` like the rollout shards: the driver gets a small box, the payload stays in the store until the paired rank of a later group unboxes it. Also note the train-output hook as a `--teacher-load` consumer in docs/fsdp_model_roles.md, which cli-reference.md already did. Co-Authored-By: Claude Mythos 5.1 <noreply@anthropic.com> * fix(ray): hand later trainer groups the output boxes, not the raw refs train returns Box(ray.put(output)); the driver discarded its ray.get and passed the task refs on, so the consumer's box.inner hit an ObjectRef. Keep the boxes the way the rollout shards are kept. Co-Authored-By: Claude Mythos 5.1 <noreply@anthropic.com> * feat(fsdp): add a train-input hook that preprocesses train pairs before micro-batching * refactor(fsdp): rename the train-input hook to custom_train_input_postprocess Review on #338: `custom_train_input_postprocess_func` says what the hook does, a post-processing step on the converted train pairs. The flag follows the prepare hook's pattern: `--custom-train-input-postprocess-path` sets `args.custom_train_input_postprocess_path`, loaded into `self.custom_train_input_postprocess_func`. Docs and the hook test follow. Co-Authored-By: Claude Mythos 5.1 <noreply@anthropic.com> * feat(rollout): stop the rollout at a random step for DMD2 backward simulation * feat(dmd2): train a fake-score critic group from the teacher and score the student's samples * feat(dmd2): train the student with the distribution matching gradient from the critic group's scores * fix(dmd2): follow the train-input hook rename in the DMD2 defaults #338 renamed the hook to --custom-train-input-postprocess-path; the DMD2 defaults and the critic override still set custom_train_input_function_path, so --loss-type dmd2 failed in parse_args. Co-Authored-By: Claude Mythos 5.1 <noreply@anthropic.com> * perf(ray): sleep each trainer group right after its train * feat(dmd2): add the MiniMax H3 4-step DMD2 recipe with the media and metrics to judge it * feat(dmd2): skip the engine's VAE decode of training rollouts when no reward reads the media * build: declare imageio-ffmpeg, which the eval media muxing imports miles/ray/rollout.py imports imageio_ffmpeg to mux the eval audio; the GPU image had it only as a transitive dependency and the hosted CPU suite not at all, so tests/fast/rollout/test_eval_media.py failed to import there. Co-Authored-By: Claude Mythos 5.1 <noreply@anthropic.com> * docs(h3): say the DMD2 recipe downloads the VidProM prompt list from the Hub * docs(dmd2): count the VidProM prompts after dedup * fix(dmd2): load the FastH3 reference by its Hub id on every node * refactor(args): drop the teacher consumer check Whether a hook forwards the teacher is the hook's business; the argument layer cannot tell from --loss-type and the hook paths who will read it, so the check only ever guessed. --teacher-load is now accepted under every loss type (dmd2 still requires it). * refactor(fsdp): select loss hooks from a (loss_type, role) table Stop filling the --custom-*-path arguments on behalf of nft / dmd2 and stop overriding them per trainer role: a group's prepare, formula, train-input and train-output hooks come from loss_hub.LOSS_HOOKS keyed by (loss_type, role), the way miles core's get_loss_function does. Only custom_loss reads the path flags. The critic overrides shrink to the six values that really differ (checkpoint, lr, load/save dirs, steps, batch). * fix(args): reject --custom-*-path outside --loss-type custom_loss A path given under any other loss type would be silently ignored now that the table selects the hooks, so fail at parse time instead. * fix(args): reject --diffusion-recompute-old-log-prob under dmd2 DMD2 has no log-prob ratio to recompute, and once the recompute loop passes its real step index the critic's last window would run its scoring prepare and formula a second time. sft_loss and nft already reject the flag. * refactor(fsdp): pass the loop counters to the hooks as arguments microbatch_id and optim_step_idx were written onto DiffusionLossContext before every forward and read back by the hooks; the planned micro-batch prefetch would race on them. They are now required keyword arguments of every prepare and formula hook, passed by the two trunk loops; the numbering is unchanged (training from 0, the old-log-prob recompute of windows 1.. reuses their training forwards' numbers). * refactor(fsdp): hand the train-output list to the hooks as an argument The DMD2 critic formula appended its scoring inputs to a list on the context and the train-output hook read them back: a hook-to-hook channel invisible in the call graph. The trunk now owns one list per train call and passes it to the formulas and to the train-output hook. * refactor(fsdp): freeze DiffusionLossContext Nothing writes the context after construction any more: it is the rollout-level, read-only view the hooks share. eq=False keeps identity comparison; a synthesized hash would fail on the dict fields. * refactor(fsdp): run the DMD2 teacher forward in the actor, requested by the critic's prepare hook * fix(fsdp): run the teacher forward without the student's true_cfg_scale * refactor(dmd2): pull the teacher request out of prepare_critic_batch prepare_critic_batch did three things in one body: build the SFT batch, pick the scoring step's grid range, and collate the teacher's CFG inputs. The teacher part is its own function now, _request_teacher_forward, and the two helpers it shared with #351 are back, so the critic's prepare reads as SFT plus the last step's teacher request. No behaviour change. --------- Co-authored-by: Claude Mythos 5.1 <noreply@anthropic.com> Co-authored-by: rockdu <kangrdu@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fourth of the DMD2 stacked series, on top of #330. Adds a preprocessing step for a trainer group's train pairs. Once the rollout output has been converted into this rank's train pairs, and before they are split into micro-batches, a hook may filter, reorder or rewrite them, with the earlier groups' outputs at hand. In [DMD2 7/10], DMD2's actor uses it to keep only the samples the critic group scored.
What
--custom-train-input-postprocess-path:fn(ctx, train_pairs) -> list, the preprocessing step on this rank's train pairs._train_coreruns it once the rollout data arrives, before the micro-batch schedule. It gets the sameDiffusionLossContextas the prepare, loss and train-output hooks, withctx.external_datafrom the earlier groups ([DMD2 3/10] feat(ray): hand earlier trainer groups' outputs to later groups as external_data #330), and returns the pairs to train on, in order. The schedule, the training forwards and the train-output hook all see the returned pairs.DiffusionLossContextis now built before the schedule, so the hook can read it. Its fields are unchanged, and none of them depends on the schedule.cli-reference.mdandcustomization.md.Why
loss_type == "dmd2" and role == "actor"branch. The hook keeps the trainer algorithm-agnostic, as the prepare, loss and train-output hooks do. It is the input-side counterpart of [DMD2 3/10] feat(ray): hand earlier trainer groups' outputs to later groups as external_data #330's train-output hook.--dynamic-sampling-filter-path,--rollout-sample-filter-path,--buffer-filter-path). This hook is for choices that need an earlier group's output, which exists only on the train side.--rollout-data-postprocess-path, which post-processes the rollout data once log probs and the critic's values are in place, before training ("useful for updating loss masks"). That hook only receivesargs. The same actor also merges the PPO critic's values into the rollout data underif self.args.use_critic. Here the hook receives the pairs and the earlier groups' outputs, so neither kind of use needs a role check in the trainer.Tests
tests/fast/backends/fsdp_utils/test_train_input_hook.pyruns the production_train_coreon CPU, with the forward, metrics and rank checks replaced:external_data. It filters them and adds a field fromexternal_data, and the training forwards and the train-output hook see exactly the pairs it returns, in order, with the same ctx;Removing the hook call makes the first test fail.
The e2e suite was not run locally.
Checklist
pre-commithooks (black, isort, ruff at the pinned versions) pass on the changed filespytest tests/fastis green (522 passed, 6 skipped; 520 on [DMD2 3/10] feat(ray): hand earlier trainer groups' outputs to later groups as external_data #330)python3 train_diffusion.py --helpstill parses (the new flag is listed)--custom-train-input-postprocess-pathincli-reference.mdandcustomization.md)Stack
DMD2: distill a flow-matching teacher into a few-step student with distribution matching. Merge in order:
external_data--loss-type dmd2on the actorA GAN head and other follow-ups come after this series.