Skip to content

Normalize targeted TV over broadcast mask coverage - #1091

Open
Afloat16 wants to merge 1 commit into
nerfstudio-project:mainfrom
Afloat16:fix/tv-broadcast-normalization
Open

Afloat16 wants to merge 1 commit into
nerfstudio-project:mainfrom
Afloat16:fix/tv-broadcast-normalization

Conversation

@Afloat16

Copy link
Copy Markdown

compute_tv_loss_targeted broadcasts its mask in the loss numerator but counts the unbroadcast mask in its denominator. Sharing one mask across repeated images scales the loss with batch size, and singleton spatial dimensions can produce empty cropped masks.

Broadcast the mask to the image before slicing and count those same elements in each directional denominator. This also avoids an extra channel factor for masks that already cover every channel. The unmasked path and binary-mask contract are unchanged.

Validation on Python 3.12.14 / PyTorch 2.8.0+cpu, with native full-package imports and the repository pytest configuration:

  • Unchanged main: all 5 new regressions fail; this branch: 5 passed.
  • Existing occlusion/regularizer tests plus regressions: 20 passed.
  • Tests compare values and image gradients against an independently expanded-mask expression for shared batch, singleton spatial and channel-complete masks, and check invariance under batch repetition.
  • Project-pinned Black 22.3.0 changed-file formatting, critical Ruff checks, Python compilation and git diff --check pass.

This concerns targeted TV in gsplat.regularizers, separate from the masked-mean change in #1074. CUDA, full training/rendering, image-quality and runtime/memory benchmarks were not run.

`compute_tv_loss_targeted` accepts broadcastable masks, but counts elements before broadcasting them to the image. A mask shared across three images triples the loss for identical images; singleton spatial dimensions can also produce empty cropped masks.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant