Skip to content

Fix 2DGS rasterization with several cameras per call - #1099

Open
nicolaihaeni wants to merge 1 commit into
nerfstudio-project:mainfrom
nicolaihaeni:fix/2dgs-multicamera-opacities
Open

nicolaihaeni wants to merge 1 commit into
nerfstudio-project:mainfrom
nicolaihaeni:fix/2dgs-multicamera-opacities

Conversation

@nicolaihaeni

Copy link
Copy Markdown

With more than one camera per call, rasterization_2dgs expands the opacities over the cameras without a copy, and the 2DGS rasterizer rejects the non-contiguous result:

RuntimeError: opacities must be contiguous

This makes the expanded opacities contiguous, as the 3DGS path does before its kernels. Checked: a 2-camera call now gives exactly the same outputs as two 1-camera calls, and gradients within ~1e-7 relative L2.

Repro (fails on main in the forward):

import torch
from gsplat import rasterization_2dgs
n, C = 20_000, 2
means = (torch.randn(n, 3) * 0.7 + torch.tensor([0, 0, 4.0])).cuda().requires_grad_()
quats = torch.nn.functional.normalize(torch.randn(n, 4), dim=-1).cuda().requires_grad_()
scales = (torch.rand(n, 3) * 0.03).cuda().requires_grad_()
opacities = torch.rand(n).cuda().requires_grad_()
colors = (torch.randn(n, 16, 3) * 0.3).cuda().requires_grad_()
viewmats = torch.eye(4).cuda()[None].repeat(C, 1, 1)
Ks = torch.tensor([[500.0, 0, 320], [0, 500.0, 240], [0, 0, 1]]).cuda()[None].repeat(C, 1, 1)
out = rasterization_2dgs(means, quats, scales, opacities, colors, viewmats, Ks, 640, 480,
                         sh_degree=3, render_mode="RGB+ED", distloss=True)

Opacities were expanded over the C cameras without a copy, and the 2DGS rasterizer rejects the
non-contiguous result ("opacities must be contiguous") for C > 1. Make it contiguous, as the 3DGS path
does before its kernels. A 2-camera call now matches two 1-camera calls exactly (outputs) and to ~1e-7
relative L2 (gradients).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant