Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,13 @@ semantic versioning for tagged releases.
where a same-named file beside the `.usdz` could stand in for a plane, so
the plugin anchors them itself. The size bounds are checked from
`ArAsset::GetSize()` before the bulk read.
- `gaussian-sog` tests: a test-only `gstest:` URI resolver plugin, built and
registered by CTest for one suite and never installed, covers the
resolver-matrix rows `ArDefaultResolver` cannot reach. The suite opens a
`meta.json` and a `.sog` by URI, directly, by reference, and metadata-only,
with every companion resolved and served by that resolver as a stream
without `GetBuffer()`, and checks that a plane missing from it stays
missing.
- CI: three `kind: workspace` cells build the composed root tree and run its
CTest suites on Windows, macOS, and Linux. Until now those suites (unit,
Python smoke and resolver, and cross-format equivalence tests) ran only in
Expand Down
2 changes: 1 addition & 1 deletion docs/guides/BUILDING.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ cmake --build --preset default
ctest --test-dir build/default --output-on-failure
```

This composes all three bundles plus the shared libraries and runs all fifteen
This composes all three bundles plus the shared libraries and runs all sixteen
tests, including `gaussian_ply_spz_equivalence` and
`gaussian_ply_sog_equivalence`. Regenerate the equivalence fixtures after
changing any mapping contract:
Expand Down
24 changes: 17 additions & 7 deletions docs/reference/RESOLVER_MATRIX.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ Legend: ✅ verified · ⚠️ verified on one platform family only · ⛔ known
| Non-ASCII characters in the path | opens. The USD-free readers open their UTF-8 path through `openstrata::gs::OpenInputFile` (`GaussianFileIO.h`), which on Windows opens the UTF-16 name instead of reinterpreting the bytes in the ANSI code page. PLY and SPZ share the helper. | ✅ | reader `TestNonAsciiPath` (both layouts and the directory loader), `gaussianCore_file_io`, and the PLY and SPZ decoder `TestNonAsciiPath`; observed on Windows (MSVC), macOS, and Linux in the `workspace-ctest-pr-*` CI cells ([WORKSPACE.md §8](../architecture/WORKSPACE.md)) |
| Package-relative path: an unbundled `meta.json` at the root or in a subdirectory of a `.usdz`, opened directly, by reference, or metadata-only; a `.sog` inside a `.usdz` | opens through `ArAsset`; companions are the sibling entries inside the package | ✅ | `case_package_relative`; reader `TestInjectedPrimaryLoader`; observed on Windows, macOS, and Linux |
| Packaged `meta.json` whose plane is missing from the package, with a valid same-named file beside the package | `GSSOG-E008`; the file beside the package is never used | ✅ | `case_package_relative` |
| URI-style identifier served by a custom resolver | read through `ArAsset`; the reader never interprets the identifier | ⬜ | reader `TestInjectedPrimaryLoader` covers the seam with a URI-like name; no real resolver plugin is tested |
| URI-style identifier served by a custom resolver (`gstest:cloud/meta.json`, `gstest:cloud.sog`), opened directly, by reference from a filesystem layer, or metadata-only | read through `ArAsset`; the reader never interprets the identifier. A bundled `.sog` asks the resolver for the archive alone | ✅ | `test_gaussian_sog_uri_resolver.py` `case_uri_unbundled`, `case_uri_reference`, `case_uri_bundled`, `case_uri_metadata_only`; reader `TestInjectedPrimaryLoader`; observed on Windows (MSVC), macOS, and Linux |

## Companion planes (unbundled layout)

Expand All @@ -73,8 +73,10 @@ Legend: ✅ verified · ⚠️ verified on one platform family only · ⛔ known
| Loader-reported failure | the loader's own diagnostic reaches the caller unchanged | ✅ | reader `TestResolverSeam` |
| Loader receives the primary path as anchor and bare plane names, in a fixed order | as stated | ✅ | reader `TestResolverSeam`, `TestInjectedCompanionLoader` |
| Companion larger than 512 MiB | `GSSOG-E016`, checked from `ArAsset::GetSize()` before allocation | ✅ | `case_oversized_companion` (sparse file); reader `TestOversizedUnbundledCompanion` ([IMPORT_LIMITS.md](IMPORT_LIMITS.md#test-coverage)) |
| Custom `ArResolver` plugin serving companions | the seam is verified with an injected loader; no real resolver plugin is tested | ⬜ | reader `TestInjectedCompanionLoader` covers the seam only |
| Resolver that serves a stream without `GetBuffer()` | read through `ArAsset::Read()` | ⬜ | needs a custom resolver |
| Custom `ArResolver` plugin serving companions | the plugin's `./<plane>`, anchored on the URI, is dispatched by Ar to the URI's own resolver, which resolves and serves each plane | ✅ | `case_uri_unbundled`, `case_uri_reference` (the resolver's log shows every plane requested under the URI's directory) |
| Companion missing from a custom resolver, with a valid same-named plane one directory up in the same store and in the current directory | `GSSOG-E008`, naming the plane and the URI; neither other file is requested | ✅ | `case_uri_missing_companion` |
| Resolver that serves a stream without `GetBuffer()` | read through `ArAsset::Read()` | ✅ | `case_uri_unbundled`: every companion is asked for a buffer, gets none, and still decodes |
| Registering a URI resolver | leaves scheme-less paths with the primary resolver; a filesystem `meta.json` never reaches the URI resolver | ✅ | `case_filesystem_unaffected` |

## Metadata-only reads

Expand All @@ -90,14 +92,22 @@ Legend: ✅ verified · ⚠️ verified on one platform family only · ⛔ known
`gaussianSog_python_resolver`. Each case copies the fixture into a fresh
temporary directory, so the layer registry cannot reuse an earlier case's
result.
- `plugins/gaussian-sog/tests/test_gaussian_sog_uri_resolver.py`: the rows
that need a resolver plugin of their own. CTest name:
`gaussianSog_python_uri_resolver`. CTest builds a test-only `gstest:` URI
resolver (`tests/uri_resolver/GsTestUriResolver.cpp`) into its own build
directory beside a generated `plugInfo.json` and registers it for this test
alone; it is never installed or packaged. It maps `gstest:<path>` to
`<GSTEST_URI_RESOLVER_ROOT>/<path>`, serves every asset as a stream only
(`GetBuffer()` returns null), and logs each `OpenAsset` and `GetBuffer`
call to `GSTEST_URI_RESOLVER_LOG`, which is how the suite asserts what the
plugin requested.
- `plugins/gaussian-sog/tests/test_gaussian_sog_reader.cpp`: the
resolver-independent seams (`TestResolverSeam`,
`TestInjectedCompanionLoader`, `TestInjectedPrimaryLoader`). It needs no
USD.

Both run in the `workspace-ctest-pr-*` CI cells on all three hosted
All three run in the `workspace-ctest-pr-*` CI cells on all three hosted
platforms, as well as in the local gate.

Next step for the ⬜ rows: add a minimal test resolver plugin (URI-style
identifiers, companions served by a custom resolver, and a stream without
`GetBuffer()`). The ⚠️ row is Windows-only by nature.
No row is ⬜ any more. The ⚠️ row is Windows-only by nature.
12 changes: 8 additions & 4 deletions docs/roadmap/current.md
Original file line number Diff line number Diff line change
Expand Up @@ -177,7 +177,7 @@ real environments and package paths were actually verified.
[coverage table](../reference/IMPORT_LIMITS.md#test-coverage). That work
found and fixed the SPZ full read loading up to 1 GiB before refusing an
oversized file.
- 🚧 Asset-resolver test matrix: the
- ✅ Asset-resolver test matrix: the
[matrix](../reference/RESOLVER_MATRIX.md) is defined and its
`ArDefaultResolver` rows are automated. That work found and fixed a missing
companion being replaced by a same-named file from the current directory or
Expand All @@ -186,11 +186,15 @@ real environments and package paths were actually verified.
Windows ANSI code page. The SOG primary asset is now read through
`ArAsset`, so a `meta.json` or `.sog` inside a `.usdz` opens, with
companions anchored inside the package rather than beside it. The
`workspace-ctest-pr-*` CI cells now run all fifteen CTest suites on
`workspace-ctest-pr-*` CI cells now run the workspace CTest suites on
Windows, macOS, and Linux as a hosted gate. Their first green run observed
the non-ASCII tests on Windows and the package-relative resolver cases on
all three. Remaining: a test resolver plugin for URI-style identifiers and
resolver-served companions.
all three. A test-only `gstest:` URI resolver plugin now covers the last
⬜ rows: a URI-style primary identifier, companions served by a custom
resolver, and assets served as a stream without `GetBuffer()`
(`gaussianSog_python_uri_resolver`). Its first hosted run passed in all
three `workspace-ctest-pr-*` cells. Only the Windows-only separator row
stays ⚠️, by nature.
- ⬜ Define the large-asset benchmark corpus.
- ⬜ Investigate specifications, fixtures, and licenses for next-format candidates.

Expand Down
43 changes: 43 additions & 0 deletions plugins/gaussian-sog/tests/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -61,3 +61,46 @@ add_test(
"${CMAKE_CURRENT_SOURCE_DIR}/test_gaussian_sog_resolver.py")
set_tests_properties(gaussianSog_python_resolver PROPERTIES
ENVIRONMENT_MODIFICATION "${_smoke_env}")

# The rows of the same matrix that ArDefaultResolver cannot reach: a URI-style
# primary identifier, companions served by a custom resolver, and an asset that
# serves a stream without GetBuffer(). A test-only `gstest:` URI resolver
# plugin serves them. It is built into its own directory beside a generated
# plugInfo.json, registered for this one test through PXR_PLUGINPATH_NAME,
# and never installed, so no other test or package sees it.
set(_uri_resolver_dir "${CMAKE_CURRENT_BINARY_DIR}/gstest-uri-resolver")
add_library(gsTestUriResolver SHARED uri_resolver/GsTestUriResolver.cpp)
target_compile_features(gsTestUriResolver PRIVATE cxx_std_17)
target_include_directories(gsTestUriResolver PRIVATE ${PXR_INCLUDE_DIRS})
set_target_properties(gsTestUriResolver PROPERTIES
PREFIX ""
LIBRARY_OUTPUT_DIRECTORY "${_uri_resolver_dir}"
RUNTIME_OUTPUT_DIRECTORY "${_uri_resolver_dir}"
ARCHIVE_OUTPUT_DIRECTORY "${_uri_resolver_dir}")
foreach(_config DEBUG RELEASE RELWITHDEBINFO MINSIZEREL)
set_target_properties(gsTestUriResolver PROPERTIES
LIBRARY_OUTPUT_DIRECTORY_${_config} "${_uri_resolver_dir}"
RUNTIME_OUTPUT_DIRECTORY_${_config} "${_uri_resolver_dir}"
ARCHIVE_OUTPUT_DIRECTORY_${_config} "${_uri_resolver_dir}")
endforeach()
if(MSVC)
target_compile_definitions(gsTestUriResolver PRIVATE NOMINMAX)
target_compile_options(gsTestUriResolver PRIVATE /utf-8)
endif()
openstrata_link_openusd(
TARGET gsTestUriResolver
COMPONENTS arch tf ar)
set(GSTEST_URI_RESOLVER_LIBRARY
"gsTestUriResolver${CMAKE_SHARED_LIBRARY_SUFFIX}")
configure_file(
"${CMAKE_CURRENT_SOURCE_DIR}/uri_resolver/plugInfo.json.in"
"${_uri_resolver_dir}/plugInfo.json"
@ONLY)

add_test(
NAME gaussianSog_python_uri_resolver
COMMAND "${Python3_EXECUTABLE}"
"${CMAKE_CURRENT_SOURCE_DIR}/test_gaussian_sog_uri_resolver.py")
set_tests_properties(gaussianSog_python_uri_resolver PROPERTIES
ENVIRONMENT_MODIFICATION
"${_smoke_env};PXR_PLUGINPATH_NAME=path_list_prepend:${_uri_resolver_dir};GSTEST_URI_RESOLVER_ROOT=set:${CMAKE_CURRENT_BINARY_DIR}/gstest-uri-store;GSTEST_URI_RESOLVER_LOG=set:${CMAKE_CURRENT_BINARY_DIR}/gstest-uri-resolver.log")
250 changes: 250 additions & 0 deletions plugins/gaussian-sog/tests/test_gaussian_sog_uri_resolver.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,250 @@
# SPDX-License-Identifier: Apache-2.0
"""Custom-resolver rows of the asset-resolver matrix for gaussian-sog.

test_gaussian_sog_resolver.py drives the stock ArDefaultResolver. This suite
covers the rows of docs/reference/RESOLVER_MATRIX.md that need a resolver
plugin of their own, using the test-only `gstest:` URI resolver
(uri_resolver/GsTestUriResolver.cpp) that CTest builds and registers for this
test alone:

- a URI-style primary identifier, opened directly, by reference, and
metadata-only, for both layouts;
- companion planes served by that resolver, anchored on the URI;
- assets that serve a stream only (GetBuffer() returns null);
- a missing companion that stays missing when a same-named file exists
elsewhere.

The resolver maps `gstest:<path>` to `<GSTEST_URI_RESOLVER_ROOT>/<path>` and
appends `open <identifier>` and `buffer <identifier>` to
GSTEST_URI_RESOLVER_LOG, which is how this suite observes what the plugin
asked the resolver for. Each case uses its own store directory, so the layer
registry cannot let one case satisfy another.
"""

import os
import pathlib
import shutil
import sys
import tempfile

from pxr import Plug, Sdf, Tf, Usd


FIXTURES = pathlib.Path(__file__).parent / "fixtures"
SOURCE = FIXTURES / "unbundled-kit-multi-degree3"
BUNDLED = FIXTURES / "kit-multi-degree3.sog"
EXPECTED_COUNT = 3
PLANES = sorted(p.name for p in SOURCE.iterdir() if p.suffix == ".webp")


def required_env(name):
value = os.environ.get(name)
assert value, f"{name} is not set; run this test through CTest"
return pathlib.Path(value)


STORE = required_env("GSTEST_URI_RESOLVER_ROOT")
LOG = required_env("GSTEST_URI_RESOLVER_LOG")


def safe_error_text(exc):
try:
return str(exc)
except Exception:
return " ".join(repr(a) for a in exc.args)


def store_unbundled(name):
"""A copy of the unbundled fixture at `gstest:<name>/`; returns the
meta.json identifier."""
target = STORE / name
target.mkdir(parents=True, exist_ok=False)
for source in SOURCE.iterdir():
shutil.copyfile(source, target / source.name)
return f"gstest:{name}/meta.json"


def reset_log():
LOG.write_text("", encoding="utf-8")


def log_lines():
if not LOG.exists():
return []
return [line for line in LOG.read_text(encoding="utf-8").splitlines()
if line]


def opened():
return {line[len("open "):] for line in log_lines()
if line.startswith("open ")}


def buffered():
return {line[len("buffer "):] for line in log_lines()
if line.startswith("buffer ")}


def gaussian_count(stage, prim_path):
prim = stage.GetPrimAtPath(prim_path)
assert prim, f"no prim at {prim_path}"
positions = prim.GetAttribute("positions").Get()
return len(positions) if positions is not None else 0


def expect_failure(open_call, code):
caught = ""
try:
stage = open_call()
except Tf.ErrorException as exc:
stage = None
caught = safe_error_text(exc)
assert stage is None, f"expected {code}, but the open succeeded"
assert code in caught, (code, caught)
return caught


def write_referencing_layer(path, asset_path):
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(
"#usda 1.0\n"
"(\n"
' defaultPrim = "Root"\n'
")\n\n"
'def Xform "Root" (\n'
f" references = @{asset_path}@\n"
")\n"
"{\n"
"}\n",
encoding="utf-8")


# --- rows of the matrix ------------------------------------------------------


def case_filesystem_unaffected(root):
"""Registering a URI resolver leaves the primary resolver in charge of
every scheme-less path: a filesystem meta.json still opens, and the URI
resolver is never asked for anything."""
target = root / "filesystem"
shutil.copytree(SOURCE, target)
reset_log()
stage = Usd.Stage.Open(str((target / "meta.json").resolve()))
assert stage, "filesystem open failed with the URI resolver registered"
assert gaussian_count(stage, "/Asset/Splat") == EXPECTED_COUNT
assert not log_lines(), log_lines()


def case_uri_unbundled():
"""A URI-style meta.json is read through ArAsset, and each companion is
the plugin's `./<plane>` anchored on the URI -- resolved and served by the
URI resolver, as a stream without GetBuffer()."""
identifier = store_unbundled("direct")
reset_log()
stage = Usd.Stage.Open(identifier)
assert stage, f"failed to open {identifier}"
assert gaussian_count(stage, "/Asset/Splat") == EXPECTED_COUNT

companions = {f"gstest:direct/{plane}" for plane in PLANES}
assert opened() == {identifier} | companions, sorted(opened())
# The plugin asked every companion for a buffer, got none, and still
# decoded the plane, so it read the stream through ArAsset::Read().
assert companions <= buffered(), sorted(buffered())


def case_uri_reference():
"""A filesystem layer references a URI; the companions are anchored on
the URI the reference resolved to, not on the referencing layer."""
identifier = store_unbundled("referenced/cloud")
with tempfile.TemporaryDirectory(prefix="gaussian-sog-uri-ref-") as tmp:
layer_path = pathlib.Path(tmp) / "shot.usda"
write_referencing_layer(layer_path, identifier)
reset_log()
stage = Usd.Stage.Open(str(layer_path))
assert stage, "URI reference failed"
assert gaussian_count(stage, "/Root/Splat") == EXPECTED_COUNT
del stage
assert {f"gstest:referenced/cloud/{plane}" for plane in PLANES} \
<= opened(), sorted(opened())


def case_uri_bundled():
"""A bundled `.sog` behind a URI has no companions: the resolver is asked
for the archive alone."""
target = STORE / "bundled"
target.mkdir(parents=True)
shutil.copyfile(BUNDLED, target / "cloud.sog")
identifier = "gstest:bundled/cloud.sog"
reset_log()
stage = Usd.Stage.Open(identifier)
assert stage, f"failed to open {identifier}"
assert gaussian_count(stage, "/Asset/Splat") == EXPECTED_COUNT
assert opened() == {identifier}, sorted(opened())


def case_uri_missing_companion():
"""A companion the resolver cannot resolve is GSSOG-E008, naming the
plane and the URI it was anchored on. A same-named valid plane at the
store root and in the current directory is never used."""
identifier = store_unbundled("missing/cloud")
(STORE / "missing" / "cloud" / "quats.webp").unlink()
shutil.copyfile(SOURCE / "quats.webp", STORE / "missing" / "quats.webp")
with tempfile.TemporaryDirectory(prefix="gaussian-sog-uri-cwd-") as tmp:
shutil.copyfile(SOURCE / "quats.webp",
pathlib.Path(tmp) / "quats.webp")
previous = os.getcwd()
os.chdir(tmp)
try:
reset_log()
caught = expect_failure(lambda: Usd.Stage.Open(identifier),
"GSSOG-E008")
finally:
os.chdir(previous)
assert "quats.webp" in caught, caught
assert identifier in caught, caught
assert "gstest:missing/quats.webp" not in opened(), sorted(opened())


def case_uri_metadata_only():
"""A metadata-only read of a URI parses meta.json alone: with every
companion gone it still succeeds, and the resolver is never asked for a
plane."""
identifier = store_unbundled("metadata-only")
for plane in PLANES:
(STORE / "metadata-only" / plane).unlink()
reset_log()
layer = Sdf.Layer.OpenAsAnonymous(identifier, metadataOnly=True)
assert layer, "URI metadata-only open failed without companions"
gs = layer.GetPrimAtPath("/Asset").customData.get("gs", {})
assert gs.get("gaussianCount") == EXPECTED_COUNT, gs
assert gs.get("shDegree") == 3, gs
assert opened() == {identifier}, sorted(opened())


def main():
plugin_path = os.environ.get("PXR_PLUGINPATH_NAME")
if plugin_path:
Plug.Registry().RegisterPlugins(plugin_path.split(os.pathsep))
assert Sdf.FileFormat.FindByExtension("sog"), \
"SOG file format is not registered"
assert Plug.Registry().GetPluginWithName("gsTestUriResolver"), \
"the gstest: URI resolver plugin is not registered"

if STORE.exists():
shutil.rmtree(STORE)
STORE.mkdir(parents=True)

with tempfile.TemporaryDirectory(prefix="gaussian-sog-uri-") as tmp:
case_filesystem_unaffected(pathlib.Path(tmp))
case_uri_unbundled()
case_uri_reference()
case_uri_bundled()
case_uri_missing_companion()
case_uri_metadata_only()

print("gaussian-sog URI resolver matrix: OK")
return 0


if __name__ == "__main__":
sys.exit(main())
Loading
Loading