Repository navigation
perf(models): reduce discriminated streaming event validation - #2045
Open
Linxiushen wants to merge 2 commits into
Open
Linxiushen wants to merge 2 commits into
Linxiushen wants to merge 2 commits into
Conversation
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This addresses the union-decoding part of #1649: each streaming event currently attempts validation against the ordinary union before falling back to the SDK discriminator. For eligible Pydantic v2 unions, this caches a tagged-union validator built from the existing core schema and selects the event branch by its literal tag.
The shortcut requires model branches with disjoint string tags, consistent simple aliases, and schemas without validation callbacks. A bounded JSON-value check excludes iterators and custom objects. Unsupported schemas, unknown tags, invalid payloads and larger values retain the existing whole-union validation and loose-construction fallback. Pydantic v1, strict response validation and the existing fallback discriminator cache are unchanged. Event variants are derived from the schema rather than a hardcoded event list.
Measurements and tradeoffs
An offline benchmark on one Windows host with Python 3.12.14 / Pydantic 2.12.5 compared
202db72with this implementation. Two alternating baseline/candidate process pairs each ran seven measured repetitions with default deferred schemas. The stream cases use the actual stable/betaAsyncAnthropic.messages.stream().get_final_message()path throughhttpx2.MockTransport, with 3,000 text deltas and assertions on final text, usage and stop reason.Median of process medians, in microseconds per event:
The two large cases contain 1,000 nested tool-input rows, exceed the eligibility budget, and pay the guard overhead before fallback. Cold first-decode samples were 13.22/12.90 ms before and 14.40/14.67 ms after; two samples per version are insufficient for a cold-latency conclusion. These results describe a text-delta-heavy synthetic workload, not API/model latency or a universal speedup. In particular, the 28.8% result applies only to isolated decoding.
Benchmark script and all four result files (local source paths omitted) are kept outside this PR's source diff.
Validation
Linux validation for exact candidate
cb46b2apassed:./scripts/lintanduv build../scripts/test tests/test_models.py tests/test_client.py tests/test_response.py tests/test_streaming.py tests/lib/streaming/test_messages.py tests/lib/streaming/test_beta_messages.py -n 2: Python 3.10/Pydantic v2 and Python 3.14/Pydantic v2 each had 453 passed / 2 skipped; Python 3.10/Pydantic v1 had 436 passed / 19 skipped. The script's MCP v2 suite also passed 39 tests on each Python version.This picks up the union-decoding direction from withdrawn #1695, following the invitation to continue it. It does not change stream draining/aggregation (#1663) or fallback discriminator caching (#2025), and does not close the broader #1649.
The documented generator-trailer comparison was attempted, but the referenced output commit could not be fetched (
not our ref, also 404 via API). Existing lines are therefore not claimed to be handwritten; the change adds a guarded path in the mixed generated module while preserving its fallback.Release note: Reduce Pydantic v2 decoding overhead for eligible discriminated streaming events, with workload-dependent eligibility-check costs.