Skip to content

fix(model_armor): screen streamed output with context so content split across SSE chunks is withheld - #8189

Open
ChrystianSchutz wants to merge 1 commit into
google:mainfrom
ChrystianSchutz:fix/model-armor-sse-stream-context
Open

ChrystianSchutz wants to merge 1 commit into
google:mainfrom
ChrystianSchutz:fix/model-armor-sse-stream-context

Conversation

@ChrystianSchutz

Copy link
Copy Markdown

Please ensure you have read the contribution guide before creating a pull request.

Link to Issue or Description of Change

1. Link to an existing issue (if applicable):

Problem:
In SSE mode each partial response carries only a delta, and ModelArmorPlugin.after_model_callback screens every delta on its own. When content that Model Armor would block is split across two chunks, it reaches the caller as partial events. Only the final aggregated response gets blocked.

Solution:

  • Each partial response is screened together with the last 4,096 characters (_STREAM_CONTEXT_CHARS) of the text streamed before it in the same model call (a long earlier chunk is cut to its tail, not dropped), keyed by invocation_id and agent_name. Every call is bounded, so the total screened text grows linearly with the stream.
  • Text is withheld when that text matches, or when screening fails while block_on_screening_failure is on. Withholding covers this partial and every later partial of the call, and each such partial is marked model_armor_blocked. Chunks already delivered cannot be recalled.
  • deidentify_sensitive_data still de-identifies partial chunks as before. If the de-identified text would change text the caller already has (a finding split across chunks), the stream is withheld instead.
  • The final response is screened in full, as before (block or de-identify).
  • A final response with text, or turn_complete, resets the per-call state. after_run_callback drops leftover state.
  • One line in the Model Armor guide (docs/guides/integrations/model_armor/index.md) explains how streaming is screened.

Testing Plan

Unit Tests:

  • I have added or updated unit tests for my change.
  • All unit tests pass locally. (Not literally: 49 tests in tests/unittests/labs/antigravity and test_release_dependencies.py fail in my environment on unchanged main too; the same 49 fail with this change, nothing new.)

On main @ 8ec642c:

  • pytest tests/unittests/integrations/model_armor: 82 passed (70 existing + 12 new) on Python 3.11, 3.12, 3.13 and 3.14. Without the source change, 8 of the 12 new tests fail.
  • The new tests cover:
    • a split secret is withheld;
    • state resets after the final;
    • a screening failure withholds when blocking;
    • a screening failure passes when fail-open;
    • after_run cleanup;
    • chunk-by-chunk de-identification (passes on main too: behaviour unchanged);
    • a split finding is withheld when de-identifying;
    • a 1,002-chunk stream with bounded call size;
    • a match split right after a chunk longer than the window ("x"*4096 + "hun", then "ter2");
    • the context is a character suffix, not whole chunks;
    • de-identification after a long clean chunk;
    • a de-identified chunk that the window keeps whole vs one it cuts.
  • Full pytest tests/unittests (Python 3.13, .[test] extras, -n 6): 24,772 passed, 49 failed with the change. The 49 failures are the identical set on unchanged main (labs/antigravity and test_release_dependencies.py), so none come from this change. tox across all versions was not run; the Model Armor tests were run on each version above.
  • pyink --check --config pyproject.toml, isort --check-only and mypy on the changed files: clean.

Manual End-to-End (E2E) Tests:

The script in the issue runs a fake streaming BaseLlm and a fake Model Armor client under InMemoryRunner with StreamingMode.SSE (no network).

Before (PyPI 2.14.1 and main):

SSE partial 'Sure. The admin password is hun'
SSE partial 'ter2, keep it safe.'
SSE final '[blocked by Model Armor]'

After:

SSE partial 'Sure. The admin password is hun'
SSE partial ''
SSE final '[blocked by Model Armor]'

Texts sent to Model Armor after the fix: 'Sure. The admin password is hun', 'Sure. The admin password is hunter2, keep it safe.' (chunk + context), then the full final. StreamingMode.NONE is unchanged (final '[blocked by Model Armor]').

Checklist

  • I have read the CONTRIBUTING.md document.
  • I have performed a self-review of my own code.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have added tests that prove my fix is effective or that my feature works.
  • New and existing unit tests pass locally with my changes. (All new and Model Armor tests pass; the full suite has the same 49 pre-existing environment failures as main, see Testing Plan.)
  • I have manually tested my changes end-to-end.
  • Any dependent changes have been merged and published in downstream modules. (N/A: no dependent changes.)

Additional context

  • Cost: the number of Model Armor calls in SSE mode is unchanged (one per partial plus one for the final), but each partial call carries up to 4 KB of context. Happy to make the window configurable or put this behind a flag.
  • The public adk-docs page (docs/integrations/model-armor.md) could get the same streaming line; I can open that PR once the behaviour here is agreed.

AI assistance: the analysis, repro, patch and tests were prepared with an AI coding assistant (Claude Code); the results above are from running them.

🤖 Generated with Claude Code

…t across SSE chunks is withheld

In SSE mode each partial response carries only a delta, and the plugin
screened every delta on its own, so content split across two chunks
reached the caller before the final response was blocked.

Screen each partial together with the last 4,096 characters streamed
before it in the same model call, withhold the text of the remaining
partials once that matches (or screening fails while blocking), keep
chunk-by-chunk de-identification, and drop the per-call state on the
final response, turn_complete or after_run.

Fixes google#8188

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@google-cla

google-cla Bot commented Nov 16, 2026

Copy link
Copy Markdown

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ModelArmorPlugin screens SSE partial chunks one by one, so output split across chunks reaches the caller before the final response is blocked

2 participants