Skip to content

ModelArmorPlugin screens SSE partial chunks one by one, so output split across chunks reaches the caller before the final response is blocked #8188

Open

Description

@ChrystianSchutz

🔴 Required Information

Describe the Bug:

With RunConfig(streaming_mode=StreamingMode.SSE), ModelArmorPlugin.after_model_callback runs on every partial response, and each partial carries only its delta (src/google/adk/integrations/model_armor/_plugin.py, L139-166 on 8ec642c). Each delta is screened on its own. Every delta that does not match by itself is yielded to the caller: flows/llm_flows/core/_model_call.py L297-306 yields each llm_response after the callback. Only the final, aggregated non-partial response is screened as a whole.

So when the content Model Armor would block spans two chunks, the caller has already received all of it as partial events. The final event is then replaced with output_blocked_message, and the session stores the blocked text, which makes the run look screened.

The guide says the response template "Screens model responses before delivering to the user". Its Limitations list live streaming tools and long-running tools, but not SSE text streaming.

Steps to Reproduce:

  1. pip install google-adk==2.14.1 google-cloud-modelarmor
  2. Run the script below. It needs no network: a fake streaming BaseLlm stands in for the model, and a fake Model Armor client reports MATCH_FOUND when the text contains hunter2.
  3. Compare StreamingMode.NONE with StreamingMode.SSE.
import asyncio
from unittest import mock
from google.adk.agents.llm_agent import LlmAgent
from google.adk.agents.run_config import RunConfig, StreamingMode
from google.adk.integrations.model_armor import ModelArmorConfig, ModelArmorPlugin
from google.adk.models.base_llm import BaseLlm
from google.adk.models.llm_response import LlmResponse
from google.adk.runners import InMemoryRunner
from google.cloud import modelarmor_v1
from google.genai import types

CHUNKS = ["Sure. The admin password is hun", "ter2, keep it safe."]

class FakeStreamingLlm(BaseLlm):
    model: str = "fake-stream"
    async def generate_content_async(self, llm_request, stream=False):
        if stream:
            for c in CHUNKS:
                yield LlmResponse(content=types.Content(role="model", parts=[types.Part(text=c)]), partial=True)
        yield LlmResponse(content=types.Content(role="model", parts=[types.Part(text="".join(CHUNKS))]),
                          partial=False, turn_complete=True)

async def sanitize_model_response(request):
    found = "hunter2" in request.model_response_data.text
    state = modelarmor_v1.FilterMatchState.MATCH_FOUND if found else modelarmor_v1.FilterMatchState.NO_MATCH_FOUND
    return modelarmor_v1.SanitizeModelResponseResponse(sanitization_result=modelarmor_v1.SanitizationResult(
        filter_match_state=state, invocation_result=modelarmor_v1.InvocationResult.SUCCESS))

async def run(mode):
    client = mock.Mock(sanitize_model_response=mock.AsyncMock(side_effect=sanitize_model_response))
    plugin = ModelArmorPlugin(config=ModelArmorConfig(
        response_template_name="projects/p/locations/us-central1/templates/t",
        output_blocked_message="[blocked by Model Armor]"), client=client)
    runner = InMemoryRunner(agent=LlmAgent(name="a", model=FakeStreamingLlm()), app_name="app", plugins=[plugin])
    s = await runner.session_service.create_session(app_name="app", user_id="u")
    async for ev in runner.run_async(user_id="u", session_id=s.id,
            new_message=types.Content(role="user", parts=[types.Part(text="admin password?")]),
            run_config=RunConfig(streaming_mode=mode)):
        print(mode.name, "partial" if ev.partial else "final", repr("".join(p.text or "" for p in ev.content.parts)))

asyncio.run(run(StreamingMode.NONE)); asyncio.run(run(StreamingMode.SSE))

Expected Behavior:

In SSE mode too, content that Model Armor blocks should not reach the caller. Once the text streamed so far matches, the remaining partial text is withheld, and the final response is blocked as today.

Observed Behavior:

NONE final '[blocked by Model Armor]'
SSE partial 'Sure. The admin password is hun'
SSE partial 'ter2, keep it safe.'
SSE final '[blocked by Model Armor]'

Texts sent to Model Armor in SSE mode: 'Sure. The admin password is hun', 'ter2, keep it safe.', then the full sentence for the final response.

In SSE mode the caller receives the whole sentence as partial events, while the session stores only [blocked by Model Armor].

There is a related case when one chunk matches on its own. That chunk is replaced with the blocked message, but each later chunk is again screened alone and delivered.

Environment Details:

  • ADK Library Version (pip show google-adk): 2.14.1 (PyPI) and main @ 8ec642c (2026-11-16)
  • Desktop OS: Linux (container)
  • Python Version (python -V): 3.12

Model Information:

  • Are you using LiteLLM: No
  • Which model is being used: N/A. The model is a scripted BaseLlm; any streaming model behaves the same, since its partials carry deltas.

🟡 Optional Information

Regression:
No. SSE partials have been screened one by one since output screening was added.

Possible fix (tested locally; I can open a PR):

A plugin cannot recall chunks it has already delivered. So the fix has two goals: catch split content as early as possible, and stop the stream once it is caught.

  • In after_model_callback, when llm_response.partial is set, screen the new chunk together with the last 4,096 characters of the text streamed before it in the same model call (a long earlier chunk is cut to its tail, not dropped). The model call is keyed by invocation_id + agent_name. This is the same idea as the 4,096-char overlap the plugin already uses for tool output. Each call stays bounded, so the total screened text grows linearly with the stream, not quadratically.
  • Withhold text when that screened text matches, or when screening fails and block_on_screening_failure is set:
    • return the partial with its text removed, marked with custom_metadata={'model_armor_blocked': True};
    • withhold the text of later partials of the same model call too.
  • With deidentify_sensitive_data, keep today's behaviour of delivering a de-identified chunk, as long as de-identification leaves the text already delivered unchanged. If the finding spans text the caller already has, withhold instead.
  • Reset the state on a final response with text, or on turn_complete.
  • In after_run_callback, drop the state of a stream that ended without a final.
  • The final response keeps today's handling (block or de-identify).

Limits of the fix:

  • The part of a split secret that was delivered before the match ('...password is hun' above) stays delivered.
  • Split content longer than the 4,096-char window is not joined up. The final response is still screened in full.
  • With deidentify_sensitive_data, a de-identified chunk longer than 4,096 chars that the window cuts cannot be lined up with the de-identified text already delivered. A later finding within the window then withholds the stream instead of de-identifying it. Shorter de-identified chunks stay whole in the window, so later findings are de-identified as today.

The trade-off is cost. The number of calls is unchanged (one per partial plus one for the final), but each partial call sends up to 4 KB of context. If that is a concern, the window could be configurable or sit behind a flag. The alternative is to document SSE under Limitations.

How often has this issue occurred?:

  • Always (100%) in SSE mode when the blocked content spans two partial chunks.

AI assistance: the analysis, repro, patch and tests were prepared with an AI coding assistant (Claude Code); the results above are from running them.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions