Skip to content

InMemorySessionService retries with temp state duplicate history and roll back saved state #8143

Description

@OUAO-FRANK

🔴 Required Information

Describe the Bug: An equal copy of an event containing a temp: state delta can bypass InMemorySessionService.append_event() deduplication. If the event is redelivered after a newer write, it adds a duplicate history entry and reapplies the old persistent state.

Steps to Reproduce:

  1. Check out main at 12aab4aaaa1a72e3edcc65fa4f9d849896ea6c48 and run uv sync --no-dev.
  2. Save the script below as repro.py and run uv run --no-sync python repro.py.
  3. Compare the two cases printed by the script. No model, credentials, or network service is needed for the reproduction.

Expected Behavior: Both cases keep history ["early", "late"] and persisted state {"step": "done"}. The replayed copy is equal to the original input before the service trims temporary state.

Observed Behavior: With a temporary key, history is ["early", "late", "early"] and state is {"step": "draft"}. Without that key, deduplication works and state stays "done".

Environment Details:

  • ADK: 2.14.1 from the source commit above
  • OS: Windows
  • Python: 3.12.13
  • LiteLLM: No. Model: N/A.

🟡 Optional Information

Additional Context: The service compares the retried event against a stored event whose temp: fields have already been removed. The comparison fails before the retry reaches the same trimming step. Unlike the concurrent broadcast discussed in #5723, this uses one session reference and sequential writes. It models a consumer retrying the same buffered event, a use case documented in the current append_event implementation. #7454 addresses a stale caller snapshot when deduplication succeeds; here deduplication is bypassed and duplicate history is persisted.

Minimal Reproduction Code:

import asyncio

from google.adk.events import Event, EventActions
from google.adk.sessions import InMemorySessionService


async def main(include_temp):
    service = InMemorySessionService()
    ids = dict(app_name="demo", user_id="user", session_id="session")
    session = await service.create_session(**ids)
    delta = {"step": "draft"}
    if include_temp:
        delta["temp:working"] = True
    early = Event(
        id="early", author="worker", invocation_id="run",
        actions=EventActions(state_delta=delta),
    )
    retry = early.model_copy(deep=True)
    await service.append_event(session, early)
    await service.append_event(
        session,
        Event(
            id="late", author="worker", invocation_id="run",
            actions=EventActions(state_delta={"step": "done"}),
        ),
    )
    await service.append_event(session, retry)
    loaded = await service.get_session(**ids)
    print(include_temp, [e.id for e in loaded.events], loaded.state)
    expected = (["early", "late", "early"], "draft") if include_temp else (["early", "late"], "done")
    assert ([e.id for e in loaded.events], loaded.state["step"]) == expected


for _ in range(5):
    asyncio.run(main(False))
    asyncio.run(main(True))

How often has this issue occurred?: 5 of 5 trials on this checkout, with a passing control each time.

AI disclosure: OpenAI Codex assisted with source investigation, the reproduction, and writing this report.

Activity

  1. ALDRIN121 commented on Nov 14, 2026

    @ALDRIN121
    Contributor

    Claiming this one. I reproduced the shape from your description and traced it in main at 287c0b0e6.

    Mechanism. InMemorySessionService.append_event() dedupes at in_memory_session_service.py:378 with any(e == event for e in storage_session.events if e.id == event.id). The stored copies have already been through BaseSessionService._trim_temp_delta_state() (base_session_service.py:178), which drops temp: keys from state_delta; the redelivered event still carries them. So the comparison is pre-trim-input against post-trim-history and can never be equal. The event is appended twice, and storage_session.state.update(session_state_delta) at :408 then re-applies the stale non-temp delta over the newer write — which is the state rollback you saw, not just the duplicate history entry.

    Fix I'm preparing. Normalise the incoming event for the comparison only (the same trim the persistence path already applies), leaving what is stored and what is applied unchanged, so a re-delivery carrying a temp: delta matches its stored twin. Regression test in tests/unittests/sessions/test_in_memory_session_service.py using your repro shape: append the pair, re-append an equal copy that still carries a temp: key, then assert history stays ["early", "late"] and persisted state stays {"step": "done"} — plus the no-temp: control case to pin the existing behaviour.

    Checked open PRs — none reference this issue. Following the assign-first rule in CONTRIBUTING, could a maintainer assign this to me? The PR will follow today.

  2. Bensonluo commented on Nov 14, 2026

    @Bensonluo

    @ALDRIN121 thanks for tracing the mechanism — the equality check in append_event() comparing the untrimmed re-delivery against stored copies that already had their temp:-scoped state keys stripped is exactly what #8144 fixes: the dedup comparison now applies the same temp-key trim to the incoming event before comparing, so a retry that differs only in temp values matches its stored twin instead of duplicating history and re-applying the stale persistent delta. Regression tests cover the buffered-retry shape from the issue. If you were planning your own patch, feel free to compare approaches on the PR — happy to adjust.

  3. surajksharma07 commented on Nov 16, 2026

    @surajksharma07
    Collaborator

    @OUAO-FRANK I get the same thing on 2.14.1 and current main: with temp:working the retry lands as a second early and step rolls back to draft, while the no-temp control stays clean.

    As a workaround, compare a copy of the incoming event, with the temp: keys stripped from state_delta, against the stored events. On a match, just apply the temp values and skip the append. That kept history at ['early', 'late'] and state at done for me.

    Could you try that against your actual buffered-retry flow and confirm it holds up, especially when the retry carries a different temp value than the original?

    It’s the same approach as #8144, so if it checks out on your side that PR looks like the right place to land it.

  4. Bensonluo commented on Nov 16, 2026

    @Bensonluo

    For the specific case you flagged — a retry carrying a different temp value than the original — that shape is already pinned offline in #8144:

    • test_append_event_dedup_ignores_temp_value_differences re-delivers the same event with temp:working changed from initial to rewritten: history stays a single entry, the persisted session:step value is untouched, and temp:working never reaches persisted state.
    • test_append_event_temp_delta_dedupe_boundaries pins the surrounding shapes: an all-temp delta dedupes against its empty stored form; a same-id event whose persisted content genuinely differs is still kept (two entries, session:step = second); and a retry through a fresh get_session handle with a rewritten temp value still applies the retry's latest temp value to that handle while persisted state stays unchanged — the same last-value-wins application as the workaround above.

    A run against a real buffered-retry flow is still the more meaningful confirmation, so that check on the reporter's side remains the open question.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

No labels
No labels

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions