Skip to content

LiteLlm does not pass the ADK session to a LiteLLM proxy: every call gets its own session_id #8186

Description

@thomaslprr

🔴 Required Information

Is your feature request related to a specific problem?

LiteLlm does not tell LiteLLM which conversation a call belongs to. Behind a LiteLLM Proxy, the proxy therefore gives every call its own session_id in LiteLLM_SpendLogs, and the calls of one ADK session cannot be grouped (cost per conversation, evaluation, traces).

Measured with a root agent that has one tool and a sub-agent reached through transfer_to_agent, run for 3 turns in a single ADK session with LiteLlm("litellm_proxy/<model>") (LiteLLM 1.105.1, ADK 2.5.0):

calls distinct session_id in LiteLLM_SpendLogs
today 6 6
with the header x-litellm-session-id: <session.id> 6 1, equal to the ADK session id, sub-agent included

Describe the Solution You'd Like

When the model is served through a LiteLLM Proxy, LiteLlm sends the ADK session id in the x-litellm-session-id header, which is the header the proxy reads to set the session of a call.

  • No header when the request has no session id (calls made outside a Runner), and no header to providers other than the proxy.
  • A session id header already set by the caller (extra_headers at construction, or http_options.headers) is kept.

Impact on your work

Without it, per-conversation cost and quality analysis on the proxy side needs a callback copied into every agent. Not blocking, there is a workaround (below).

Willingness to contribute

Yes. A PR with tests is ready and linked below.


🟡 Recommended Information

Describe Alternatives You've Considered

  • Workaround today: a before_model_callback (or a plugin) that sets llm_request.config.http_options.headers["x-litellm-session-id"] from the session. It works, but every app has to know the header and re-implement it.
  • A label in config.labels (like adk_agent_name): the Gemini model sends labels to Vertex, so this would add a very high-cardinality billing label.

Proposed API / Implementation

The model does not see the session today, so the id needs a way to reach it:

class LlmRequest(BaseModel):
  session_id: str | None = Field(default=None, exclude=True)
  • The LLM flow sets it from invocation_context.session.id, next to where it adds the adk_agent_name label, unless it is already set.
  • exclude=True keeps it out of model_dump(), so recorded or traced requests do not change.
  • LiteLlm reads it and adds the header for proxied models.

Open questions:

  1. Is a field on LlmRequest the channel you would want for this?
  2. Should it be on by default (as in the PR), or opt-in?

Additional Context

ADK already adds adk_agent_name to every request so that billing can be sliced per agent. This is the same idea, per session.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions