Repository navigation
Description
🔴 Required Information
Is your feature request related to a specific problem?
LiteLlm does not tell LiteLLM which conversation a call belongs to. Behind a LiteLLM Proxy, the proxy therefore gives every call its own session_id in LiteLLM_SpendLogs, and the calls of one ADK session cannot be grouped (cost per conversation, evaluation, traces).
Measured with a root agent that has one tool and a sub-agent reached through transfer_to_agent, run for 3 turns in a single ADK session with LiteLlm("litellm_proxy/<model>") (LiteLLM 1.105.1, ADK 2.5.0):
| calls | distinct session_id in LiteLLM_SpendLogs |
|
|---|---|---|
| today | 6 | 6 |
with the header x-litellm-session-id: <session.id> |
6 | 1, equal to the ADK session id, sub-agent included |
Describe the Solution You'd Like
When the model is served through a LiteLLM Proxy, LiteLlm sends the ADK session id in the x-litellm-session-id header, which is the header the proxy reads to set the session of a call.
- No header when the request has no session id (calls made outside a Runner), and no header to providers other than the proxy.
- A session id header already set by the caller (
extra_headersat construction, orhttp_options.headers) is kept.
Impact on your work
Without it, per-conversation cost and quality analysis on the proxy side needs a callback copied into every agent. Not blocking, there is a workaround (below).
Willingness to contribute
Yes. A PR with tests is ready and linked below.
🟡 Recommended Information
Describe Alternatives You've Considered
- Workaround today: a
before_model_callback(or a plugin) that setsllm_request.config.http_options.headers["x-litellm-session-id"]from the session. It works, but every app has to know the header and re-implement it. - A label in
config.labels(likeadk_agent_name): the Gemini model sends labels to Vertex, so this would add a very high-cardinality billing label.
Proposed API / Implementation
The model does not see the session today, so the id needs a way to reach it:
class LlmRequest(BaseModel):
session_id: str | None = Field(default=None, exclude=True)- The LLM flow sets it from
invocation_context.session.id, next to where it adds theadk_agent_namelabel, unless it is already set. exclude=Truekeeps it out ofmodel_dump(), so recorded or traced requests do not change.LiteLlmreads it and adds the header for proxied models.
Open questions:
- Is a field on
LlmRequestthe channel you would want for this? - Should it be on by default (as in the PR), or opt-in?
Additional Context
ADK already adds adk_agent_name to every request so that billing can be sliced per agent. This is the same idea, per session.
Activity