Repository navigation
fix(litellm): report unusable tool call arguments instead of raising - #7803
Closed
yasumorishima wants to merge 2 commits into
Closed
yasumorishima wants to merge 2 commits into
yasumorishima wants to merge 2 commits into
Conversation
Tool call arguments are parsed with json.loads and repaired with
ast.literal_eval, but only JSONDecodeError and (SyntaxError, ValueError)
were caught. json.loads raises a plain ValueError for an integer over
the int max str digits limit and RecursionError for deep nesting,
literal_eval raises TypeError for an unhashable key such as `{[]: 1}`
(and MemoryError or RecursionError for some inputs), and a repaired
literal can hold what a function call cannot carry, such as a
non-string key, a set or bytes. Each of these raised out of response
parsing instead of being reported as MALFORMED_FUNCTION_CALL.
Catch the errors the parsers raise, and check what the arguments parse
to before using it: top-level keys must be strings and the value must
encode as JSON, since the function call is serialized back to JSON
later (the streaming path does so right away; on Python 3.14 a
50,000-level object decodes but does not encode). The scans for JSON
tool calls written into text, DeepSeek arguments and the streamed
completion check get the same catch, and a tool call written into text
whose arguments cannot be encoded again stays text. _JsonPathTracker,
shared with the other streaming models, returns no partial args for a
huge integer or for nesting too deep for its recursive diff, and a
valueless partial arg for an integer too large for a float.
Fixes google#7802
A Python literal repair can give a non-string key inside a nested dict,
such as {'a': {1: 2}}, and json.dumps turned it into {"a": {"1": 2}}
instead of reporting the call. Check keys at every level, after the
encoding check, which has already ruled out cycles and deep nesting.
copybara-service Bot
pushed a commit
that referenced
this pull request
Nov 4, 2026
Merge #7803 Tool call arguments with an unhashable or non-string key, a set or bytes value from the Python-literal repair, an integer over the int max str digits limit, or very deep nesting no longer raise out of LiteLlm response parsing; they are reported as MALFORMED_FUNCTION_CALL like other unparseable arguments. A Python literal holding a set, bytes, a complex number or `...` used to reach the tool as that Python value on the non-streaming path and is now reported as malformed. A non-string key at any depth, such as `{'a': {1: 2}}`, used to reach the tool unchanged (non-streaming) or as a string key (streaming), and is now reported as malformed; in streaming this also covers top-level `{1.5: 1}`, `{None: 1}` and `{True: 1}`. The streaming partial-args tracker used by the LiteLLM, Anthropic, OpenAI and Apigee models returns no partial args for these inputs instead of raising. Fixes #7802 PiperOrigin-RevId: 1013363783
Collaborator
|
Thank you @yasumorishima for your contribution! 🎉 Your changes have been successfully imported and merged via Copybara in commit d5ecb99. Closing this PR as the changes are now in the main branch. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Link to Issue or Description of Change
1. Link to an existing issue (if applicable):
Problem:
Tool call arguments that cannot become a function call raised out of
LiteLlmresponse parsing instead of being reported asMALFORMED_FUNCTION_CALL. The cases are an unhashable or non-string key, a set or bytes value from theast.literal_evalrepair, an integer over 4,300 digits, and deep nesting. The issue has a reproduction.Solution:
ast.literal_evaldocumentsTypeError,MemoryErrorandRecursionErrorbeyondSyntaxErrorandValueError.json.loadsraises a plainValueErrorfor a huge integer andRecursionErrorfor deep nesting._JsonPathTracker, which also serves the Anthropic, OpenAI and Apigee models, stop raising. It already caughtJSONDecodeErrorandRecursionErrorfrom decoding. It now returns no partial args for an integer over 4,300 digits or nesting too deep for its recursive diff, and a valueless partial arg for an integer too large for a float (like other valuesPartialArgcannot hold). After a subtree too deep to diff, that call gets no more partial args; the final call is still complete.Behaviour that changes besides the crashes:
...used to reach the tool as that Python value. It is now reported as malformed; streaming raised on these.{1.5: 1},{None: 1}and{True: 1}used to become{"1.5": 1},{"null": 1}and{"true": 1}. They are now reported as malformed, as the non-streaming path did by raising.{'a': {1: 2}}, is now reported as malformed, instead of being encoded with the key turned into a string.Testing Plan
Unit Tests:
pytest tests/unittests/models/test_litellm.py tests/unittests/utils/test_streaming_utils.py: 884 passed on Python 3.12.14 and 3.14.7. I reverted each of the 17 changed spots one at a time, and every revert fails at least one test on both versions. For the nested key check, dropping tuples from the walk fails 3 tests, and checking keys before the encoding check hangs the self-reference test. The full suite on the fork's CI (3.11 to 3.14) has one failure,test_remote_a2a_agent.py::TestRemoteA2aAgentResolution::test_ensure_resolved_uses_custom_factory. It also fails onmaintoday.Manual End-to-End (E2E) Tests:
The reproduction in the issue, run against
main:and against this branch, including the existing warning for a discarded call:
The streaming path, driven the same way with a mocked
acompletion, now reportsMALFORMED_FUNCTION_CALLfor these four and for{'a': {1, 2}}. Onmain,{[]: 1}and{'a': {1, 2}}raisedTypeError, the huge integer raisedValueError, the deep nesting raisedRecursionError, and{1: 2}became{"1": 2}.Checklist
Additional context
🤖 Generated with Claude Code