Skip to content

Custom OpenAI-compatible models: reasoning_content is never sent back when the provider streams it without an id #338819

Description

Describe the bug

With a custom Chat Completions model (github.copilot.chat.customOAIModels, "thinking": true), VS Code parses streamed reasoning_content but never sends it back on later requests.

OpenAIEndpoint.getCompletionsCallback only echoes reasoning if (data?.id) (openAIEndpoint.ts#L160). The id comes only from cot_id, reasoning_opaque or signature (thinkingUtils.ts#L30). Standard OpenAI-compatible servers (SGLang, vLLM and similar) stream reasoning_content with no id, so every later request carries the assistant message's content and tool_calls, but no reasoning.

Models that expect their reasoning back on every assistant message degrade badly. With Kimi K3 served by SGLang, the model reasoned for 68 tokens on the first step and exactly 3 tokens on every later step. After six steps it repeated the same turn until cancelled. Replaying the second request with the first step's reasoning added back gave 175, 303 and 244 reasoning tokens, versus 3, 3 and 3 without it. Adding a cot_id to each reasoning delta in a proxy fixes it.

Affected version

1.138.0 and 1.139.1. The code is unchanged on main.

Steps to reproduce the behavior

  1. Serve a reasoning model behind an OpenAI-compatible server that streams reasoning_content, for example SGLang with --reasoning-parser, and log the requests it receives
  2. Add it under github.copilot.chat.customOAIModels with "thinking": true and "toolCalling": true
  3. Run an agent request that makes a few tool calls
  4. The responses stream reasoning_content, but the assistant messages in later requests have no reasoning_content

Expected behavior

With "thinking": true, reasoning received as reasoning_content is sent back on its assistant message, whether or not the provider supplied an id.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions