Create chat completion
Chat
Create chat completion
POST /v1/chat/completions
POST
Create chat completion
OpenAI-compatible chat completions endpoint. Streaming and non-streaming.
Request body
array
required
Conversation history. Each entry is
{ role: "system" | "user" | "assistant" | "tool", content: string }.boolean
default:"false"
When
true, responses are streamed as Server-Sent Events. For routing
matrices, set stream: false. See the streaming guide.integer
default:"1"
Number of choices to generate. For routing matrices, set
n: 1.boolean
default:"false"
When
true, return generated-token logprobs in
choices[].logprobs.content. Required for routing matrices.boolean
default:"false"
On supported MoE rollout models, add the selected expert indices for each
generated token to
choices[].logprobs.content[].routing_matrix. Requires
logprobs: true, stream: false, and n: 1.string
On rollout-enabled models, return post-filter sampling metadata on every
generated token. Accepted values are
count, non_zero_list, and
non_zero_buffer. Requires logprobs: true. Every mode adds
sampling_logprob and sampling_mask_count to
choices[].logprobs.content[]; list mode adds the complete kept set as
integer token IDs, while buffer mode adds the same IDs as base64
little-endian uint32 values. The field is compatible with
include_routing_matrix.number
default:"1"
Sampling temperature,
0 to 2. Lower → more deterministic.number
default:"1"
Nucleus sampling. Use either
temperature or top_p, not both.integer
Maximum output tokens. Capped per tier — see pricing.
Clamped to the model’s
max_output_length (visible on /v1/models).integer
Same semantics as
max_tokens; OpenAI’s canonical field for o1/o3
reasoning models. Either field is accepted; if both are sent,
max_completion_tokens wins. Clamped to the model’s
max_output_length.array
Available tools the model may call. See function calling.
string | object
default:"auto"
"auto" (default), "none", or { type: "function", function: { name: ... } } to force.object
{ type: "json_object" } or { type: "json_schema", json_schema: {...} }. See structured outputs.string | string[]
Up to 4 stop sequences.
number
default:"0"
-2.0 to 2.0. Penalize tokens by their frequency in the response so far.number
default:"0"
-2.0 to 2.0. Penalize tokens that have appeared at all.string
Reasoning effort hint for models that emit a chain of thought —
accepted as the standard OpenAI top-level field. For DeepSeek-V4
the gateway mirrors this value into
chat_template_kwargs.reasoning_effort and strips the top-level
field before forwarding, because DeepSeek-V4’s chat template only
consumes the engine-specific form. Without this mirror, top-level
reasoning_effort is silently a no-op on V4 (it’s also a
SamplingParams interference source when the value is outside
OpenAI’s enum). The OpenRouter-style alias "xhigh" is mapped to
DeepSeek’s "max" (“Think Max” mode). DeepSeek-V4 documents
"high" and "max"; other OpenAI tiers
(minimal | low | medium) are forwarded literally but fall back to
the encoder’s default branch on this build — i.e. they don’t 400
but may produce reasoning depth indistinguishable from sending no
hint. Disable thinking entirely with
chat_template_kwargs.enable_thinking: false. If you set
chat_template_kwargs.reasoning_effort explicitly, the gateway
honors your value and leaves the top-level field alone.object
Engine-specific chat-template knobs forwarded verbatim to the
upstream. On DeepSeek-V4:
{ enable_thinking: false } disables
reasoning and routes output directly to content;
{ drop_thinking: true } drops prior assistant reasoning_content
from the encoded prompt. Cogito defaults drop_thinking to false
for reasoning-capable models so multi-turn requests preserve prior
reasoning traces unless you explicitly opt out;
{ reasoning_effort: "high" | "max" } selects the model’s
Think-High vs Think-Max mode. The gateway force-injects
enable_thinking: false when response_format is set (so JSON-mode
and structured outputs land in content, not reasoning_content);
any caller-supplied value here always wins.Response (non-streaming)
Response (streaming)
Content-Type: text/event-stream. Each event is
data: <chat.completion.chunk JSON>. Stream ends with data: [DONE]. See the
streaming guide.
When sampling_mask is set, sampling metadata stays on the same token
logprob entry in both non-streaming responses and streamed token chunks.
Provider-private transport fields are never returned.
Headers on every response
x-request-id— opaque ID. Log it. We trace it through every layer.x-tokens-used— billed total for this request (omitted on errors that didn’t consume tokens).