Create completion
Chat
Create completion
POST /v1/completions
POST
Create completion
Legacy OpenAI-compatible text completions endpoint for prompt-based clients and
benchmark harnesses. Use chat completions for
new applications.
This route is available for Dynamo-backed models. Placeholder catalog rows are
rejected with
model_not_found instead of returning a synthetic completion.
Request body
string | string[] | integer[] | integer[][]
required
Prompt text or token ids to complete. For prompt replay, pass one string or
one non-empty token-id list.
boolean
default:"false"
When
true, responses are streamed as Server-Sent Events and end with
data: [DONE].integer
Maximum output tokens. Clamped to the model’s
max_output_length.integer
default:"1"
Number of completions to generate.
boolean | integer | null
Request a number of most likely tokens at each position from
0 through
5. The boolean value true is accepted for token-aligned metadata requests
using sampling_mask or include_routing_matrix. Completion logprobs use the
legacy OpenAI arrays: tokens,
token_logprobs, top_logprobs, and text_offset. Routing matrices require
true or a numeric value.boolean | null
default:"false"
On supported rollout models,
true prepends the full prompt to
choices[].text. Use stream: false and n: 1 for prompt replay.integer | null
On supported rollout models, prepends the last N authoritative prompt
tokens. It must be a non-negative safe integer.
0 selects zero prompt
tokens; a value longer than the prompt selects the full prompt. A non-null
echo_last overrides echo, including echo_last: 0. Use stream: false
and n: 1.boolean
default:"false"
On supported MoE rollout models, add the selected expert indices for each
scored token to
choices[].logprobs.content[].routing_matrix. Requires
logprobs: true or numeric logprobs, stream: false, and n: 1.string
On rollout-enabled models, return post-filter sampling metadata on every
generated token. Accepted values are
count, non_zero_list, and
non_zero_buffer; all require logprobs: true. Each
choices[].logprobs.content[] entry receives sampling_logprob and
sampling_mask_count. List mode returns the complete kept set as integer
token IDs; buffer mode returns the same IDs as base64 little-endian uint32
values. The field composes with include_routing_matrix.number
default:"1"
Sampling temperature,
0 to 2. Lower values are more deterministic.number
default:"1"
Nucleus sampling. Use either
temperature or top_p, not both.string | string[]
Stop sequence or sequences.
number
default:"0"
-2.0 to 2.0. Penalize tokens by their frequency in the response so far.number
default:"0"
-2.0 to 2.0. Penalize tokens that have appeared at all.boolean
Engine extension used by fixed-length benchmark harnesses. When supported by
the selected upstream, the model continues until
max_tokens or another stop
condition is reached.Example
Response
Prompt replay response
An active selector returns the full authoritative prompt ids separately from the generated ids:choices[].prompt_token_idscontains the full prompt, even whenecho_lastselects only a suffix or zero tokens.choices[].token_idscontains generated tokens only.choices[].textcontains the selected prompt suffix followed by generated text.
usage.prompt_tokens counts the full prompt and
usage.completion_tokens counts generated tokens; echoing text does not change
billing counts.
For a full prompt echo, choices[].text, prompt_token_ids, and the legacy
tokens array begin with the first global prompt token, whose token_logprobs
entry is null. The additive choices[].logprobs.content extension begins
with the second prompt token and continues through every generated token,
keeping each score aligned with its route.
When sampling_mask is set, the generated-token content entries also carry
the public sampling fields. Provider-private transport fields are stripped
from both non-streaming responses and streamed chunks.
Headers on every response
x-request-id— opaque ID. Log it. We trace it through every layer.