# Create chat completion
Source: https://docs.cogito.decart.ai/api-reference/chat-completions

POST https://api.cogito.decart.ai/v1/chat/completions
POST /v1/chat/completions

OpenAI-compatible chat completions endpoint. Streaming and non-streaming.

## Request body

<ParamField type="string">
  Model slug from the [catalog](/getting-started/models). Example: `gpt-oss-120b`.
</ParamField>

<ParamField type="array">
  Conversation history. Each entry is `{ role: "system" | "user" | "assistant" | "tool", content: string }`.
</ParamField>

<ParamField type="boolean">
  When `true`, responses are streamed as Server-Sent Events. For routing
  matrices, set `stream: false`. See the [streaming guide](/guides/streaming).
</ParamField>

<ParamField type="integer">
  Number of choices to generate. For routing matrices, set `n: 1`.
</ParamField>

<ParamField type="boolean">
  When `true`, return generated-token logprobs in
  `choices[].logprobs.content`. Required for routing matrices.
</ParamField>

<ParamField type="boolean">
  On supported MoE rollout models, add the selected expert indices for each
  generated token to `choices[].logprobs.content[].routing_matrix`. Requires
  `logprobs: true`, `stream: false`, and `n: 1`.
</ParamField>

<ParamField type="string">
  On rollout-enabled models, return post-filter sampling metadata on every
  generated token. Accepted values are `count`, `non_zero_list`, and
  `non_zero_buffer`. Requires `logprobs: true`. Every mode adds
  `sampling_logprob` and `sampling_mask_count` to
  `choices[].logprobs.content[]`; list mode adds the complete kept set as
  integer token IDs, while buffer mode adds the same IDs as base64
  little-endian `uint32` values. The field is compatible with
  `include_routing_matrix`.
</ParamField>

<ParamField type="number">
  Sampling temperature, `0` to `2`. Lower → more deterministic.
</ParamField>

<ParamField type="number">
  Nucleus sampling, `0` to `1`. Use either `temperature` or `top_p`, not both.
  `0` keeps only the most likely token (greedy decoding).
</ParamField>

<ParamField type="integer">
  Maximum output tokens. Capped per tier — see [pricing](/getting-started/pricing).
  Clamped to the model's `max_output_length` (visible on `/v1/models`).
  A negative value is rejected; a value that is not a positive integer is
  ignored and the model's default budget applies.
</ParamField>

<ParamField type="integer">
  Same semantics as `max_tokens`; OpenAI's canonical field for o1/o3
  reasoning models. Either field is accepted; if both are sent,
  `max_completion_tokens` wins. Clamped to the model's
  `max_output_length`.
</ParamField>

<ParamField type="array">
  Available tools the model may call. See [function calling](/guides/function-calling).
</ParamField>

<ParamField type="string | object">
  `"auto"` (default), `"none"`, or `{ type: "function", function: { name: ... } }` to force.
</ParamField>

<ParamField type="object">
  `{ type: "json_object" }` or `{ type: "json_schema", json_schema: {...} }`. See [structured outputs](/guides/structured-outputs).
</ParamField>

<ParamField type="string | string[]">
  Up to 4 stop sequences.
</ParamField>

<ParamField type="number">
  `-2.0` to `2.0`. Penalize tokens by their frequency in the response so far.
</ParamField>

<ParamField type="number">
  `-2.0` to `2.0`. Penalize tokens that have appeared at all.
</ParamField>

<ParamField type="string">
  Reasoning effort hint for models that emit a chain of thought —
  accepted as the standard OpenAI top-level field. For DeepSeek-V4
  the gateway mirrors this value into
  `chat_template_kwargs.reasoning_effort` and strips the top-level
  field before forwarding, because DeepSeek-V4's chat template only
  consumes the engine-specific form. Without this mirror, top-level
  `reasoning_effort` is silently a no-op on V4 (it's also a
  SamplingParams interference source when the value is outside
  OpenAI's enum). The OpenRouter-style alias `"xhigh"` is mapped to
  DeepSeek's `"max"` ("Think Max" mode). DeepSeek-V4 documents
  `"high"` and `"max"`; other OpenAI tiers
  (`minimal | low | medium`) are forwarded literally but fall back to
  the encoder's default branch on this build — i.e. they don't 400
  but may produce reasoning depth indistinguishable from sending no
  hint. Disable thinking entirely with
  `chat_template_kwargs.enable_thinking: false`. If you set
  `chat_template_kwargs.reasoning_effort` explicitly, the gateway
  honors your value and leaves the top-level field alone.
</ParamField>

<ParamField type="object">
  Engine-specific chat-template knobs forwarded verbatim to the
  upstream. On DeepSeek-V4: `{ enable_thinking: false }` disables
  reasoning and routes output directly to `content`;
  `{ drop_thinking: true }` drops prior assistant `reasoning_content`
  from the encoded prompt. Cogito defaults `drop_thinking` to `false`
  for reasoning-capable models so multi-turn requests preserve prior
  reasoning traces unless you explicitly opt out;
  `{ reasoning_effort: "high" | "max" }` selects the model's
  Think-High vs Think-Max mode. The gateway force-injects
  `enable_thinking: false` when `response_format` is set (so JSON-mode
  and structured outputs land in `content`, not `reasoning_content`);
  any caller-supplied value here always wins.
</ParamField>

## Response (non-streaming)

```json theme={null}
{
  "id": "req_...",
  "object": "chat.completion",
  "created": 1714521600,
  "model": "gpt-oss-120b",
  "choices": [
    {
      "index": 0,
      "message": { "role": "assistant", "content": "..." },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 32,
    "completion_tokens": 71,
    "total_tokens": 103,
    "prompt_tokens_details": { "cached_tokens": 0 },
    "completion_tokens_details": { "reasoning_tokens": 0 }
  }
}
```

## Response (streaming)

`Content-Type: text/event-stream`. Each event is
`data: <chat.completion.chunk JSON>`. Stream ends with `data: [DONE]`. See the
[streaming guide](/guides/streaming).

When `sampling_mask` is set, sampling metadata stays on the same token
logprob entry in both non-streaming responses and streamed token chunks.
Provider-private transport fields are never returned.

## Headers on every response

* `x-request-id` — opaque ID. Log it. We trace it through every layer.
* `x-tokens-used` — billed total for this request (omitted on errors that didn't consume tokens).


# Create completion
Source: https://docs.cogito.decart.ai/api-reference/completions

POST https://api.cogito.decart.ai/v1/completions
POST /v1/completions

Legacy OpenAI-compatible text completions endpoint for prompt-based clients and
benchmark harnesses. Use [chat completions](/api-reference/chat-completions) for
new applications.

This route is available for live inference models. Placeholder catalog rows are
rejected with `model_not_found` instead of returning a synthetic completion.

## Request body

<ParamField type="string">
  Model id from the [catalog](/getting-started/models). Example:
  `moonshotai/kimi-k2.6:appliedcompute`.
</ParamField>

<ParamField type="string | string[] | integer[] | integer[][]">
  Prompt text or token ids to complete. For prompt replay, pass one string or
  one non-empty token-id list.
</ParamField>

<ParamField type="boolean">
  When `true`, responses are streamed as Server-Sent Events and end with
  `data: [DONE]`.
</ParamField>

<ParamField type="integer">
  Maximum output tokens. Clamped to the model's `max_output_length`.
</ParamField>

<ParamField type="integer">
  Number of completions to generate.
</ParamField>

<ParamField type="boolean | integer | null">
  Request a number of most likely tokens at each position from `0` through
  `5`. The boolean value `true` is accepted for token-aligned metadata requests
  using `sampling_mask` or `include_routing_matrix`. Completion logprobs use the
  legacy OpenAI arrays: `tokens`,
  `token_logprobs`, `top_logprobs`, and `text_offset`. Routing matrices require
  `true` or a numeric value.
</ParamField>

<ParamField type="boolean | null">
  On supported rollout models, `true` prepends the full prompt to
  `choices[].text`. Use `stream: false` and `n: 1` for prompt replay.
</ParamField>

<ParamField type="integer | null">
  On supported rollout models, prepends the last N authoritative prompt
  tokens. It must be a non-negative safe integer. `0` selects zero prompt
  tokens; a value longer than the prompt selects the full prompt. A non-null
  `echo_last` overrides `echo`, including `echo_last: 0`. Use `stream: false`
  and `n: 1`.
</ParamField>

<ParamField type="boolean">
  On supported MoE rollout models, add the selected expert indices for each
  scored token to `choices[].logprobs.content[].routing_matrix`. Requires
  `logprobs: true` or numeric `logprobs`, `stream: false`, and `n: 1`.
</ParamField>

<ParamField type="string">
  On rollout-enabled models, return post-filter sampling metadata on every
  generated token. Accepted values are `count`, `non_zero_list`, and
  `non_zero_buffer`; all require `logprobs: true`. Each
  `choices[].logprobs.content[]` entry receives `sampling_logprob` and
  `sampling_mask_count`. List mode returns the complete kept set as integer
  token IDs; buffer mode returns the same IDs as base64 little-endian `uint32`
  values. The field composes with `include_routing_matrix`.
</ParamField>

<ParamField type="number">
  Sampling temperature, `0` to `2`. Lower values are more deterministic.
</ParamField>

<ParamField type="number">
  Nucleus sampling, `0` to `1`. Use either `temperature` or `top_p`, not both.
  `0` keeps only the most likely token (greedy decoding).
</ParamField>

<ParamField type="string | string[]">
  Stop sequence or sequences.
</ParamField>

<ParamField type="number">
  `-2.0` to `2.0`. Penalize tokens by their frequency in the response so far.
</ParamField>

<ParamField type="number">
  `-2.0` to `2.0`. Penalize tokens that have appeared at all.
</ParamField>

<ParamField type="boolean">
  Engine extension used by fixed-length benchmark harnesses. When supported by
  the selected upstream, the model continues until `max_tokens` or another stop
  condition is reached.
</ParamField>

## Example

```bash theme={null}
curl https://api.cogito.decart.ai/v1/completions \
  -H "Authorization: Bearer $COGITO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "moonshotai/kimi-k2.6:appliedcompute",
    "prompt": "Write one sentence about fast inference:",
    "max_tokens": 32
  }'
```

## Response

```json theme={null}
{
  "id": "cmpl_...",
  "object": "text_completion",
  "created": 1714521600,
  "model": "moonshotai/kimi-k2.6",
  "choices": [
    {
      "index": 0,
      "text": " Fast inference keeps agent loops tight.",
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 8,
    "completion_tokens": 8,
    "total_tokens": 16
  }
}
```

## Prompt replay response

An active selector returns the full authoritative prompt ids separately from
the generated ids:

* `choices[].prompt_token_ids` contains the full prompt, even when
  `echo_last` selects only a suffix or zero tokens.
* `choices[].token_ids` contains generated tokens only.
* `choices[].text` contains the selected prompt suffix followed by generated
  text.

The standard completion envelope, usage accounting, and legacy logprob arrays
keep their OpenAI meanings. `usage.prompt_tokens` counts the full prompt and
`usage.completion_tokens` counts generated tokens; echoing text does not change
billing counts.

For a full prompt echo, `choices[].text`, `prompt_token_ids`, and the legacy
`tokens` array begin with the first global prompt token, whose `token_logprobs`
entry is `null`. The additive `choices[].logprobs.content` extension begins
with the second prompt token and continues through every generated token,
keeping each score aligned with its route.

When `sampling_mask` is set, the generated-token content entries also carry
the public sampling fields. Provider-private transport fields are stripped
from both non-streaming responses and streamed chunks.

## Headers on every response

* `x-request-id` — opaque ID. Log it. We trace it through every layer.


# Errors
Source: https://docs.cogito.decart.ai/api-reference/errors

Deterministic error schema. Always includes a request_id.

Cogito returns a single, deterministic error shape on every failure:

```json theme={null}
{
  "error": {
    "type": "invalid_request_error",
    "code": "model_not_found",
    "message": "Model 'foo' is not available.",
    "param": null,
    "request_id": "req_..."
  }
}
```

Always log `request_id`. We trace it through the gateway, scheduler, and inference cluster. Support can pinpoint your request from this single ID.

`error.param` is a string or `null`: when set it points at the offending
request field. Codes that populate it today: `context_length_exceeded` sets it
to `"messages"`, `"prompt"`, or `"max_tokens"`; `invalid_image_format`,
`invalid_image_url`, and `invalid_video_url` set it to the path of the
rejected media part (e.g. `"messages[1].content[2].image_url.url"`);
`grammar_not_supported` sets it to
`"response_format"` or `"tool_choice"`; `invalid_tool_call_arguments` sets it
to the path of the offending arguments string (e.g.
`"messages[1].tool_calls[0].function.arguments"`); and `backend_invalid_argument`
relays the serving stack's own param path when it provides one. On every
other error it is `null`.

## Error types

`error.type` is set per HTTP status by the gateway. The mapping is
deterministic — the table below is the full set you can encounter.

| Type | HTTP | When |
| - | - | - |
| `invalid_request_error` | 400, 404 | Malformed JSON, missing required field, invalid params, unknown model slug. All 404s file here (both body-level and gateway-level missing routes) — distinguish via `error.code` (`model_not_found` vs. `not_found`). |
| `authentication_error` | 401 | Missing / revoked API key |
| `insufficient_quota` | 402 | Balance exhausted (out of credit) |
| `permission_denied` | 403 | Key doesn't have access to a model or feature |
| `rate_limit_error` | 429 | Token / request rate limit hit, **or** monthly hard spend cap reached |
| `api_error` | 500–502, 504+ | Cogito problem; retry safe |
| `service_unavailable` | 503 | Billing not configured, or inference cluster temporarily degraded; retry with backoff |

## Common codes

`error.code` is the machine-readable handle. Always log it alongside
`request_id`. Grouped by category:

**Auth**

* `invalid_api_key` — Bearer header missing or revoked (401)
* `missing_authorization` — no Bearer header at all (401)

**Billing**

* `insufficient_quota` — balance ≤ 0 (402). The wire code matches
  `error.type` — both are `insufficient_quota`. Renamed from the
  earlier `insufficient_balance` for OpenAI parity.
* `spend_cap_reached` — month-to-date spend has hit your hard cap (429).
  Renamed from the earlier `spend_capped`. Raise the cap in the
  dashboard or wait for the month to roll over.
* `billing_not_configured` — server misconfiguration on our side (503)

**Validation**

* `invalid_request_error` — generic body-shape failure (400)
* `model_not_found` — slug typo or model retired (404)
* `context_length_exceeded` — the serving route rejected input that cannot fit
  the model's context window (400). Ultra-fast routes can reject estimated
  input or input plus an explicit `max_tokens` budget before dispatch; other
  routes validate with their serving tokenizer and may clamp only the output
  budget. Compact or shorten the input before retrying; `error.param` names
  the offending field when available: `messages`, `prompt`, or `max_tokens`.
* `invalid_image_format` — an `image_url` part carries data that is not a
  decodable image (400): wrong bytes for any supported format, empty or
  invalid base64, or a format the serving stack cannot decode (e.g. HEIC).
  Send `png`, `jpeg`, `gif`, or `webp`; `error.param` is the path of the
  rejected part (e.g. `messages[0].content[1].image_url.url`). Do not
  retry unchanged — the same payload will always be rejected.
* `invalid_image_url` / `invalid_video_url` — an `image_url` (or Responses
  `input_image`) or `video_url` part carries a URL whose scheme is not
  supported (400). Media URLs must be `http://`, `https://`, or an inline
  base64 `data:` URL; `file://`, other schemes, and local paths are rejected
  before dispatch. `error.param` is the path of the rejected part. Do not
  retry unchanged.
* `grammar_not_supported` — the request pinned grammar-constrained output
  (`response_format` with `json_schema`, or a forced `tool_choice`) on a
  serving route that runs no grammar backend (400). `error.param` is
  `response_format` or `tool_choice`. Do not retry unchanged.
* `invalid_tool_call_arguments` — a replayed assistant `tool_calls[]` entry
  (or a Responses `function_call` input item) carries `function.arguments`
  that is not a string containing a JSON object (400). The serving stack
  cannot render the chat template for such a conversation. `error.param` is
  the path of the offending string (e.g.
  `messages[1].tool_calls[0].function.arguments`). Fix the replayed
  arguments before retrying — the same history will always be rejected.
* `backend_invalid_argument` — the serving stack rejected the request as
  invalid with its own explicit 4xx verdict (400): an image URL that could
  not be fetched, a schema its grammar compiler cannot express, and similar
  request-content faults. `error.message` carries the serving stack's
  detail and `error.param` the offending field path when available. Do not
  retry unchanged — the same request will be rejected again.

**Upstream / cluster**

* `backend_unavailable` — gateway-side error before reaching the
  upstream (e.g. missing backend credentials) (500, surfaces as
  `error.type: api_error`)
* `upstream_unavailable` — TCP/connection failure to the upstream
  cluster (502, surfaces as `error.type: api_error`)
* `upstream_<status>` — verbatim upstream HTTP status pass-through
  (e.g. `upstream_503` when the inference cluster itself returned 503)

## Retry-after

`429` and `503` responses include a `Retry-After` header (seconds). Honor it. Exponential backoff on top of `Retry-After` is fine; ignoring it will get you rate-limited harder.


# List models
Source: https://docs.cogito.decart.ai/api-reference/list-models

GET https://api.cogito.decart.ai/v1/models
GET /v1/models

Returns the catalog of models available to your account. OpenAI-compatible.

## Response

```json theme={null}
{
  "object": "list",
  "data": [
    {
      "id": "gpt-oss-120b",
      "object": "model",
      "created": 1714521600,
      "owned_by": "cogito",
      "context_window": 131072
    },
    ...
  ]
}
```

## Notes

* Fine-tuned variants tied to your account appear in this list with their own slugs (e.g. `ft:gpt-oss-120b:my-org:summary-v1`).
* The `context_window` field is a Cogito extension; standard OpenAI clients ignore it.

## Retrieve one model

Use the OpenAI-compatible retrieve-model endpoint to fetch an entitled model by
its exact, URL-encoded ID. This also works for a dedicated model that is
callable but intentionally absent from the collection.

```bash theme={null}
curl https://api.cogito.decart.ai/v1/models/<url-encoded-model-id> \
  -H "Authorization: Bearer $COGITO_API_KEY"
```

Rollout-enabled MoE models include `routing_matrix`, an additive Cogito field
with the static format needed to decode router replay payloads. Fetch it once
when the trainer starts; completion responses do not repeat it.

```json theme={null}
{
  "id": "<model_id>",
  "routing_matrix": {
    "dtype": "uint16",
    "shape": [60, 8]
  }
}
```


# Create response
Source: https://docs.cogito.decart.ai/api-reference/responses

POST https://api.cogito.decart.ai/v1/responses
POST /v1/responses

OpenAI-compatible Responses endpoint for stateless text generation and
function-tool workflows. Streaming and non-streaming requests use the same
authentication, model access, usage accounting, and billing as chat
completions.

<Warning>
  This endpoint is stateless. It does not store response objects or support
  background responses, conversations, or `previous_response_id`. Send the
  complete request state in `input` on every call.
</Warning>

## Request body

<ParamField type="string">
  Model slug from the [catalog](/getting-started/models).
</ParamField>

<ParamField type="string | array">
  A text prompt or an array of Responses input items. Message items,
  `function_call`, and `function_call_output` are supported. Vision-capable
  models also accept URL or inline-data `input_image` parts; the URL must be
  `http://`, `https://`, or a base64 `data:` URL (other schemes reject with
  `invalid_image_url`).
</ParamField>

<ParamField type="string">
  System or developer instructions for this response.
</ParamField>

<ParamField type="boolean">
  When `true`, returns typed Server-Sent Events such as
  `response.output_text.delta` and `response.completed`.
</ParamField>

<ParamField type="integer">
  Maximum output tokens. The gateway caps this at the model's
  `max_output_length`.
</ParamField>

<ParamField type="array">
  Function tools the model may call. Hosted tools such as web search, file
  search, computer use, code interpreter, shell, and MCP are not supported.
</ParamField>

<ParamField type="string | object">
  Function-tool selection using the OpenAI Responses shape.
</ParamField>

<ParamField type="boolean">
  Allow the model to return multiple function calls in one response.
</ParamField>

<ParamField type="object">
  Reasoning configuration. `reasoning.effort` is forwarded to compatible
  models.
</ParamField>

<ParamField type="object">
  Text output configuration. `text.format` accepts text, JSON object, and JSON
  schema formats on models that support structured outputs.
</ParamField>

<ParamField type="boolean">
  Must be `false` when provided. Cogito does not persist response objects.
</ParamField>

## Example

```bash theme={null}
curl https://api.cogito.decart.ai/v1/responses \
  -H "Authorization: Bearer $COGITO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "moonshotai/kimi-k2.6",
    "input": "Write a one-line release note.",
    "max_output_tokens": 128,
    "store": false
  }'
```

## Unsupported features

The gateway returns `400 unsupported_feature` before inference for:

* `store: true`, `background: true`, `previous_response_id`, `conversation`,
  hosted `prompt`, and `max_tool_calls`
* automatic truncation (`truncation: "auto"`)
* hosted or custom non-function tools and their history items
* file, audio, and video input; image file IDs; and output logprobs

Only `POST /v1/responses` is exposed. Retrieve, delete, input-items, and cancel
routes are not available.

## Streaming

Streaming responses preserve the Responses API's typed SSE events. Usage is
read from the terminal response event for billing. Provider-private extension
fields are removed before events leave the public gateway.


# Auto-recharge
Source: https://docs.cogito.decart.ai/billing/auto-recharge

Keep your balance topped up automatically so requests never 402 mid-deploy.

## What it does

When your Cogito balance drops below a threshold you set, we charge your saved card for a fixed amount and credit the proceeds to your token balance. No interactive checkout, no 402s mid-deploy.

Configure it under **Dashboard → Billing → Auto-recharge**. You set two values:

* **Threshold** — when balance falls below this, we trigger a recharge.
* **Recharge amount** — how much to add. Must be greater than the threshold.

Bounds: $5 ≤ threshold < amount ≤ $10,000.

## Saving a card

Auto-recharge needs a saved card. Open **Dashboard → Billing → Saved card** and click **Save a card** — you'll go through Stripe Checkout, which collects your card details. We don't charge anything; the card is stored on your Stripe Customer for future use.

To replace the card, click **Replace** in the same section and enter the new one. To remove it entirely, click **Remove** — auto-recharge will silently stop firing until you save a card again.

Saved cards are also used for one-click manual top-ups: when you click a top-up amount, Stripe Checkout pre-selects your saved card so you don't re-enter details every time.

## When it triggers

A recharge fires within seconds of any API request that drops your balance below the threshold. We hold a per-account cooldown of about a minute, so a burst of concurrent requests results in a single charge, not many.

We never preemptively charge — recharges are always reactive to actual usage.

## When it fails

If your card is declined (expired, frozen, insufficient funds, fraud hold), we:

1. Send an email at the recipient on file describing the bank's reason.
2. Pause auto-recharge for an hour to avoid spamming retries.
3. After three consecutive failures, **disable auto-recharge** and email you. To re-enable, replace the saved card under **Saved card → Replace** and turn auto-recharge back on.

Your saved card stays on file regardless — disabling auto-recharge does not detach it.

## Receipts

We email a receipt for every successful auto-recharge to the address on your account. Manual top-ups also send a receipt. There's no per-request charge email — those would be too noisy. Detailed transaction history is always available in **Dashboard → Billing**.

## Disabling

Flip the **Enable auto-recharge** toggle off and click Save. We immediately stop firing new charges. Pending charges already in flight at Stripe will still complete.

## FAQ

**Will auto-recharge run if I'm offline / not in the app?**
Yes. The trigger fires on the API server side as soon as the balance crosses the threshold.

**Does the threshold include pending charges?**
No. We check the credited balance only — a recharge in flight at Stripe doesn't bump the threshold check until the webhook lands.

**What if my account is suspended for non-payment?**
Auto-recharge halts. You'll get the failure email. Resolve the card issue, then re-enable from the dashboard.

**Can I set a maximum monthly auto-recharge spend?**
Use the [hard spend cap](/getting-started/pricing#hard-spend-caps) on the same billing page. It applies to all spend, including the credit you've added via auto-recharge.


# Authentication
Source: https://docs.cogito.decart.ai/getting-started/authentication

API keys, header format, and best practices.

Cogito authenticates with bearer tokens — the same shape OpenAI uses, so any OpenAI-compatible SDK works without modification.

```http theme={null}
Authorization: Bearer cog-live-...
```

## Creating keys

1. Sign in at [cogito.decart.ai](https://cogito.decart.ai).
2. Open **Dashboard → API keys**.
3. Click **Create**, name the key (we recommend naming it after the environment — `production-api`, `local-dev`), and copy the secret immediately. The full secret is shown once.

## Best practices

* **One key per environment**. Local dev, staging, and production should each have their own key.
* **Rotate on schedule**. We surface `last_used_at` on every key — keys that haven't been used in 30 days are good rotation candidates.
* **Revoke instantly**. Compromised keys can be revoked from the dashboard; revocation propagates within seconds.
* **Don't commit keys**. Use environment variables (`COGITO_API_KEY`) or your platform's secret manager.

## Scopes

Model access is scoped with **projects**. A project belongs to your organization, selects which models it allows, and owns its own API keys:

1. Open **Dashboard → Projects** and create a project (org owners and admins only).
2. Pick **All models** or **Selected models only** on the project page.
3. Create keys inside the project. Those keys can call only the project's models — everything else returns the same `404 model_not_found` as a model that doesn't exist, on `/v1/chat/completions`, `/v1/completions`, `/v1/models`, and `/v1/metrics` alike.

Keys created from **Dashboard → API keys** (outside any project) keep full access to every model your organization can use. Deleting a project permanently revokes its keys.

Other per-key scoping (read-only, monthly budget) ships in a later release.

## Errors

A request without a valid key returns:

```json theme={null}
{
  "error": {
    "type": "invalid_request_error",
    "code": "invalid_api_key",
    "message": "The API key provided is missing or revoked.",
    "request_id": "req_..."
  }
}
```

Always log the `request_id` — we trace it through every layer of the gateway and inference cluster, so support can find your request immediately.


# Model catalog
Source: https://docs.cogito.decart.ai/getting-started/models

Available open-source models, with context windows, output caps, throughput, and pricing.

Cogito serves a curated set of frontier open-source models — **Kimi K2.6 · GLM-5.2 · GPT-OSS · Qwen**.

The full served catalog below. We add models the day they ship; older models stay supported with deprecation notice.

| Model | Family | Context | Max output | TPS | License |
| - | - | - | - | - | - |
| `moonshotai/kimi-k2.6` | Moonshot | 256K | 256K | 70 | Modified MIT |
| `moonshotai/kimi-k2.6:ultra-fast-new` | Moonshot | 9K | 8K | 70 | Modified MIT |
| `zai-org/GLM-5.2-NVFP4` | Zhipu | 1M | 1M | 70 | MIT |
| `gpt-oss-120b` | OpenAI | 128K | 32K | 70 | Apache 2.0 |
| `qwen-3-235b` | Alibaba | 256K | 32K | 70 | Apache 2.0 |

**Live pricing** — input, cached input, and output rates per model — is published on the [model catalog](https://cogito.decart.ai/models) and returned by `GET /v1/models`. Single source of truth so the website, the gateway, and your code never drift.

For Kimi K2.6 routes, Moonshot documents the limit as input plus output fitting within the 256K context window; Cogito therefore advertises a 256K max-output cap while upstream may still reject requests whose prompt leaves insufficient room.

**Throughput** is locked at 70 tokens/sec across the fleet for the MVP — we run a uniform serving target while we tune Trainium / GPU autoscaling. Real-world per-request throughput varies with prompt length and concurrent batch saturation.

**Hardware** is managed by Cogito. We route each model to the right silicon for the workload and may transparently shift between tiers if it produces lower P99 — output is bit-identical either way.

## Capabilities

All catalog models support:

* Streaming SSE
* Function / tool calling (OpenAI-shape `tools[]`)
* Multi-turn chat with system prompts

Models that advertise `structured_outputs` from `GET /v1/models` also support
Structured JSON outputs through grammar-constrained decoding. Routes that omit
that feature, including GPT-OSS and current ultra-fast TPU preview lanes, reject
enforced `response_format` requests with a 400.

## Picking a model

* **Frontier reasoning + long context** → `zai-org/GLM-5.2-NVFP4`. 1M-token window, strong coding and agentic reasoning.
* **Coding agents and tool-heavy workflows** → `moonshotai/kimi-k2.6`. Served from the AWS B300 high-capacity route.
* **TPU preview Kimi requests** → `moonshotai/kimi-k2.6:ultra-fast-new`. Dedicated `new-kimi` Basalt TPU lane with a 9,216-token context limit.
* **Cheap general chat / coding** → `gpt-oss-120b`. Cheapest path to GPT-4-class output quality.
* **Multilingual + tool use** → `qwen-3-235b`. Strong non-English coverage.

For more on each model, see the per-model pages in the sidebar.


# Overview
Source: https://docs.cogito.decart.ai/getting-started/overview

Stop watching your agent type. Cogito serves open-source LLMs 14× faster than typical providers, on AWS Trainium and NVIDIA Blackwell.

> **Stop watching your agent type.**

Cogito runs your agents 14× faster than typical inference providers — on the open-source models you already use. Calibrated against the \~70 tok/s baseline that Together publishes for Llama 70B class models; Cogito's premium tier sustains 1000+ tok/s on supported workloads.

Cogito is the LLM inference platform from Decart, an efficiency-first research lab. It's built on DOS — the Decart Optimization Stack — which extracts frontier performance from any silicon. The same stack that runs Lucy 2.0 (a large diffusion model) at sub-50ms on AWS Trainium now serves open-source LLMs at 14× the speed of typical providers.

**Now serving:** Kimi K2.6 · GLM-5.2 · GPT-OSS · Qwen.

Inference runs on a hybrid fleet: **AWS Trainium** for cost-efficient throughput, **NVIDIA Blackwell** for the largest configurations. Cogito picks the right silicon per workload; you call one API.

## Built on DOS

Cogito is built on **DOS — the Decart Optimization Stack** — a multi-silicon performance layer that routes workloads to optimal silicon and extracts frontier throughput. The same stack runs Lucy 2.0, a large diffusion model, at sub-50ms latency on AWS Trainium.

For LLM inference, DOS lets Cogito serve open-source models at 14× the speed of typical providers — particularly for agentic workloads, where multi-step inference latency compounds.

## Drop-in OpenAI compatible

The whole API matches the OpenAI spec — chat completions, legacy text completions, streaming SSE, function calling, structured JSON outputs. Swap two values in your existing code and you're on Cogito:

```python theme={null}
from openai import OpenAI

client = OpenAI(
    base_url="https://api.cogito.decart.ai/v1",  # was: https://api.openai.com/v1
    api_key=os.environ["COGITO_API_KEY"],
)
```

## Three-step quickstart

<Steps>
  <Step title="Create an API key">
    Sign up at [cogito.decart.ai/sign-up](https://cogito.decart.ai/sign-up). You start with free credits — no card required.
  </Step>

  <Step title="Pick a model">
    Browse the [model catalog](/getting-started/models). Each model lists context window, throughput, and per-million-token pricing.
  </Step>

  <Step title="Send a request">
    Copy the [quickstart snippet](/getting-started/quickstart). Five-minute time-to-first-token.
  </Step>
</Steps>

## Why Cogito

<CardGroup>
  <Card title="Trainium + Blackwell" icon="microchip">
    Hybrid fleet, automatic routing. Frontier-tier throughput on the open-source models you actually use.
  </Card>

  <Card title="Built for operators" icon="circle-check">
    Token-aware rate limits, deterministic errors with `request_id`, hard spend caps, zero retention by default.
  </Card>

  <Card title="Cogito, ergo ship." icon="rocket">
    Five-minute TTFT. Same API for production and your fine-tunes. Get back to shipping.
  </Card>
</CardGroup>


# Pricing
Source: https://docs.cogito.decart.ai/getting-started/pricing

How tokens are counted, billed, and discounted.

> **Pay for performance, not for the wait.**

## How billing works

Cogito charges per token of usage — separate rates for **standard input tokens**, **cached input tokens**, and **output tokens**. Per-model rates are published in one place: the [model catalog](https://cogito.decart.ai/models). Each model card lists all three; the same numbers come back from `GET /v1/models` (`pricing.prompt`, `pricing.input_cache_read`, `pricing.completion`).

For each request you're billed:

```
cost = (input_tokens − cached_input_tokens) × input_rate
     + cached_input_tokens × cached_input_rate
     + output_tokens × output_rate
```

## Context cache

When you re-send the same prefix tokens (a long system prompt, a reference document, a multi-turn conversation), Cogito automatically charges them at the model's cached-input rate, which is published per-model alongside the standard rate. No code change required.

Every response includes `usage.cached_input_tokens` alongside `prompt_tokens` and `completion_tokens` so you can verify the discount applied.

## Fine-tuned variants

Your fine-tunes price the **same per-token rate** as the base model. We don't charge a premium for serving them.

## Hard spend caps

Set a maximum monthly spend in **Dashboard → Billing**. Once you hit it, requests return:

```json theme={null}
{
  "error": {
    "type": "insufficient_quota",
    "code": "spend_cap_reached",
    "message": "Monthly spend cap reached. Raise the cap or top up to continue.",
    "request_id": "req_..."
  }
}
```

This protects you from runaway loops and DDoS-by-bug. We never silently bypass your cap.

## Free credits

New accounts start with \$5 in free credits. No card required to spend them. Card is required only when you top up beyond the free tier.

## Plans

| Plan | Pricing | When to pick it |
| - | - | - |
| Free | \$0 + free credits | Evaluation, prototyping |
| Pro | Pay as you go | Production traffic |
| Enterprise | Contract | VPC isolation, P99 SLA, BAA, SSO |

[See full plans →](https://cogito.decart.ai/pricing)


# Quickstart
Source: https://docs.cogito.decart.ai/getting-started/quickstart

Send your first Cogito request in under five minutes.

## 1. Get an API key

Sign in at [cogito.decart.ai](https://cogito.decart.ai), then create a key from **Dashboard → API keys**. Copy it once — we never show the full secret again.

```bash theme={null}
export COGITO_API_KEY="cog-live-..."
```

## 2. Send a request

Cogito is OpenAI-compatible. Use the OpenAI SDK in any language; just point `base_url` at `https://api.cogito.decart.ai/v1`.

<CodeGroup>
  ```bash curl theme={null}
  curl https://api.cogito.decart.ai/v1/chat/completions \
    -H "Authorization: Bearer $COGITO_API_KEY" \
    -H "Content-Type: application/json" \
    -d '{
      "model": "deepseek-v4-flash",
      "messages": [
        { "role": "user", "content": "Why is the sky blue?" }
      ]
    }'
  ```

  ```python python theme={null}
  from openai import OpenAI

  client = OpenAI(
      base_url="https://api.cogito.decart.ai/v1",
      api_key=os.environ["COGITO_API_KEY"],
  )

  response = client.chat.completions.create(
      model="deepseek-v4-flash",
      messages=[{"role": "user", "content": "Why is the sky blue?"}],
  )

  print(response.choices[0].message.content)
  ```

  ```typescript node theme={null}
  import OpenAI from "openai";

  const client = new OpenAI({
    baseURL: "https://api.cogito.decart.ai/v1",
    apiKey: process.env.COGITO_API_KEY,
  });

  const response = await client.chat.completions.create({
    model: "deepseek-v4-flash",
    messages: [{ role: "user", content: "Why is the sky blue?" }],
  });

  console.log(response.choices[0].message.content);
  ```
</CodeGroup>

## 3. Stream tokens

Add `stream: true` to receive an SSE stream of OpenAI-compatible chunks.

<CodeGroup>
  ```python python theme={null}
  stream = client.chat.completions.create(
      model="deepseek-v4-flash",
      messages=[{"role": "user", "content": "Why is the sky blue?"}],
      stream=True,
  )

  for chunk in stream:
      print(chunk.choices[0].delta.content or "", end="", flush=True)
  ```

  ```typescript node theme={null}
  const stream = await client.chat.completions.create({
    model: "deepseek-v4-flash",
    messages: [{ role: "user", content: "Why is the sky blue?" }],
    stream: true,
  });

  for await (const chunk of stream) {
    process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
  }
  ```
</CodeGroup>

## Next

* [Authentication](/getting-started/authentication) — keys, scopes, header format
* [Streaming](/guides/streaming) — backpressure, cancellation, error recovery
* [Function calling](/guides/function-calling) — tools, parallel calls, JSON arguments
* [Structured outputs](/guides/structured-outputs) — guaranteed JSON schema adherence
* [Model catalog](/getting-started/models) — what's available, what's coming


# Function calling
Source: https://docs.cogito.decart.ai/guides/function-calling

Wire LLMs into your tools with reliable, parallel function calls.

Cogito supports OpenAI-compatible function calling on every model in the catalog. Define your tools once, and the model picks which to call, in what order, with which arguments.

## Defining tools

```python theme={null}
tools = [
    {
        "type": "function",
        "function": {
            "name": "get_current_weather",
            "description": "Get the current weather in a given city",
            "parameters": {
                "type": "object",
                "properties": {
                    "city": {"type": "string"},
                    "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
                },
                "required": ["city"],
            },
        },
    }
]
```

## Calling

```python theme={null}
response = client.chat.completions.create(
    model="gpt-oss-120b",
    messages=[{"role": "user", "content": "What's the weather in Tokyo?"}],
    tools=tools,
    tool_choice="auto",
)

tool_calls = response.choices[0].message.tool_calls
# [{ "id": "call_...", "type": "function",
#    "function": { "name": "get_current_weather",
#                  "arguments": "{\"city\": \"Tokyo\", \"unit\": \"celsius\"}" } }]
```

## Parallel calls

Models in the catalog can issue multiple calls in one turn. The response's `tool_calls` array can contain more than one entry — execute them concurrently and return all results in your next turn.

```python theme={null}
messages.append(response.choices[0].message)

for call in tool_calls:
    result = run_tool(call.function.name, json.loads(call.function.arguments))
    messages.append({
        "role": "tool",
        "tool_call_id": call.id,
        "content": json.dumps(result),
    })

final = client.chat.completions.create(
    model="gpt-oss-120b",
    messages=messages,
    tools=tools,
)
```

## Reliability

Cogito uses **grammar-constrained decoding** at the inference engine level. The model can only produce token sequences that match the JSON schema of one of your declared tools — there's no parsing fallback. Argument JSON is guaranteed valid, even on small models.

If the model decides no tool is appropriate, it returns a normal text response with `tool_calls: null`.

## Force a specific tool

```python theme={null}
tool_choice = {"type": "function", "function": {"name": "get_current_weather"}}
```

Useful when you want to skip the tool selection step.


# Streaming
Source: https://docs.cogito.decart.ai/guides/streaming

SSE chunks, cancellation, error recovery.

Set `stream: true` to receive an OpenAI-compatible Server-Sent Events stream of chat completion chunks.

## Chunk shape

Each `data:` line carries one JSON object with the same structure as a non-streaming response, but with `delta` instead of `message`:

```json theme={null}
{
  "id": "req_...",
  "object": "chat.completion.chunk",
  "created": 1714521600,
  "model": "gpt-oss-120b",
  "choices": [
    {
      "index": 0,
      "delta": { "content": "tok" },
      "finish_reason": null
    }
  ]
}
```

The first chunk includes `delta.role: "assistant"`. The final chunk has `finish_reason: "stop"` (or `length`, `tool_calls`, `content_filter`). The stream ends with the literal sentinel:

```
data: [DONE]
```

## Cancellation

Closing the connection cancels generation server-side and stops billing immediately. We don't charge you for tokens you cancelled before they arrived.

```typescript theme={null}
const controller = new AbortController();

const response = await client.chat.completions.create(
  { model: "gpt-oss-120b", messages: [...], stream: true },
  { signal: controller.signal },
);

// Cancel after 3 seconds
setTimeout(() => controller.abort(), 3000);
```

## Error recovery

Network blips during a stream show up as truncated SSE — the connection drops without a `[DONE]` sentinel. The OpenAI SDK throws on this. On retry, replay your full message history; chunks already received are not retained server-side.

Every response (streamed or not) includes an `x-request-id` header. Log it. When you open a support ticket we trace through the entire stack from that single ID.

## Backpressure

Cogito's gateway respects HTTP/2 backpressure. If you're consuming the stream slowly (e.g. piping to a slow UI), the server pauses generation rather than buffering. This means a lazy reader doesn't get billed for tokens it never reads.


# Structured outputs
Source: https://docs.cogito.decart.ai/guides/structured-outputs

Guaranteed JSON schema adherence on models that advertise structured outputs.

When you need the model to return JSON that exactly matches a schema, use
`response_format` with a model that advertises `structured_outputs` from
`GET /v1/models`. Cogito enforces the schema with grammar-constrained decoding
on those routes — the model literally cannot emit tokens that would break the
schema, so you don't need to ask nicely or add "respond only in JSON" to the
system prompt.

## JSON schema mode

```python theme={null}
response = client.chat.completions.create(
    model="deepseek-v4-flash",
    messages=[
        {"role": "user", "content": "Extract entities from: 'Apple acquired Beats in 2014 for $3B.'"}
    ],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "entity_extraction",
            "strict": True,
            "schema": {
                "type": "object",
                "properties": {
                    "company": {"type": "string"},
                    "acquired": {"type": "string"},
                    "year": {"type": "integer"},
                    "amount_usd": {"type": "string"}
                },
                "required": ["company", "acquired", "year", "amount_usd"],
                "additionalProperties": False
            }
        }
    }
)

import json
data = json.loads(response.choices[0].message.content)
# {"company": "Apple", "acquired": "Beats", "year": 2014, "amount_usd": "$3B"}
```

## What "strict" guarantees

With `strict: true`, every property in `required` is present, every value matches its declared type, and no extra fields are emitted. You can `json.loads()` without a try/except in production.

## json\_object mode

For "give me any valid JSON" without a schema:

```python theme={null}
response = client.chat.completions.create(
    model="deepseek-v4-flash",
    messages=[...],
    response_format={"type": "json_object"},
)
```

Less strict — the structure is up to the model — but the result is still guaranteed valid JSON.

## Streaming structured outputs

Both modes work with `stream: true`. Chunks arrive as token-level deltas and assemble into valid JSON once the stream completes. Don't `JSON.parse` partial chunks; only the assembled string is guaranteed valid.

## Errors

If the schema is malformed (recursive without a depth limit, mutually exclusive constraints), Cogito returns a 400 before generation starts. Models that do not advertise `structured_outputs` return a 400 for enforced `response_format` requests.


# GLM-5.2
Source: https://docs.cogito.decart.ai/models/glm-5-2

Zhipu's frontier open-weight MoE — 1M-token context and strong agentic reasoning.

Zhipu AI's frontier open-weight model. A large MoE with a 1M-token context window and strong coding and agentic reasoning, served on Cogito's optimized stack.

| | |
| - | - |
| **Model ID** | `zai-org/GLM-5.2-NVFP4` |
| **Route slug** | `glm-5.2-nvfp4` |
| **Parameters** | 744B (MoE, \~40B active) |
| **Context** | 1,048,576 tokens (1M) |
| **License** | MIT |
| **Pricing** | [cogito.decart.ai/models/glm-5.2-nvfp4](https://cogito.decart.ai/models/glm-5.2-nvfp4) |

## Best for

* Coding agents and multi-step tool use
* Long-context workloads (up to 1M tokens)
* Frontier reasoning at open-weight economics

```python theme={null}
client.chat.completions.create(
    model="zai-org/GLM-5.2-NVFP4",
    messages=[...],
)
```

## License

GLM-5.2 ships under the MIT license — permissive enough for commercial deployments; check the upstream license for redistribution specifics.


# GPT-OSS 120B
Source: https://docs.cogito.decart.ai/models/gpt-oss-120b

OpenAI's open-weight 120B MoE. Strong general reasoning at near-commodity pricing.

OpenAI's first open-weight release since GPT-2. A 120B mixture-of-experts model with around 5B active parameters per token, strong general reasoning, and reliable function calling.

| | |
| - | - |
| **Slug** | `gpt-oss-120b` |
| **Parameters** | 120B (MoE, \~5B active) |
| **Context** | 131,072 tokens (128K) |
| **Throughput** | 70 tokens/sec |
| **TTFT** | 240ms |
| **License** | Apache 2.0 |
| **Pricing** | [cogito.decart.ai/models/gpt-oss-120b](https://cogito.decart.ai/models/gpt-oss-120b) |

## Best for

* Default chat / agent
* Coding assistance
* Tool calling and low-cost reasoning
* Workloads where GPT-4-class output quality at open-weight pricing matters

## Sample request

```python theme={null}
from openai import OpenAI

client = OpenAI(
    base_url="https://api.cogito.decart.ai/v1",
    api_key=os.environ["COGITO_API_KEY"],
)

response = client.chat.completions.create(
    model="gpt-oss-120b",
    messages=[{"role": "user", "content": "Hello!"}],
)
```

## License

Apache 2.0. Use it however you want, including commercial deployments. Cogito's serving doesn't change the license terms.


# Kimi K2.6
Source: https://docs.cogito.decart.ai/models/kimi-k2-6

Moonshot's flagship MoE for coding agents and long-horizon reasoning.

Moonshot's flagship MoE. Trained for long-horizon agentic workflows; the model engineering teams reach for when the cheap models stop being enough.

| | |
| - | - |
| **Model ID** | `moonshotai/kimi-k2.6` |
| **Route slug** | `kimi-k2.6` |
| **Parameters** | 1T MoE (\~32B active) |
| **Context** | 262,144 tokens (256K) |
| **Throughput** | 70 tokens/sec |
| **TTFT** | 280ms |
| **License** | Modified MIT |
| **Pricing** | [cogito.decart.ai/models/kimi-k2.6](https://cogito.decart.ai/models/kimi-k2.6) |

## Best for

* Coding agents (multi-file refactors, multi-step PRs)
* Long-horizon agentic reasoning
* Complex tool use with parallel function calls
* Workloads that benefit from long, reliable reasoning chains

```python theme={null}
client.chat.completions.create(
    model="moonshotai/kimi-k2.6",
    messages=[...],
)
```

The default `moonshotai/kimi-k2.6` route is served from the AWS B300 high-capacity Moonshot deployment.

## License

Kimi K2.6 ships under a Modified MIT license. Permissive enough for commercial deployments; check the upstream license for redistribution specifics.


# Kimi K2.7 Code
Source: https://docs.cogito.decart.ai/models/kimi-k2-7

Moonshot's newest flagship MoE — the latest Kimi. Early-access / reserved capacity.

Moonshot's newest flagship MoE — the latest Kimi. Currently available for early-access and reserved capacity; [contact us](https://cogito.decart.ai/enterprise#contact) to get on it.

| | |
| - | - |
| **Model ID** | `moonshotai/kimi-k2.7` |
| **Route slug** | `kimi-k2.7` |
| **Parameters** | 1T MoE (\~32B active) |
| **Context** | 262,144 tokens (256K) |
| **License** | Modified MIT |
| **Availability** | Early-access / reserved — [contact sales](https://cogito.decart.ai/enterprise#contact) |

## Best for

* Coding agents and long-horizon tool use
* Teams that want the newest Kimi the moment it ships

## License

Modified MIT — same as the Kimi K2.6 family.


# Qwen3 235B
Source: https://docs.cogito.decart.ai/models/qwen-3-235b

Alibaba's open-weight flagship — multilingual, tool-friendly, Apache 2.0.

Alibaba's open-weight flagship. Strong multilingual coverage and tool use at a price point that holds up against the proprietary mid-tier.

| | |
| - | - |
| **Slug** | `qwen-3-235b` |
| **Parameters** | 235B MoE (\~22B active) |
| **Context** | 262,144 tokens (256K) |
| **Throughput** | 70 tokens/sec |
| **TTFT** | 260ms |
| **License** | Apache 2.0 |
| **Pricing** | [cogito.decart.ai/models/qwen-3-235b](https://cogito.decart.ai/models/qwen-3-235b) |

## Best for

* Multilingual agents (40+ languages with strong coverage)
* Coding agents and tool-heavy workflows
* Long-horizon planning where the cheap models stop being enough
* Workloads that require an unrestricted Apache 2.0 license

```python theme={null}
client.chat.completions.create(
    model="qwen-3-235b",
    messages=[{"role": "user", "content": "..."}],
)
```

<Note>
  This row is currently mock-backed — the live Qwen3 deployment is being provisioned. The catalog tile shows "Contact sales" until the upstream is up. [Talk to sales](https://cogito.decart.ai/enterprise#contact) if you need access ahead of GA.
</Note>