Skip to main content

Chat Completions

POST /v1/chat/completions accepts the OpenAI Chat Completions format, with or without streaming. Every text model can be called through this endpoint, including function calling.

Example request

curl https://octalabs.id/v1/chat/completions \  -H "Authorization: Bearer $API_KEY" \  -H "Content-Type: application/json" \  -d '{    "model": "abliterated-large",    "max_tokens": 256,    "messages": [      {"role": "user", "content": "Hi! Explain what an API is in one sentence."}    ]  }'

Three groups of models

What a request may carry depends on how the model is served. The API format column on Models & Pricing tells you which group a model is in.

  • OpenAI format. The column lists only OpenAI. Your request is passed on as you sent it.
  • Claude models. The column also lists Anthropic. Your request is translated to the Anthropic Messages format and the reply is translated back. You can also call these models natively through /v1/messages.
  • Text-and-tools models. A few models that are not Claude models also list both formats. They take text and function calling only. The page does not mark them separately: when a request carries something they cannot take, the error says “for this model”.

A model can be served in more than one of these ways. A request that one of them cannot take is then handled by another that can, and you get the 400 only when none can. Read the tables below as what each group accepts, not as a promise that a field refused for one group always fails.

Supported fields

Every request field has an explicit rule. Yes means the field is honoured. Ignored means it is accepted and has no effect. Refused means the request is rejected with 400 before it runs, at no cost; error.param names the field and the message says why. A field that is not in this table is refused the same way. A field sent as null counts as not set.

Field OpenAI format Claude models Text-and-tools models Notes
modelYesYesYesRequired.
messagesYesYesYesRequired and non-empty. See Message content below.
max_tokens
max_completion_tokens
YesYesYesSee Output limit below.
streamYesYesYestrue for Server-Sent Events.
stream_optionsYesYesYesinclude_usage: true adds a usage chunk at the end of the stream.
toolsYesYesYesOpenAI format: function and custom tools. The other two groups: function tools only.
tool_choiceYesYesYes"none", "auto", "required" or a named function. OpenAI format also takes a named custom tool and allowed_tools.
parallel_tool_callsYesYesYesfalse limits a reply to one tool call.
temperatureYesYesRefusedClaude models: 0 to 1. A higher value is refused, not lowered. Some Claude models refuse any value other than 1.
top_pYesYesRefusedClaude models: 0 to 1. Some Claude models refuse it.
stopYesYesRefusedA string or an array of strings. Claude models: a sequence cannot be only whitespace.
response_formatYesYesRefusedClaude models: json_schema (or text). json_object is refused; send a json_schema with a schema instead.
thinkingYesYesRefused{"type":"enabled","budget_tokens":N}, {"type":"adaptive"} or {"type":"disabled"}. max_tokens must be larger than the budget. Text-and-tools models: use reasoning_effort.
reasoning_effortYesRefusedYesClaude models: use thinking. Text-and-tools models: minimal, low, medium, high or xhigh.
seedYesRefusedRefused
frequency_penalty
presence_penalty
YesRefusedRefused
logit_biasYesRefusedRefused
logprobs
top_logprobs
YesRefusedRefused
verbosityYesRefusedRefused
prompt_cache_keyYesRefusedRefused
nYesIgnoredIgnoredOnly 1. Any other value is refused for every model.
user
safety_identifier
YesIgnoredIgnoredA string.
metadata
store
IgnoredIgnoredIgnoredmetadata must be an object and store a boolean. Completions are never stored.
service_tierIgnoredIgnoredIgnoredOnly "auto" or "default". Any other tier is refused.
modalitiesIgnoredIgnoredIgnoredOnly ["text"]. Anything else is refused.
functions
function_call
RefusedRefusedRefusedDeprecated. Use tools and tool_choice.
audioRefusedRefusedRefusedAudio output is not available.
predictionRefusedRefusedRefusedPredicted outputs are not available.
web_search_optionsRefusedRefusedRefusedWeb search is not available. Run your own search as a function tool.
moderationRefusedRefusedRefused
prompt_cache_retention
prompt_cache_options
RefusedRefusedRefusedPrompt cache settings cannot be chosen; the default applies.

Message content

Roles are system, developer, user, assistant and tool. The content of a message is a string or an array of content parts.

Content OpenAI format Claude models Text-and-tools models Notes
TextYesYesYesA string, or text parts.
image_urlYesYesRefusedAn https:// URL or a base64 data:image/… URL. Claude models: in user messages only, as JPEG, PNG, GIF or WebP, and detail: "low" is refused.
input_audioRefusedRefusedRefusedAudio input is not available.
fileRefusedRefusedRefusedFile inputs are not available. Send the content as text or images.
Any other part typeRefusedRefusedRefused
  • A request that contains an image is held against the model’s whole context, because an image’s token count cannot be read from its size. See Billing & balance.
  • audio on an assistant message is refused for every model. Claude and text-and-tools models also refuse the deprecated function_call on a message and the function role: use tool_calls and tool messages.

Prompt caching

On OpenAI-format models, caching is applied by the model provider on its own; a cache_control marker on a content part or a tool is passed on with its default 5-minute lifetime ("ttl": "1h" is refused). For Claude models this endpoint refuses cache_control: to control caching, call them through /v1/messages. prompt_cache_breakpoint is refused for every model.

Function calling

Describe your functions in tools. When the model wants one, the reply carries tool_calls and ends with finish_reason: "tool_calls". Run the function yourself, then send its result back as a tool message with the same tool_call_id.

curl https://octalabs.id/v1/chat/completions \  -H "Authorization: Bearer $API_KEY" \  -H "Content-Type: application/json" \  -d '{    "model": "abliterated-large",    "max_tokens": 512,    "messages": [      {"role": "user", "content": "What is the weather in Jakarta right now?"}    ],    "tools": [{      "type": "function",      "function": {        "name": "get_weather",        "description": "Get the current weather for a city.",        "parameters": {          "type": "object",          "properties": {"city": {"type": "string"}},          "required": ["city"]        }      }    }]  }'
  • Only tools that your application runs are available. A tool the provider would run (web search, code execution, file search and the like) is refused by name.
  • Tool definitions are billed as input tokens and tool calls as output tokens, from the usage the provider reports, like any reply.

For Claude models the translation adds these rules:

  • A function name is 1 to 128 characters of letters, digits, _ and -, and is used once. parameters must describe an object.
  • Every tool call must be answered by a tool message directly after the assistant message that made it, before the conversation continues.
  • The arguments you send back in tool_calls must be the JSON object string the model produced.
  • tool_choice "required" or a named function needs tools. allowed_tools is not supported: list only the functions you allow in tools.
  • When a reply is only tool calls, message.content is null.

Output limit

  • Set max_tokens or max_completion_tokens. If you set both, the larger one is used as the limit.
  • Without a limit, OpenAI-format models use the model’s context limit. Claude models are sent with a limit of 16,384 tokens, or less where the model’s own maximum output is smaller; a thinking budget is added on top.
  • Thinking and tool-call arguments count towards the limit. A reply cut at the limit ends with finish_reason: "length", and a tool call cut there has incomplete arguments, so leave room for them.
  • Text-and-tools models take no output limit: max_tokens is accepted but does not cut the reply, and the hold always covers the model’s whole context.
  • A value of zero or less, or above the model’s context limit, is rejected with 400.
  • The limit determines how much balance is held before the request runs. A smaller limit means a smaller hold; the final charge is still based on actual usage. See Billing & balance.

Streaming

Send "stream": true to receive tokens as Server-Sent Events. Each token is forwarded to you as soon as it arrives from the provider.

# -N turns off buffering so tokens appear as they arrivecurl -N https://octalabs.id/v1/chat/completions \  -H "Authorization: Bearer $API_KEY" \  -H "Content-Type: application/json" \  -d '{    "model": "abliterated-large",    "max_tokens": 256,    "stream": true,    "stream_options": {"include_usage": true},    "messages": [{"role": "user", "content": "Hello!"}]  }'
  • We always request the final usage from the provider for billing. The usage chunk is forwarded to you only if you send "stream_options": {"include_usage": true} yourself.
  • While the model is quiet (for example while it is thinking), a comment line : keep-alive is sent about every 15 seconds so that no proxy closes the idle connection.
  • The x-billing-state response header is streaming. The billing outcome is sent as an SSE comment just before the stream closes.

A quiet moment and the end of the stream

data: {"id":"chatcmpl-…","choices":[{"index":0,"delta":{"content":"!"},"finish_reason":null}]}: keep-alivedata: {"id":"chatcmpl-…","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}data: [DONE]: x-billing-state settled

Lines starting with a colon are SSE comments. The official SDKs ignore them; a parser you write yourself must skip every line that starts with :. Read the last one if you want to confirm that the request was charged (settled), cost nothing (released) or was not finalised yet (pending).

Time limits

A stream has to start within 90 seconds, may then stay silent for up to 300 seconds at a time, and can run for 30 minutes in total. A reply without streaming has to be complete within 90 seconds, so use streaming for long generations. See Limits & fields.

Replies from Claude models

A reply from a Claude model is returned in the Chat Completions shape. finish_reason is mapped as follows:

finish_reason When
stopThe model finished its turn or hit one of your stop sequences.
lengthThe output limit (or the model’s context) was reached.
tool_callsThe model called one or more of your tools.
content_filterThe model declined to answer.
  • Thinking text is returned as reasoning_content on the message and in stream deltas. You do not need to send it back.
  • usage.prompt_tokens includes cached tokens; usage.prompt_tokens_details.cached_tokens is the part read from the cache.

Response headers

Header Meaning
x-billing-request-idThe billing ID of this request. It matches the Usage page of the dashboard.
x-billing-statesettled (charged), released (nothing charged), pending (not finalised yet) or streaming (outcome sent at the end of the stream).
x-request-idThe request ID to include when you contact support.

Start with any balance,
however small.

Create a free accountSign in AI API · pay as you go