Chat Completions
POST /v1/chat/completions accepts the OpenAI Chat Completions format, with or without streaming. Every text model can be called through this endpoint, including function calling.
Example request
curl https://octalabs.id/v1/chat/completions \ -H "Authorization: Bearer $API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "abliterated-large", "max_tokens": 256, "messages": [ {"role": "user", "content": "Hi! Explain what an API is in one sentence."} ] }'Three groups of models
What a request may carry depends on how the model is served. The API format column on Models & Pricing tells you which group a model is in.
- OpenAI format. The column lists only OpenAI. Your request is passed on as you sent it.
- Claude models. The column also lists Anthropic. Your request is translated to the Anthropic Messages format and the reply is translated back. You can also call these models natively through /v1/messages.
- Text-and-tools models. A few models that are not Claude models also list both formats. They take text and function calling only. The page does not mark them separately: when a request carries something they cannot take, the error says “for this model”.
A model can be served in more than one of these ways. A request that one of them cannot take is then handled by another that can, and you get the 400 only when none can. Read the tables below as what each group accepts, not as a promise that a field refused for one group always fails.
Supported fields
Every request field has an explicit rule. Yes means the field is honoured. Ignored means it is accepted and has no effect. Refused means the request is rejected with 400 before it runs, at no cost; error.param names the field and the message says why. A field that is not in this table is refused the same way. A field sent as null counts as not set.
| Field | OpenAI format | Claude models | Text-and-tools models | Notes |
|---|---|---|---|---|
model | Yes | Yes | Yes | Required. |
messages | Yes | Yes | Yes | Required and non-empty. See Message content below. |
max_tokensmax_completion_tokens | Yes | Yes | Yes | See Output limit below. |
stream | Yes | Yes | Yes | true for Server-Sent Events. |
stream_options | Yes | Yes | Yes | include_usage: true adds a usage chunk at the end of the stream. |
tools | Yes | Yes | Yes | OpenAI format: function and custom tools. The other two groups: function tools only. |
tool_choice | Yes | Yes | Yes | "none", "auto", "required" or a named function. OpenAI format also takes a named custom tool and allowed_tools. |
parallel_tool_calls | Yes | Yes | Yes | false limits a reply to one tool call. |
temperature | Yes | Yes | Refused | Claude models: 0 to 1. A higher value is refused, not lowered. Some Claude models refuse any value other than 1. |
top_p | Yes | Yes | Refused | Claude models: 0 to 1. Some Claude models refuse it. |
stop | Yes | Yes | Refused | A string or an array of strings. Claude models: a sequence cannot be only whitespace. |
response_format | Yes | Yes | Refused | Claude models: json_schema (or text). json_object is refused; send a json_schema with a schema instead. |
thinking | Yes | Yes | Refused | {"type":"enabled","budget_tokens":N}, {"type":"adaptive"} or {"type":"disabled"}. max_tokens must be larger than the budget. Text-and-tools models: use reasoning_effort. |
reasoning_effort | Yes | Refused | Yes | Claude models: use thinking. Text-and-tools models: minimal, low, medium, high or xhigh. |
seed | Yes | Refused | Refused | |
frequency_penaltypresence_penalty | Yes | Refused | Refused | |
logit_bias | Yes | Refused | Refused | |
logprobstop_logprobs | Yes | Refused | Refused | |
verbosity | Yes | Refused | Refused | |
prompt_cache_key | Yes | Refused | Refused | |
n | Yes | Ignored | Ignored | Only 1. Any other value is refused for every model. |
usersafety_identifier | Yes | Ignored | Ignored | A string. |
metadatastore | Ignored | Ignored | Ignored | metadata must be an object and store a boolean. Completions are never stored. |
service_tier | Ignored | Ignored | Ignored | Only "auto" or "default". Any other tier is refused. |
modalities | Ignored | Ignored | Ignored | Only ["text"]. Anything else is refused. |
functionsfunction_call | Refused | Refused | Refused | Deprecated. Use tools and tool_choice. |
audio | Refused | Refused | Refused | Audio output is not available. |
prediction | Refused | Refused | Refused | Predicted outputs are not available. |
web_search_options | Refused | Refused | Refused | Web search is not available. Run your own search as a function tool. |
moderation | Refused | Refused | Refused | |
prompt_cache_retentionprompt_cache_options | Refused | Refused | Refused | Prompt cache settings cannot be chosen; the default applies. |
Message content
Roles are system, developer, user, assistant and tool. The content of a message is a string or an array of content parts.
| Content | OpenAI format | Claude models | Text-and-tools models | Notes |
|---|---|---|---|---|
| Text | Yes | Yes | Yes | A string, or text parts. |
image_url | Yes | Yes | Refused | An https:// URL or a base64 data:image/… URL. Claude models: in user messages only, as JPEG, PNG, GIF or WebP, and detail: "low" is refused. |
input_audio | Refused | Refused | Refused | Audio input is not available. |
file | Refused | Refused | Refused | File inputs are not available. Send the content as text or images. |
| Any other part type | Refused | Refused | Refused |
- A request that contains an image is held against the model’s whole context, because an image’s token count cannot be read from its size. See Billing & balance.
audioon an assistant message is refused for every model. Claude and text-and-tools models also refuse the deprecatedfunction_callon a message and thefunctionrole: usetool_callsandtoolmessages.
Prompt caching
On OpenAI-format models, caching is applied by the model provider on its own; acache_control marker on a content part or a tool is passed on with its default 5-minute lifetime ("ttl": "1h" is refused). For Claude models this endpoint refuses cache_control: to control caching, call them through /v1/messages. prompt_cache_breakpoint is refused for every model. Function calling
Describe your functions in tools. When the model wants one, the reply carries tool_calls and ends with finish_reason: "tool_calls". Run the function yourself, then send its result back as a tool message with the same tool_call_id.
curl https://octalabs.id/v1/chat/completions \ -H "Authorization: Bearer $API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "abliterated-large", "max_tokens": 512, "messages": [ {"role": "user", "content": "What is the weather in Jakarta right now?"} ], "tools": [{ "type": "function", "function": { "name": "get_weather", "description": "Get the current weather for a city.", "parameters": { "type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"] } } }] }'- Only tools that your application runs are available. A tool the provider would run (web search, code execution, file search and the like) is refused by name.
- Tool definitions are billed as input tokens and tool calls as output tokens, from the usage the provider reports, like any reply.
For Claude models the translation adds these rules:
- A function name is 1 to 128 characters of letters, digits,
_and-, and is used once.parametersmust describe an object. - Every tool call must be answered by a
toolmessage directly after the assistant message that made it, before the conversation continues. - The
argumentsyou send back intool_callsmust be the JSON object string the model produced. tool_choice"required"or a named function needstools.allowed_toolsis not supported: list only the functions you allow intools.- When a reply is only tool calls,
message.contentisnull.
Output limit
- Set
max_tokensormax_completion_tokens. If you set both, the larger one is used as the limit. - Without a limit, OpenAI-format models use the model’s context limit. Claude models are sent with a limit of 16,384 tokens, or less where the model’s own maximum output is smaller; a
thinkingbudget is added on top. - Thinking and tool-call arguments count towards the limit. A reply cut at the limit ends with
finish_reason: "length", and a tool call cut there has incomplete arguments, so leave room for them. - Text-and-tools models take no output limit:
max_tokensis accepted but does not cut the reply, and the hold always covers the model’s whole context. - A value of zero or less, or above the model’s context limit, is rejected with
400. - The limit determines how much balance is held before the request runs. A smaller limit means a smaller hold; the final charge is still based on actual usage. See Billing & balance.
Streaming
Send "stream": true to receive tokens as Server-Sent Events. Each token is forwarded to you as soon as it arrives from the provider.
# -N turns off buffering so tokens appear as they arrivecurl -N https://octalabs.id/v1/chat/completions \ -H "Authorization: Bearer $API_KEY" \ -H "Content-Type: application/json" \ -d '{ "model": "abliterated-large", "max_tokens": 256, "stream": true, "stream_options": {"include_usage": true}, "messages": [{"role": "user", "content": "Hello!"}] }'- We always request the final usage from the provider for billing. The usage chunk is forwarded to you only if you send
"stream_options": {"include_usage": true}yourself. - While the model is quiet (for example while it is thinking), a comment line
: keep-aliveis sent about every 15 seconds so that no proxy closes the idle connection. - The
x-billing-stateresponse header isstreaming. The billing outcome is sent as an SSE comment just before the stream closes.
A quiet moment and the end of the stream
data: {"id":"chatcmpl-…","choices":[{"index":0,"delta":{"content":"!"},"finish_reason":null}]}: keep-alivedata: {"id":"chatcmpl-…","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}data: [DONE]: x-billing-state settled Lines starting with a colon are SSE comments. The official SDKs ignore them; a parser you write yourself must skip every line that starts with :. Read the last one if you want to confirm that the request was charged (settled), cost nothing (released) or was not finalised yet (pending).
Time limits
A stream has to start within 90 seconds, may then stay silent for up to 300 seconds at a time, and can run for 30 minutes in total. A reply without streaming has to be complete within 90 seconds, so use streaming for long generations. See Limits & fields.Replies from Claude models
A reply from a Claude model is returned in the Chat Completions shape. finish_reason is mapped as follows:
| finish_reason | When |
|---|---|
stop | The model finished its turn or hit one of your stop sequences. |
length | The output limit (or the model’s context) was reached. |
tool_calls | The model called one or more of your tools. |
content_filter | The model declined to answer. |
- Thinking text is returned as
reasoning_contenton the message and in stream deltas. You do not need to send it back. usage.prompt_tokensincludes cached tokens;usage.prompt_tokens_details.cached_tokensis the part read from the cache.
Response headers
| Header | Meaning |
|---|---|
x-billing-request-id | The billing ID of this request. It matches the Usage page of the dashboard. |
x-billing-state | settled (charged), released (nothing charged), pending (not finalised yet) or streaming (outcome sent at the end of the stream). |
x-request-id | The request ID to include when you contact support. |