API compatibility and parameters

Parameters pass through

OpenAI fields in the request body (model, messages, stream, max_tokens, temperature, tools, and so on) are forwarded upstream as-is. We don't rewrite them and we don't add defaults.

Which means: the exact meaning of these parameters is whatever the upstream model's own documentation says. The same temperature doesn't behave identically across vendors, and some models silently ignore fields they don't recognise. We deliberately don't "translate" in the middle — doing so would make your results differ from what the upstream docs led you to expect.

model must be the full name from the Models & pricing page, and must be allowed by this key's allowlist.

Don't cross the endpoints

Sending a chat model to /v1/embeddings (or the reverse) returns 400 model_not_supported_for_endpoint, and the message says which kind of model it actually is. This is caught before forwarding, so it doesn't burn an upstream call.

GET /v1/models returns everything by default. To get only chat models, pass the non-standard query parameter ?modality=chat. Behaviour without it is unchanged.

Streamed responses carry an extra chunk

When you pass "stream": true, the gateway forces usage reporting on (equivalent to adding stream_options.include_usage=true, whether or not you sent it). So the stream ends with a chunk like this:

data: {"choices":[],"usage":{"prompt_tokens":..,"completion_tokens":..,"total_tokens":..}}
data: [DONE]

Its choices is an empty array. If your parser assumes every chunk has choices[0], this one will throw — code written against the official OpenAI SDKs is fine; hand-rolled SSE parsing needs to allow for it.

The chunk exists so we can meter the call. If the stream drops you won't receive it, but the tokens already produced are still billed.

/v1/embeddings does not support streaming — it returns the full vector in one response.

Context limits are judged on input alone

If the input by itself exceeds the model's declared context length, you get 400 context_length_exceeded, rejected before forwarding.

That check looks only at the input and does not add max_tokens — doing so would be guessing how much you're going to generate, and a wrong guess rejects a request that would have worked. So "input plus max_tokens overflows" is not caught here; the upstream handles it by its own rules.

⚠️ Automatic retries are billed twice

The gateway performs no cross-request de-duplication. This matches OpenAI's own behaviour on this endpoint — it doesn't support Idempotency-Key either.

The consequence is concrete: when your SDK re-sends the same logical call after a network timeout, the gateway treats it as a brand-new request and bills it again. The first attempt may well have completed upstream; only the response failed to reach you.

So:

• Check the default retry count in your SDK (the official openai SDK retries by default) and lower or disable it to suit your case;
• When you do need retries, de-duplicate on your side (for example, record against an order id whether this call was already sent);
• The request_id in every response is the only thread for investigating a double charge — keep it.

Error shape

Always OpenAI-compatible:

{ "error": { "message": "…", "type": "…", "param": null, "code": "…" } }

For parameter errors (400) the code is passed straight through from the upstream — those are usually a malformed request body, and the upstream's own wording is more precise than anything we would paraphrase. Every other code, and the next step for each, is in Error codes and rate limits.