Chat

POST /v1/chat/completions takes OpenAI's request and returns OpenAI's response. Point an OpenAI client at this base URL, change the model to a slug from GET /v1/models, and the rest of the code is unchanged. The gateway picks a route that can serve that model, falls back to the next when one fails, and says in the response headers which one answered.

A first request

curl https://api.routehook.ai/v1/chat/completions \
  -H "Authorization: Bearer $ROUTEHOOK_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-4o-mini",
    "messages": [
      { "role": "user", "content": "Hello" }
    ]
  }'

Request parameters

Messages

Each turn carries a role and content. content is a string, or an array of parts for multimodal input: a text part, an image_url part taking a URL or a data URI, and an input_audio part. Part types newer than this gateway are forwarded as they arrive rather than refused. A tool turn carries the tool_call_id it answers; an assistant turn that asked for a tool carries tool_calls and may have content: null.

Model fallback

models is a list of slugs to try after the first, and it is the only fallback lever you hold: the whole chain behind model is exhausted before the second slug is considered at all. Use it where a second model is an acceptable answer to the same question (a cheaper sibling, or another vendor's equivalent), and price it accordingly. The second model has its own rate, so a request that falls through costs what the model that answered costs, not what you budgeted for the first.

Falling back to another model

{
  "model": "openai/gpt-4o-mini",
  "models": ["anthropic/claude-sonnet-4"],
  "messages": [{ "role": "user", "content": "Summarise this in one line." }]
}

What the response headers say

A fallback is otherwise invisible: you get an ordinary 200 with no sign that the first attempt failed. Every response carries headers describing what your own call did. How many attempts, whether it fell back, what it cost, how long the upstream leg took. None of them names the host that answered.

What it cost

The body carries the charge too. usage.cost follows OpenRouter as a JSON number; usage.cost_decimal is Routehook's exact decimal-string extension for accounting code. usage.prompt_tokens_details.cached_tokens and usage.completion_tokens_details.reasoning_tokens are broken out where the upstream reports them, because both are billed differently from ordinary tokens.

The response

{
  "id": "chatcmpl_5f81c0",
  "object": "chat.completion",
  "created": 1786312455,
  "model": "openai/gpt-4o-mini",
  "choices": [
    {
      "index": 0,
      "message": { "role": "assistant", "content": "Hello. How can I help?" },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 9,
    "completion_tokens": 8,
    "total_tokens": 17,
    "cost": "0.0000042"
  }
}

Tool calling

Tools are OpenAI's shape and pass through unchanged. The model answers with finish_reason: "tool_calls" and one or more tool_calls on the assistant message; you run them, append one tool message per call carrying the matching tool_call_id, and send the whole conversation back. Each hop is a separate billed request. A three-hop tool conversation is three completions, not one, and every hop re-sends the growing message list as prompt tokens.

A tool round trip

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.routehook.ai/v1",
  apiKey: process.env.ROUTEHOOK_API_KEY,
});

const tools = [
  {
    type: "function",
    function: {
      name: "get_tide",
      description: "Tide height in metres for a port, right now.",
      parameters: {
        type: "object",
        properties: { port: { type: "string" } },
        required: ["port"],
      },
    },
  },
];

const messages = [{ role: "user", content: "What is the tide at Felixstowe?" }];

const first = await client.chat.completions.create({
  model: "openai/gpt-4o-mini",
  messages,
  tools,
  // Skip any endpoint that has not declared tool support, rather than
  // sending the tools and getting prose back from one that ignores them.
});

const call = first.choices[0].message.tool_calls?.[0];
messages.push(first.choices[0].message);

if (call) {
  const { port } = JSON.parse(call.function.arguments);

  messages.push({
    role: "tool",
    tool_call_id: call.id,
    content: JSON.stringify({ port, metres: 2.4 }),
  });

  const second = await client.chat.completions.create({
    model: "openai/gpt-4o-mini",
    messages,
    tools,
  });

  console.log(second.choices[0].message.content);
}

n multiplies the bill

n asks for that many completions of one prompt, between 1 and 8, and is forwarded verbatim. The prompt is charged once; every completion is charged in full, so n: 4 costs about four times the output side of n: 1. The reservation taken before the call is multiplied to match, which means a large n against a thin balance can be refused with insufficient_credits before anything runs. Not every model honours it: where one does not you get a single choice back, and pay for one.

Failures worth handling

409 model_unavailable means nothing could serve that model. A wrong slug, or a model with no live route right now. 402 insufficient_credits is refused before any upstream call is made, so nothing was spent. 503 upstream_unavailable means every endpoint in the chain failed; it is safe to retry, and sending an Idempotency-Key stops a retry reserving the money a second time.

Related