Tool calling

Let the model call your functions with tools, tool_choice and tool messages.

Tool calling works as in the OpenAI API. With a non-empty tools list the router sends the request only to routes that support tools; models with the Tools capability in the catalog do.

1. Describe the tools

{
  "model": "openai/gpt-4o-mini",
  "messages": [{ "role": "user", "content": "What is the weather in Kazan?" }],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Current weather in a city",
        "parameters": {
          "type": "object",
          "properties": { "city": { "type": "string" } },
          "required": ["city"]
        }
      }
    }
  ],
  "tool_choice": "auto"
}

2. Run the call the model asks for

The answer ends with finish_reason: "tool_calls":

{
  "role": "assistant",
  "content": null,
  "tool_calls": [
    { "id": "call_1", "type": "function", "function": { "name": "get_weather", "arguments": "{\"city\":\"Kazan\"}" } }
  ]
}

arguments is a JSON string: parse it and validate it before you run anything.

3. Send the result back

Repeat the conversation with the assistant message and a tool message for each call, then send the request again with the same tools:

[
  { "role": "user", "content": "What is the weather in Kazan?" },
  { "role": "assistant", "content": null, "tool_calls": [ { "id": "call_1", "type": "function", "function": { "name": "get_weather", "arguments": "{\"city\":\"Kazan\"}" } } ] },
  { "role": "tool", "tool_call_id": "call_1", "content": "{\"temp_c\": 12, \"sky\": \"cloudy\"}" }
]

Options

Field	Values
`tool_choice`	`auto` (default), `none`, `required`, or `{"type": "function", "function": {"name": "get_weather"}}`
`parallel_tool_calls`	`false` asks for at most one call per answer

In a stream, delta.tool_calls arrives in pieces: join function.arguments by index until the chunk with finish_reason: "tool_calls".

Tool definitions are part of the prompt: they are billed as input tokens on every request.