Tool Calling

Let the model call functions you define. Tool calling is supported across both the OpenAI-compatible and Anthropic-compatible APIs.

OpenAI Format

In the OpenAI-compatible API (/v1/chat/completions), tools are defined using a tools array where each entry has type: "function" and a function object describing the name, description, and parameters.

Control tool invocation with tool_choice:

Set parallel_tool_calls: true to allow the model to make multiple tool calls in a single response.

Tool Definition

Field Type Description
type string Always "function"
function.name string Function name
function.description string Description of what the function does
function.parameters object JSON Schema for the function parameters
function.strict boolean Enable strict schema adherence

Tool Call Response

When the model decides to call a tool, the response will have finish_reason: "tool_calls" and the assistant message will contain a tool_calls array:

{
  "id": "call_abc123",
  "type": "function",
  "function": {
    "name": "get_weather",
    "arguments": "{\"location\": \"San Francisco\"}"
  }
}

Sending Tool Results

After executing the function, send the result back by adding a message with role: "tool", the matching tool_call_id, and content containing the result.

Each tool call must be answered with a corresponding tool message. The tool_call_id must match the id from the tool call.

Anthropic Format

In the Anthropic-compatible API (/v1/messages), tools are defined using a tools array where each entry has a name, description, and input_schema.

Control tool invocation with tool_choice:

Tool Definition

Field Type Description
name string Tool name
description string Tool description
input_schema object JSON Schema for tool input

Tool Use Response

When the model decides to call a tool, the response will have stop_reason: "tool_use" and the content array will contain a tool_use block:

{
  "type": "tool_use",
  "id": "toolu_abc123",
  "name": "get_weather",
  "input": {"location": "San Francisco"}
}

Sending Tool Results

After executing the function, send the result back by adding a user message with a tool_result content block:

{
  "type": "tool_result",
  "tool_use_id": "toolu_abc123",
  "content": "72°F, sunny"
}

The tool_use_id must match the id from the tool_use content block.

Full Multi-Turn Flow

A complete tool calling interaction follows this pattern, illustrated with a weather lookup example:

Step 1: Send a request with tools defined and a user message (e.g., "What's the weather in San Francisco?").

Step 2: The model responds with a tool call instead of a text answer. The response has finish_reason: "tool_calls" (OpenAI) or stop_reason: "tool_use" (Anthropic) and includes the function name and arguments.

Step 3: Your application executes the function (e.g., calls a weather API with location: "San Francisco") and obtains the result.

Step 4: Send the tool result back to the model. Include the original messages, the assistant's tool call message, and a new tool result message with the function output.

Step 5: The model generates a final natural language response incorporating the tool result (e.g., "The weather in San Francisco is 72 degrees and sunny.").

Examples

Non-Streaming Tool Call (OpenAI Format)

curl https://api.routerhub.ai/v1/chat/completions \
  -H "Authorization: Bearer $ROUTERHUB_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "anthropic/claude-sonnet-4",
    "messages": [
      {"role": "user", "content": "What is the weather in San Francisco?"}
    ],
    "tools": [
      {
        "type": "function",
        "function": {
          "name": "get_weather",
          "description": "Get the current weather for a location",
          "parameters": {
            "type": "object",
            "properties": {
              "location": {"type": "string", "description": "City name"}
            },
            "required": ["location"]
          }
        }
      }
    ]
  }'
from openai import OpenAI
import json

client = OpenAI(
    base_url="https://api.routerhub.ai/v1",
    api_key="YOUR_API_KEY",
)

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the current weather for a location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {"type": "string", "description": "City name"}
                },
                "required": ["location"],
            },
        },
    }
]

response = client.chat.completions.create(
    model="anthropic/claude-sonnet-4",
    messages=[{"role": "user", "content": "What's the weather in SF?"}],
    tools=tools,
)

# Handle tool calls
if response.choices[0].finish_reason == "tool_calls":
    tool_call = response.choices[0].message.tool_calls[0]
    args = json.loads(tool_call.function.arguments)

    # Execute your function
    result = get_weather(args["location"])

    # Send results back
    follow_up = client.chat.completions.create(
        model="anthropic/claude-sonnet-4",
        messages=[
            {"role": "user", "content": "What's the weather in SF?"},
            response.choices[0].message,
            {
                "role": "tool",
                "tool_call_id": tool_call.id,
                "content": json.dumps(result),
            },
        ],
        tools=tools,
    )
    print(follow_up.choices[0].message.content)
from anthropic import Anthropic

client = Anthropic(
    base_url="https://api.routerhub.ai",
    api_key="YOUR_API_KEY",
)

response = client.messages.create(
    model="anthropic/claude-sonnet-4",
    max_tokens=1024,
    tools=[
        {
            "name": "get_weather",
            "description": "Get the current weather for a location",
            "input_schema": {
                "type": "object",
                "properties": {
                    "location": {"type": "string", "description": "City name"}
                },
                "required": ["location"],
            },
        }
    ],
    messages=[{"role": "user", "content": "What's the weather in SF?"}],
)

# Handle tool use
if response.stop_reason == "tool_use":
    tool_block = next(b for b in response.content if b.type == "tool_use")

    # Execute your function
    result = get_weather(tool_block.input["location"])

    # Send results back
    follow_up = client.messages.create(
        model="anthropic/claude-sonnet-4",
        max_tokens=1024,
        tools=[...],  # same tools
        messages=[
            {"role": "user", "content": "What's the weather in SF?"},
            {"role": "assistant", "content": response.content},
            {
                "role": "user",
                "content": [
                    {
                        "type": "tool_result",
                        "tool_use_id": tool_block.id,
                        "content": str(result),
                    }
                ],
            },
        ],
    )
    print(follow_up.content[0].text)
from langchain_openai import ChatOpenAI
from langchain_core.tools import tool

@tool
def get_weather(location: str) -> str:
    """Get the current weather for a location."""
    return "72°F, sunny"

llm = ChatOpenAI(
    base_url="https://api.routerhub.ai/v1",
    api_key="YOUR_API_KEY",
    model="anthropic/claude-sonnet-4",
)

llm_with_tools = llm.bind_tools([get_weather])
response = llm_with_tools.invoke("What's the weather in SF?")
print(response.tool_calls)

Streaming Tool Calls

Tool calls are also supported in streaming mode. The behavior differs slightly from regular text streaming:

In the OpenAI format, tool call arguments arrive as deltas across multiple chunks. Each chunk contains a tool_calls array in the delta with partial data:

Do not attempt to parse function.arguments until the stream is complete and finish_reason is "tool_calls". The partial strings are not valid JSON on their own.

In the Anthropic format, tool use arguments stream via content_block_delta events with input_json_delta deltas. The partial JSON fragments must be concatenated similarly before parsing.