Chat Completions

Endpoint

POST /v1/chat/completions

Send a conversation to an AI model and receive a text response. Compatible with the OpenAI Chat Completions API.

Request Body

string

required

The model to use for completion. Use provider prefix for explicit routing (e.g., geminicli:gemini-2.5-pro) or omit prefix for auto-routing.

array

required

Array of message objects forming the conversation.

Show Message Object Properties

string

required

The role of the message author: system, user, or assistant

string | array

required

The message content. Can be a string or array of content parts for multimodal inputs.

string

Optional name for the message author

boolean

default:false

If true, returns a stream of server-sent events instead of a single response.

number

Sampling temperature between 0 and 2. Higher values make output more random.

integer

Maximum number of tokens to generate in the completion.

number

Nucleus sampling parameter. Alternative to temperature.

number

default:0

Penalize tokens based on frequency in the text so far (-2.0 to 2.0).

number

default:0

Penalize tokens based on presence in the text so far (-2.0 to 2.0).

string | array

Up to 4 sequences where the API will stop generating tokens.

array

List of tools the model may call. Currently supports function calling.

object

Provider-specific extensions. See CLI Attachments for CLI provider options.

Response Format

string

Unique identifier for the completion

string

Object type, always chat.completion or chat.completion.chunk for streaming

integer

Unix timestamp of when the completion was created

string

The model used for completion

array

Array of completion choices

Show Choice Object Properties

integer

The index of this choice in the array

object

The generated message

Show Message Properties

string

Always assistant

string

The generated text content

array

Tool calls made by the model, if any

string

Why the completion stopped: stop, length, tool_calls, or content_filter

object

Token usage statistics

Show Usage Properties

integer

Number of tokens in the prompt

integer

Number of tokens in the completion

integer

Total tokens used

Examples

Basic Request

curl http://localhost:18080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-test-123" \
  -d '{
    "model": "gemini-2.5-pro",
    "messages": [
      {"role": "user", "content": "What is the capital of France?"}
    ]
  }'

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:18080/v1",
    api_key="sk-test-123"
)

response = client.chat.completions.create(
    model="gemini-2.5-pro",
    messages=[
        {"role": "user", "content": "What is the capital of France?"}
    ]
)

print(response.choices[0].message.content)

import OpenAI from 'openai';

const client = new OpenAI({
  baseURL: 'http://localhost:18080/v1',
  apiKey: 'sk-test-123'
});

const response = await client.chat.completions.create({
  model: 'gemini-2.5-pro',
  messages: [
    { role: 'user', content: 'What is the capital of France?' }
  ]
});

console.log(response.choices[0].message.content);

Streaming Response

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:18080/v1",
    api_key="sk-test-123"
)

stream = client.chat.completions.create(
    model="gemini-2.5-pro",
    messages=[{"role": "user", "content": "Tell me a story"}],
    stream=True
)

for chunk in stream:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)

import OpenAI from 'openai';

const client = new OpenAI({
  baseURL: 'http://localhost:18080/v1',
  apiKey: 'sk-test-123'
});

const stream = await client.chat.completions.create({
  model: 'gemini-2.5-pro',
  messages: [{ role: 'user', content: 'Tell me a story' }],
  stream: true
});

for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content || '');
}

curl http://localhost:18080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-test-123" \
  -d '{
    "model": "gemini-2.5-pro",
    "messages": [{"role": "user", "content": "Tell me a story"}],
    "stream": true
  }'

Multi-Turn Conversation

messages = [
    {"role": "system", "content": "You are a helpful coding assistant."},
    {"role": "user", "content": "Write a Python function to calculate fibonacci."},
    {"role": "assistant", "content": "Here's a recursive implementation..."},
    {"role": "user", "content": "Now make it iterative."}
]

response = client.chat.completions.create(
    model="geminicli:gemini-2.5-pro",
    messages=messages
)

const messages = [
  { role: 'system', content: 'You are a helpful coding assistant.' },
  { role: 'user', content: 'Write a Python function to calculate fibonacci.' },
  { role: 'assistant', content: "Here's a recursive implementation..." },
  { role: 'user', content: 'Now make it iterative.' }
];

const response = await client.chat.completions.create({
  model: 'geminicli:gemini-2.5-pro',
  messages
});

Provider-Specific Routing

Auto-Routing
Gemini CLI
Claude CLI
Ollama

# Let switchAILocal choose the best provider
curl http://localhost:18080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-test-123" \
  -d '{
    "model": "gemini-2.5-pro",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

# Route to Gemini CLI (uses your CLI subscription)
curl http://localhost:18080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-test-123" \
  -d '{
    "model": "geminicli:gemini-2.5-pro",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

# Route to Claude CLI
curl http://localhost:18080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-test-123" \
  -d '{
    "model": "claudecli:claude-sonnet-4",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

# Route to local Ollama
curl http://localhost:18080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer sk-test-123" \
  -d '{
    "model": "ollama:llama3.2",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

Advanced Features

Temperature Control

Adjust randomness of responses:

response = client.chat.completions.create(
    model="gemini-2.5-pro",
    messages=[{"role": "user", "content": "Write a creative story"}],
    temperature=1.5  # Higher = more creative
)

Token Limits

Constrain response length:

response = client.chat.completions.create(
    model="gemini-2.5-pro",
    messages=[{"role": "user", "content": "Summarize quantum physics"}],
    max_tokens=100  # Limit to 100 tokens
)

Stop Sequences

Stop generation at specific strings:

response = client.chat.completions.create(
    model="gemini-2.5-pro",
    messages=[{"role": "user", "content": "List 5 colors:"}],
    stop=["\n\n", "6."]  # Stop at double newline or "6."
)

Error Handling

from openai import OpenAI, APIError, RateLimitError

client = OpenAI(
    base_url="http://localhost:18080/v1",
    api_key="sk-test-123"
)

try:
    response = client.chat.completions.create(
        model="invalid-model",
        messages=[{"role": "user", "content": "Hello"}]
    )
except RateLimitError:
    print("Rate limit exceeded")
except APIError as e:
    print(f"API error: {e.message}")

import OpenAI from 'openai';

const client = new OpenAI({
  baseURL: 'http://localhost:18080/v1',
  apiKey: 'sk-test-123'
});

try {
  const response = await client.chat.completions.create({
    model: 'invalid-model',
    messages: [{ role: 'user', content: 'Hello' }]
  });
} catch (error) {
  if (error.status === 404) {
    console.log('Model not found');
  } else {
    console.error('API error:', error.message);
  }
}

Next Steps

Provider Prefixes

Learn about routing to specific providers

CLI Attachments

Pass files and folders to CLI providers

Models

List and discover available models

WebSocket

Use real-time bidirectional streaming

Overview

Endpoints

Provider Formats

Management API

Endpoint

Request Body

Response Format

Examples

Basic Request

Streaming Response

Multi-Turn Conversation

Provider-Specific Routing

Advanced Features

Temperature Control

Token Limits

Stop Sequences

Error Handling

Next Steps

Provider Prefixes

CLI Attachments

Models

WebSocket

​Endpoint

​Request Body

​Response Format

​Examples

​Basic Request

​Streaming Response

​Multi-Turn Conversation

​Provider-Specific Routing

​Advanced Features

​Temperature Control

​Token Limits

​Stop Sequences

​Error Handling

​Next Steps

Provider Prefixes

CLI Attachments

Models

WebSocket

Endpoint

Request Body

Response Format

Examples

Basic Request

Streaming Response

Multi-Turn Conversation

Provider-Specific Routing

Advanced Features

Temperature Control

Token Limits

Stop Sequences

Error Handling

Next Steps