OpenRouter

deepseek_model_transform(model_id::String)::String

Transform model IDs for DeepSeek. Currently returns unchanged.

siliconflow_model_transform(model_id::String)::String

Map OpenRouter model IDs to SiliconFlow's native IDs (case and prefix differences). Extend this map as SiliconFlow adds or renames models.

AIMessage(schema::AbstractRequestSchema, result::Dict; endpoint=nothing, elapsed=-1.0)

Construct an AIMessage by extracting all fields from raw API result. If endpoint is provided, cost is calculated from token usage.

AbstractLLMStream

Abstract type for LLM stream callbacks.

Must have fields:

• out: Output stream (e.g., stdout or pipe)
• schema: Request schema determining API format
• chunks: List of received AbstractStreamChunk chunks
• verbose: Whether to print verbose information
• kwargs: Custom keyword arguments

Abstract base type for request schemas.

Abstract base type for response schemas.

Abstract base type for all API schemas.

AbstractStreamChunk

Abstract type for stream chunks.

Must have fields:

• event: The event name
• data: The data chunk
• json: The JSON object or nothing if chunk doesn't contain JSON

Anthropic Claude-style request schema.

ChatCompletion schema with Anthropic-style token semantics. prompt_tokens is already non-cached (additive with cache_read).

Standard OpenAI-compatible response schema.

Standard OpenAI-compatible chat completion request schema. This is the default schema used by most providers.

Google Gemini-style request schema.

HttpStreamCallback

HTTP-based streaming callback that prints content to output stream. When streaming completes, builds response body from chunks as if it was a normal API response.

HttpStreamHooks

A stream callback that combines token counting with customizable hooks for various events.

Configuration for model calls, including provider/model slug and call parameters.

Supported Parameters by Schema

ChatCompletionSchema (OpenAI-compatible)

Common parameters supported by most OpenAI-compatible providers:

• temperature::Float64: Sampling temperature (0.0-2.0, default varies by model)
• max_tokens::Int: Maximum tokens to generate
• top_p::Float64: Nucleus sampling threshold (0.0-1.0)
• frequency_penalty::Float64: Penalize frequent tokens (-2.0 to 2.0)
• presence_penalty::Float64: Penalize present tokens (-2.0 to 2.0)
• stop::Union{String, Vector{String}}: Stop sequences
• n::Int: Number of completions to generate
• stream::Bool: Enable streaming (handled automatically by streamcallback)
• logprobs::Bool: Include log probabilities
• top_logprobs::Int: Number of top log probabilities to return
• seed::Int: Random seed for deterministic sampling
• response_format::Dict: Structured output format (e.g., Dict("type" => "json_object"))

AnthropicSchema

Anthropic-specific parameters:

• max_tokens::Int: Maximum tokens to generate (required, default 1000)
• temperature::Float64: Sampling temperature (0.0-1.0)
• top_p::Float64: Nucleus sampling (0.0-1.0)
• top_k::Int: Top-k sampling
• stop_sequences::Vector{String}: Stop sequences
• cache::Symbol: Prompt caching mode (:system, :tools, :last, :all, :all_but_last)
• metadata::Dict: Request metadata

GeminiSchema

Google Gemini-specific parameters:

• temperature::Float64: Sampling temperature
• top_p::Float64 / topP::Float64: Nucleus sampling
• top_k::Int / topK::Int: Top-k sampling
• max_output_tokens::Int / maxOutputTokens::Int: Maximum output tokens
• presence_penalty::Float64 / presencePenalty::Float64: Presence penalty
• frequency_penalty::Float64 / frequencyPenalty::Float64: Frequency penalty
• response_mime_type::String / responseMimeType::String: Output MIME type
• response_schema::Dict / responseSchema::Dict: Output schema
• response_json_schema::Dict / responseJsonSchema::Dict: JSON schema
• stop_sequences::Vector{String} / stopSequences::Vector{String}: Stop sequences
• thinkingConfig::Dict: Thinking/reasoning configuration
  • thinkingLevel::Int: Reasoning depth level
  • thinkingBudget::Int: Token budget for reasoning
  • include_thoughts::Bool: Include reasoning in response
• candidateCount::Int: Number of response candidates
• seed::Int: Random seed
• responseLogprobs::Bool: Include log probabilities
• logprobs::Int: Number of log probabilities

ResponseSchema (OpenAI Response API)

For gpt-5 and o-series models:

• max_completion_tokens::Int: Maximum tokens in completion
• reasoning_effort::String: Reasoning effort level ("low", "medium", "high")
• temperature::Float64: Sampling temperature
• top_p::Float64: Nucleus sampling
• modalities::Vector{String}: Output modalities (e.g., ["text", "audio"])
• audio::Dict: Audio output configuration

Examples

# OpenAI-compatible config
config = ModelConfig("openai:openai/gpt-5.1"; 
    temperature=0.7, 
    max_tokens=1000,
    top_p=0.9
)

# Anthropic with caching
config = ModelConfig("anthropic:anthropic/claude-sonnet-4.5";
    max_tokens=2000,
    temperature=0.8,
    cache=:all
)

# Gemini with thinking
config = ModelConfig("google-ai-studio:google/gemini-2.5-flash";
    temperature=0.7,
    maxOutputTokens=2000,
    thinkingConfig=Dict(
        :thinkingLevel => 2,
        :include_thoughts => true
    )
)

# Modify config later
config.kwargs = merge(config.kwargs, (temperature=0.9,))

ModelConfig(slug::String; schema=nothing, kwargs...)

Create a ModelConfig with the given slug and optional parameters.

Example

config = ModelConfig("openai:openai/gpt-5.1"; temperature=0.7, max_tokens=1000)
response = aigen("Hello", config)

ResponseSchema <: AbstractRequestSchema

Schema for OpenAI's Responses API (v1/responses endpoint). Used by newer models like gpt-5.1 and o-series models.

RunInfo(; creation_time=time(), inference_start=nothing, last_message_time=nothing, stop_sequence=nothing, ttft_ms=nothing)

Tracks run statistics and metadata during the streaming process.

Fields

• creation_time: When the callback was created
• inference_start: When the model started processing
• last_message_time: Timestamp of the last received message
• stop_sequence: The sequence that caused the generation to stop (if any). For OpenAI this can be:
  • A specific stop sequence provided in the chunk's delta.stop_sequence
  • "stop" if finish_reason is "stop"
  For Anthropic this is the stop_sequence provided in the chunk.
• ttft_ms: Time to first token in milliseconds (from provider's sla_metrics if available)

Timing Methods

• get_total_elapsed(info): Get total elapsed time since callback creation
• get_inference_elapsed(info): Get elapsed time for inference phase only

StreamChunk

A chunk of streaming data. A message is composed of multiple chunks.

Fields

• event: The event name
• data: The data chunk
• json: The JSON object or nothing if chunk doesn't contain JSON

TokenCounts

Universal token counting struct. Fields are NON-OVERLAPPING for correct cost calculation.

Fields (non-overlapping)

• prompt_tokens::Int: Cache misses - input tokens NOT served from cache (charged at full price)
• input_cache_read::Int: Cache hits - input tokens served from cache (charged at cache price)
• completion_tokens::Int: Output tokens
• total_tokens::Int: Sum of all input + output tokens
• input_cache_write::Int: Tokens written to cache (Anthropic)
• internal_reasoning::Int: Reasoning/thinking tokens (Gemini, DeepSeek R1)
• input_audio_cache::Int: Audio tokens cached

Cost calculation

Total input = prompttokens + inputcacheread (no double counting) Cost = prompttokens × fullprice + inputcacheread × cacheprice

Tool

Schema-agnostic tool definition. Automatically converted to the right format based on the API schema (OpenAI vs Anthropic vs Gemini).

Example

tool = Tool(
    name = "create_file",
    description = "Create a file with the given content",
    parameters = Dict(
        "type" => "object",
        "properties" => Dict(
            "path" => Dict("type" => "string", "description" => "File path"),
            "content" => Dict("type" => "string", "description" => "File content")
        ),
        "required" => ["path", "content"]
    )
)

Create a ToolMessage from a tool_call dict and string result.

Create a ToolMessage by running fn(args::Dict{String,Any}) with the tool_call's parsed arguments.

GeminiConfig(model_id::String; kwargs...)

Convenience constructor for Gemini models with common parameter validation.

Gemini-specific Parameters

• temperature::Float64: Sampling temperature (0.0-2.0)
• topP::Float64: Nucleus sampling (0.0-1.0)
• topK::Int: Top-k sampling
• maxOutputTokens::Int: Maximum output tokens
• thinkingConfig::NamedTuple: Thinking configuration with fields:
  • thinkingLevel::String: Reasoning depth ("low" or "high") - only for Pro models
  • thinkingBudget::Int: Token budget for reasoning - for non-Pro models
  • include_thoughts::Bool: Include reasoning in response

Examples

# Basic Gemini config
config = GeminiConfig("google/gemini-2.5-flash"; 
    temperature=0.7,
    maxOutputTokens=2000
)

# Pro model with thinking level
config = GeminiConfig("google/gemini-2.5-pro";
    temperature=0.8,
    maxOutputTokens=3000,
    thinkingConfig=(
        thinkingLevel="high",
        include_thoughts=true
    )
)

# Non-Pro model with thinking budget
config = GeminiConfig("google/gemini-2.5-flash-thinking";
    thinkingConfig=(
        thinkingBudget=1000,
        include_thoughts=true
    )
)

response = aigen("Explain quantum entanglement", config)

Core function that handles both streaming and non-streaming API calls.

_list_models_unfiltered(api_key::String = get(ENV, "OPENROUTER_API_KEY", ""))::Vector{OpenRouterModel}

Internal helper: return all models without any provider-based filtering, using the raw API. Used by update_db to avoid recursive use of the cache.

Accumulate tokens according to schema-specific logic.

Schema-specific behavior

• AnthropicSchema: Replaces values (Anthropic sends cumulative counts)
• Other schemas: Adds values (most providers send deltas)

Accumulate tokens for ChatCompletion schema. Detects cumulative counts (DeepSeek style) vs delta counts and handles accordingly.

add_custom_model(id::String, name::String, description::String="Custom model",
                context_length::Union{Int,Nothing}=nothing,
                pricing::Union{Pricing,Nothing}=nothing,
                architecture::Union{Architecture,Nothing}=nothing)

Add a custom model to the local cache.

Example

add_custom_model("echo/100tps", "Echo 100 TPS", "Fast echo model for testing", 8192)
add_custom_model("local/llama3", "Local Llama 3", "Self-hosted Llama 3", 4096)

add_model(id::String, name::String, description::String="Custom model",
         context_length::Union{Int,Nothing}=nothing,
         pricing::Union{Pricing,Nothing}=nothing,
         architecture::Union{Architecture,Nothing}=nothing)

Add a model to the local cache.

Example

add_model("echo/100tps", "Echo 100 TPS", "Fast echo model for testing", 8192)
add_model("ollama/llama3", "Local Llama 3", "Self-hosted Llama 3", 4096)

add_provider(name::String, base_url::String, auth_header_format::String="Bearer",
            api_key_env_var::Union{String,Nothing}=nothing,
            default_headers::Dict{String,String}=Dict{String,String}(),
            model_name_transform::Union{Function,Nothing}=nothing,
            schema::AbstractRequestSchema=ChatCompletionSchema(),
            notes::String="Custom provider")

Add a provider to the registry. Warns if overwriting an existing provider.

Example

add_provider("echo", "http://localhost:8080/v1", "Bearer", "ECHO_API_KEY")
add_provider("ollama", "http://localhost:11434/v1", "Bearer")

aigen(prompt, provider_model::String; ...)
aigen(prompt, config::ModelConfig; ...)

Generate text using a specific provider and model, or a ModelConfig.

Arguments

• prompt: The input prompt (String or Vector of message dicts)
• provider_model::String: Format "Provider:model/slug" (e.g., "Together:moonshotai/kimi-k2-thinking")
• config::ModelConfig: Model configuration with slug and parameters

Keyword Arguments

• schema::Union{AbstractRequestSchema, Nothing}: Request schema to use (auto-detected if not provided)
• api_key::Union{String, Nothing}: Provider-specific API key (auto-detected from env if not provided)
• sys_msg: System message/instruction
• streamcallback::Union{Nothing, AbstractLLMStream}: Stream callback for real-time processing
• kwargs...: Additional API parameters

Returns

• AIMessage: Generated response with metadata (cost, tokens, etc.)

Example

# Using string slug
response = aigen("Write a haiku about Julia programming", "Together:moonshotai/kimi-k2-thinking")

# Using ModelConfig
config = ModelConfig("openai:openai/gpt-5.1"; temperature=0.7, max_tokens=1000)
response = aigen("Hello", config)

# Using system message
response = aigen("Hello", "Anthropic:claude-3-sonnet"; sys_msg="You are a helpful assistant")

# Using streaming
using OpenRouter
callback = HttpStreamCallback(; out=stdout)
response = aigen("Count to 10", "anthropic:anthropic/claude-haiku-4.5"; streamcallback=callback)

aigen(; prompt, model, kwargs...)

Kwarg-only version of aigen. ```

aigen_raw(prompt, provider_model::String; 
          schema::Union{AbstractRequestSchema, Nothing} = nothing,
          api_key::Union{String, Nothing} = nothing,
          sys_msg = nothing,
          streamcallback::Union{Nothing, AbstractLLMStream} = nothing,
          kwargs...)

Generate text using a specific provider and model, returning raw API response and parsing components.

This function is useful for:

• Testing equivalence between streaming and non-streaming responses
• Debugging API response formats
• Custom response processing

Returns

• NamedTuple: Contains (result, schema, provider_info, model_id, provider_endpoint, elapsed)

Example

# Compare streaming vs non-streaming raw responses
raw_stream = aigen_raw("Hello", "anthropic:claude-3-sonnet"; streamcallback=HttpStreamCallback())
raw_normal = aigen_raw("Hello", "anthropic:claude-3-sonnet")

anthropic_model_transform(model_id::String)::String

Transform model IDs for Anthropic. Removes anthropic/ prefix and replaces dots with dashes. Also handles special cases and version matching.

atlascloud_model_transform(model_id::String)::String

Transform model IDs for AtlasCloud. Maps DeepSeek models to the provider's deepseek-ai/... namespace.

Build complete headers for a provider request.

Build messages array for AnthropicSchema.

Returns a tuple: (messages, system_content)

Build messages array for ChatCompletionSchema.

Build contents array for GeminiSchema.

Returns a tuple: (contents, system_instruction)

build_native_models_url(provider_info::ProviderInfo)::String

Build the models endpoint URL for a provider's native API.

Build the request payload for AnthropicSchema.

Build the request payload for ChatCompletionSchema.

Build the request payload for GeminiSchema.

build_response_body(schema::AnthropicSchema, cb::AbstractLLMStream; verbose::Bool = false, kwargs...)

Build response body from chunks to mimic standard Anthropic API response. Supports text and tool_use content blocks.

build_response_body(schema::ChatCompletionSchema, cb::AbstractLLMStream; verbose::Bool = false, kwargs...)

Build response body from chunks to mimic standard ChatCompletion API response.

Note: Limited functionality. Does NOT support refusals, logprobs.

build_response_body(schema::GeminiSchema, cb::AbstractLLMStream; verbose::Bool = false, kwargs...)

Build response body from chunks to mimic standard Gemini API response.

build_response_body(schema::ResponseSchema, cb::AbstractLLMStream; verbose::Bool = false, kwargs...)

Build response body from chunks. Optimized to find the final response.completed object immediately (O(1) effectively), with a fallback reconstruction for interrupted streams.

Build the URL for ChatCompletionSchema.

Build the URL for AnthropicSchema.

Build the URL for GeminiSchema (handles model parameter substitution and streaming).

Calculate cost for a given endpoint and token usage. Unwraps .pricing. Warns if cost cannot be determined (e.g. missing pricing).

Calculate cost based on pricing and token usage.

Calculate cost for a given endpoint and token usage. Unwraps .pricing. Warns if cost cannot be determined (e.g. missing pricing).

callback(cb::AbstractLLMStream, chunk::AbstractStreamChunk; kwargs...)

Process chunk and print it. Wrapper for:

• extract content from chunk using extract_content
• print content to output stream using print_content

callback(cb::AbstractLLMStream, chunk::AbstractStreamChunk; kwargs...)

Process chunk and print it. Wrapper for:

• extract content from chunk using extract_content
• print content to output stream using print_content

cerebras_model_transform(model_id::String)::String

Transform model IDs for Cerebras.

cohere_model_transform(model_id::String)::String

Transform model IDs for Cohere. Currently returns unchanged.

configure_stream_callback!(cb::AbstractLLMStream, schema::AbstractRequestSchema, provider_info::ProviderInfo, provider_endpoint::ProviderEndpoint)

Configure stream callback with schema and provider information. For HttpStreamHooks, also sets up pricing for accurate cost calculation.

convert_tools(schema, tools)

Convert tools to schema-specific format. Supports:

• Tool structs (converted per schema)
• Vector{Tool} (each converted)
• Dict/Vector{Dict} (passed through as-is for backward compat)

create_stub_endpoint(provider_name, model_id; pricing=nothing)

Create a stub ProviderEndpoint for local/non-OpenRouter providers.

Create stub with zero pricing (for Ollama etc. where we want cost tracking).

Route request to appropriate response based on endpoint.

extract_chunks(schema::AbstractRequestSchema, blob::AbstractString;
    spillover::AbstractString = "", verbose::Bool = false, kwargs...)

Extract chunks from received SSE blob. Correctly implements SSE spec field parsing.

Extract slug and merge kwargs from ModelConfig with call-time kwargs. Call-time kwargs take precedence over config kwargs.

Extract response content for AnthropicSchema.

extract_content(schema::AnthropicSchema, chunk::AbstractStreamChunk;
    include_thinking::Bool = true, kwargs...)

Extract content from Anthropic chunk.

Extract response content for ChatCompletionSchema.

extract_content(schema::ChatCompletionSchema, chunk::AbstractStreamChunk; kwargs...)

Extract content from ChatCompletion chunk.

Extract response content for GeminiSchema.

extract_content(schema::GeminiSchema, chunk::StreamChunk; kwargs...)

Extract regular (non-reasoning) content from Gemini chunk.

extract_content(schema::ResponseSchema, chunk::AbstractStreamChunk; kwargs...)

Extract content from Response API chunk. Only extracts 'delta' to ensure stream consumers don't print duplicate content (since 'done' events contain the full text).

Extract finish reason from API response based on schema.

Extract generated images from Gemini API response. Returns Vector{String} of base64 data URLs or nothing if no images.

Extract generated images from Response API output. Returns Vector{String} of base64 data URLs or nothing if no images.

extract_provider_from_model(model_name::String) -> String

Extract provider name from model name in format "provider:author/model_id" or fallback to "openai".

Examples

extract_provider_from_model("openai:openai/gpt-4") # => "openai"
extract_provider_from_model("anthropic:anthropic/claude-3-5-sonnet") # => "anthropic"
extract_provider_from_model("cerebras:meta-llama/llama-3.1-8b") # => "cerebras"
extract_provider_from_model("gpt-4") # => "openai" (fallback)

Extract reasoning content from API response based on schema. Returns nothing if schema doesn't support reasoning or no reasoning found.

extract_reasoning_from_chunk(schema::ChatCompletionSchema, chunk::AbstractStreamChunk)

Extract reasoning_content from ChatCompletion chunk (DeepSeek style).

extract_reasoning_from_chunk(schema::GeminiSchema, chunk::StreamChunk)

Extract reasoning/thinking content from Gemini chunk (parts with "thought": true).

Extract full response for ChatCompletionResponseSchema.

Extract token usage information from API response based on schema. Returns TokenCounts struct with standardized field names.

Extract tool calls from API response based on schema. Returns nothing if no tool calls found.

Extract ttftms from slametrics (DeepSeek/OpenRouter style).

fetch_native_models(provider_info::ProviderInfo, api_key::String)::Vector{Dict}

Fetch models directly from a provider's native API. Returns raw model data as returned by the provider.

fireworks_model_transform(model_id::String)::String

Transform model IDs for Fireworks. Currently returns unchanged.

Parse the arguments from a tool_call dict, handling both JSON string and already-parsed Dict.

get_inference_elapsed(info::RunInfo)

Get elapsed time for inference (time between first inference and last message). Returns time in seconds or nothing if inference hasn't started.

Build an auth header pair (name => value) for a provider + API key, or nothing if provider is unknown.

Build an auth header pair (name => value) for a ProviderInfo + API key.

Get just the base URL for a provider slug, or nothing if unknown.

Return the configured API key env var name for a provider, or nothing.

Get the provider info for a given slug, or nothing if unknown.

Get the appropriate schema for a provider info and model.

get_total_elapsed(info::RunInfo)

Get total elapsed time since callback creation. Returns time in seconds or nothing if no messages received.

google_model_transform(model_id::String)::String

Transform model IDs for Google. Removes google/ prefix.

groq_model_transform(model_id::String)::String

Transform OpenRouter model IDs to Groq-specific model IDs. Handles various model mappings for Groq's native API.

Examples

groq_model_transform("moonshotai/kimi-k2-0905")  # => "moonshotai/kimi-k2-instruct-0905"
groq_model_transform("other/model")              # => "other/model"

handle_error_message(chunk::AbstractStreamChunk; kwargs...)

Handle error messages from streaming response. Always throws on error.

is_done(schema::ChatCompletionSchema, chunk::AbstractStreamChunk; kwargs...)

Check if streaming is done for ChatCompletion format. Checks for finish_reason in choices or [DONE] marker.

is_done(schema::ResponseSchema, chunk::AbstractStreamChunk; kwargs...)

Check if streaming is done for Response API format.

Check if this provider slug is known.

is_start(schema::AnthropicSchema, chunk::AbstractStreamChunk; kwargs...)

Check if streaming has started for Anthropic format.

is_start(schema::ChatCompletionSchema, chunk::AbstractStreamChunk; kwargs...)

Check if streaming has started for ChatCompletion format.

is_start(schema::GeminiSchema, chunk::AbstractStreamChunk; kwargs...)

Check if streaming has started for Gemini format. Gemini doesn't have explicit start events, so we check for first content with reasoning (thought=true).

is_start(schema::ResponseSchema, chunk::AbstractStreamChunk; kwargs...)

Check if streaming has started for Response API format.

list_aliases()::Dict{String, String}

List all available model aliases.

list_config_parameters(config::ModelConfig)

List parameters currently set in a ModelConfig.

Example

config = ModelConfig("openai:openai/gpt-5.1"; temperature=0.7, max_tokens=1000)
list_config_parameters(config)

list_embeddings_models(api_key::String = get(ENV, "OPENROUTER_API_KEY", ""))::Vector{OpenRouterEmbeddingModel}

Return parsed embedding models list as Julia structs. Uses OPENROUTERAPIKEY environment variable by default.

list_embeddings_models_raw(api_key::String = get(ENV, "OPENROUTER_API_KEY", ""))::String

Return raw JSON string of embedding models list. Uses OPENROUTERAPIKEY environment variable by default.

list_endpoints(model_id::String, api_key::String = get(ENV, "OPENROUTER_API_KEY", ""))::ModelProviders

Return parsed endpoints for a specific model from OpenRouter as Julia struct. Model ID should be in format "author/slug" (e.g., "moonshotai/kimi-k2-thinking"). Uses OPENROUTERAPIKEY environment variable by default.

list_endpoints_raw(model_id::String, api_key::String = get(ENV, "OPENROUTER_API_KEY", ""))::String

Return raw JSON string of endpoints for a specific model from OpenRouter. Model ID should be in format "author/slug" (e.g., "moonshotai/kimi-k2-thinking"). Uses OPENROUTERAPIKEY environment variable by default.

List all known provider slugs.

list_models(provider_filter::Union{String, Nothing} = nothing, api_key::String = get(ENV, "OPENROUTER_API_KEY", ""))::Vector{OpenRouterModel}

Return parsed model list as Julia structs, optionally filtered by provider. Uses OPENROUTERAPIKEY environment variable by default.

list_models_raw(api_key::String = get(ENV, "OPENROUTER_API_KEY", ""))::String

Return raw JSON string of model list. Uses OPENROUTERAPIKEY environment variable by default.

list_native_models(provider_slug::String, api_key::String = "")::Vector{Dict}

List models using a provider's native API. Returns raw model data as returned by the provider.

Example

models = list_native_models("cerebras")
models = list_native_models("openai", "your-api-key")

list_provider_endpoints(provider_filter::String, api_key::String = get(ENV, "OPENROUTER_API_KEY", ""))::Vector{ProviderEndpoint}

Return all ProviderEndpoint entries hosted by the given provider.

This uses the cached model database with endpoints; it will fetch endpoints as needed on first call.

Example:

groq_eps = list_provider_endpoints("groq")
for ep in groq_eps
    println(ep.provider_name, " ", ep.name, " (", ep.model_name, ")")
end

list_providers(model_id::String, api_key::String = get(ENV, "OPENROUTER_API_KEY", ""))::ModelProviders

Return parsed providers for a specific model as Julia struct. Model ID should be in format "author/slug" (e.g., "moonshotai/kimi-k2-thinking"). Uses OPENROUTERAPIKEY environment variable by default.

list_providers()

List all registered providers.

list_providers_raw(model_id::String, api_key::String = get(ENV, "OPENROUTER_API_KEY", ""))::String

Return raw JSON string of providers for a specific model. ...

list_schema_parameters(schema::Type{<:AbstractRequestSchema})
list_schema_parameters(schema::AbstractRequestSchema)

List common parameters supported by a schema type.

Example

list_schema_parameters(ChatCompletionSchema)
list_schema_parameters(AnthropicSchema)
list_schema_parameters(GeminiSchema)

minimax_model_transform(model_id::String)::String

Transform model IDs for Minimax. Currently returns unchanged.

mistral_model_transform(model_id::String)::String

Transform model IDs for Mistral. Currently returns unchanged.

moonshotai_model_transform(model_id::String)::String

Transform model IDs for MoonshotAI. Currently returns unchanged.

needs_tool_execution(info::RunInfo)

Check if the run was terminated because the model is requested tool execution (with stop_sequence).

Normalize prompt + sys_msg into a flat vector of AbstractMessage.

Accepted prompt forms:

• String => one UserMessage
• AbstractMessage => wrapped in a vector
• Vector of items => each element may be
  • AbstractMessage
  • String => treated as UserMessage
  • anything else => treated as UserMessage with that content

ollama_model_transform(model_id::String)::String

Transform model IDs for Ollama. Currently returns unchanged.

openai_model_transform(model_id::String)::String

Transform model IDs for OpenAI. Removes openai/ prefix and handles specific mappings.

parse_embedding_models(json_str::String)::Vector{OpenRouterEmbeddingModel}

Parse OpenRouter embedding models JSON response into Julia structs.

parse_endpoints(json_str::String)::ModelProviders

Parse model providers JSON response into Julia struct.

parse_models(json_str::String)::Vector{OpenRouterModel}

Parse OpenRouter models JSON response into Julia structs.

print_content(out::Channel, text::AbstractString; kwargs...)

Print content to Channel.

print_content(out::IO, text::AbstractString; kwargs...)

Print content to IO output stream.

print_content(out::Nothing, text::Any; kwargs...)

Do nothing if output stream is nothing.

remove_custom_model(id::String)

Remove a custom model from the local cache.

remove_model(id::String)

Remove a model from the local cache.

remove_provider(name::String)

Remove a provider from the registry.

resolve_model_alias(model_id::String)::String

Resolve a model alias to the full provider:model format. If the input is not an alias, returns it unchanged.

Example

resolve_model_alias("gemf")  # Returns "google-ai-studio:google/gemini-2.5-flash-preview-09-2025"
resolve_model_alias("anthropic:claude-3-sonnet")  # Returns unchanged

sambanova_model_transform(model_id::String)::String

Transform model IDs for SambaNova. Currently returns unchanged.

set_provider!(name::String, base_url::String, auth_header_format::String="Bearer",
             api_key_env_var::Union{String,Nothing}=nothing,
             default_headers::Dict{String,String}=Dict{String,String}(),
             model_name_transform::Union{Function,Nothing}=nothing,
             schema::AbstractRequestSchema=ChatCompletionSchema(),
             notes::String="Custom provider")

Set/override a provider in the registry. Use this when intentionally overwriting an existing provider.

streamed_request!(cb::AbstractLLMStream, url, headers, input; kwargs...)

End-to-end wrapper for POST streaming requests. Modifies callback object (cb.chunks) in-place and returns response object.

strip_provider_prefix(model_id::AbstractString, provider::AbstractString)::AbstractString

Remove provider prefix from model ID if present. Helper function for model transformations.

Examples

strip_provider_prefix("openai/gpt-4", "openai")     # => "gpt-4"
strip_provider_prefix("gpt-4", "openai")            # => "gpt-4"
strip_provider_prefix("google/gemini-pro", "google") # => "gemini-pro"

Swap echo provider ports from from to to.

together_model_transform(model_id::String)::String

Transform model IDs for Together. Currently returns unchanged.

Transform model name according to provider-specific rules.

Run function with echo server on given port.

xai_model_transform(model_id::String)::String

Transform model IDs for xAI. Removes x-ai/ prefix and handles specific mappings.