Multi-Provider AI Configuration ​

Add up to 20 AI providers for Graphne, route each task type to one with fallbacks, and track what the AI calls cost.

Requires role: Admin Related: AI Evidence Trail | AI Safety Controls | Multi-Agent Intelligence Service | Investigations | Full-Auto Execution | User Management & RBAC

Overview ​

The Admin > AI Provider page sets up the AI providers that Graphne, investigations and AI triage call. Add up to 20 providers, each under a name you choose. Route each task type to a primary provider, with fallbacks in order. Use a capable model for investigations and a cheaper one for alert triage, for example.

When a provider fails, the request moves to the next provider in the chain. Each provider has a circuit breaker. After three errors in a row within a minute, it pauses the provider for 30 seconds, and requests skip it.

GraphnAI records the tokens and cost of every AI call. The Cost tab shows this month's cost by agent and by provider type, against a monthly budget you set. You can also send a trace of each call to any OTLP collector, such as Langfuse, Datadog or Phoenix.

The AI Provider Page ​

TabWhat it holds
SafetyTurn AI off, stop running AI work and choose which Graphne tools each role can use. See AI Safety Controls.
ProvidersYour providers, task routing, model prices and prompt versions
HealthEach provider's state, request count and uptime
CostThis month's cost and tokens, the monthly budget and the hard spending ceiling
AutonomyFull-Auto settings. See Full-Auto Execution.
TriageWhat AI triage does for each detection rule
FeedbackHow people rated Graphne's answers in the last 30 days
EvidenceEvery Graphne answer and AI task, with its models, instructions, graph reads and results. See AI Evidence Trail.
GuardrailsThe safety list that automation never acts on, and the fix format check

Stop all automation, above the tabs, stops every automatic action at once: fixes, alert closes and investigations. It also stops automatic work that's running now. Graphne still answers questions.

Prerequisites ​

  • Role: Admin. Anyone else sees "Admins only" on the page.
  • License: A license that includes the AI features, to add, edit or remove providers and to change routing or prices. Without it, the page shows saved providers but locks the buttons. Saving a budget needs Full-Auto in the license.
  • API keys: Each cloud provider needs its own API key from that provider. A saved key is stored encrypted and shown only masked.

Configuring Multiple Providers ​

Each provider has a name you choose, a provider type, a model and an API key. Routing rules refer to providers by name.

Provider Types and Example Models ​

Provider typeExample modelNotes
OpenAIgpt-6-solPrompt caching is automatic.
Google Geminigemini-3.8-flashUses the provider's implicit caching. Explicit caching isn't available yet.
Anthropic Claudeclaude-sonnet-5-5GraphnAI marks the system prompt for caching. A cached read costs 2.5% to 10% of the input price, depending on the model.
Local (LM Studio / Ollama)Your choiceThe API key is optional. Enter one only if your local server requires it. For air-gapped networks. Add a price for it to track its cost (see Setting your own model prices).

The example model is the one the Model box suggests. Enter the model ID your provider account can call.

Adding Providers via the Admin UI ​

  1. Sign in with an Admin account.
  2. Select Admin > AI Provider in the sidebar.
  3. On the Providers tab, select Add provider.
  4. Fill in the dialog:
    • Name: A label for this provider in routing rules, for example anthropic-primary or gemini-batch. You can't change it later.
    • Provider: Choose the provider type.
    • Model: The model identifier for this provider.
    • API key: Your API key. For a local provider, the key is optional. Enter one only if your local server requires it, and enter the local model's base URL in Endpoint URL.
  5. Select Add provider.

The provider is used from the next request. The page has no connection test. Check the provider on the Health tab after its first requests.

The first provider you add becomes the primary for the Graphne task. Repeat to add up to 20 providers.

The Endpoint URL must be a full web address that starts with http:// or https://, for example http://localhost:1234/v1. A malformed address is refused when you save. An Endpoint URL on the server itself (a loopback address) is accepted for a local provider. That's how you tell GraphnAI the model runs on the same machine. A link-local address (169.254.x.x) or a cloud instance metadata endpoint is refused. A refused save names the provider and says why. When GraphnAI can't check the address, the provider is saved with a warning.

Editing or Removing a Provider ​

Select Edit on a provider's card to change its type, model, key or endpoint. Leave API key blank to keep the saved key. If you change the provider type, enter the new provider's key.

A card marked Re-enter API key has a key the server can't read after a security key change. Select Edit and enter the key again.

Select Remove on a provider's card to delete it. It's removed at once, with no confirmation. Its routing goes with it:

  • It's dropped from every fallback list.
  • A task that used it as primary moves to that task's first fallback, or to your first remaining provider if it had none.
  • With no provider left, that task's routing rule is deleted.

Task-Type Routing and Fallback Chains ​

With more than one provider, route each task type to a primary provider, and list fallbacks in the order to try them.

Task Types ​

TaskWhat it covers
GraphneGraphne's answers, including the graph queries it writes
Alert triageAI triage of detection alerts
InvestigationInvestigations and finding explanations
RemediationSelect Fire's risk analysis, remediation scripts and plan summaries
Batch analyticsThe written explanation of an Operational Safety Metric™ score
Knowledge searchNot used yet. Knowledge base search doesn't call a model, so this route has no effect.

A task with no routing rule tries every provider, in the order you added them.

Choosing a Model for Each Task ​

Each task calls the model differently, so each one suits a different model. Use the notes below to pick a primary for each route.

  • Graphne. It makes several model calls per question, up to 9. It decides which tool to use, then writes graph queries for the query tool. It runs at low reasoning effort. Pick a mid-tier model that follows instructions and writes queries reliably at moderate cost.
  • Alert triage. One short call per detection alert, with a few hundred words of reasoning. Alerts arrive in bursts. Pick a small, fast, cheap model.
  • Investigation. One call per investigation, at high reasoning effort. It produces the summary and recommendations that analysts act on. Each attempt has a 45-second limit before the chain moves to the next provider. Pick the most capable model that reliably finishes within 45 seconds.
  • Remediation. Select Fire's risk analysis, remediation scripts and plan summaries, at high reasoning effort. After each full sync, GraphnAI also prepares up to 10 remediation analyses in advance. This route spends even when nobody asks. Pick the most capable model. If cost matters, consider the mid tier, because the advance preparation repeats after every full sync.
  • Batch analytics. The written explanation of an Operational Safety Metric™ score. Each attempt has a 10-second limit, and an investigation explains up to 4 scores at once. Pick a small, fast model.
  • Knowledge search. Not used yet. Route it to any provider.

Local providers get longer time limits than cloud providers.

TaskAnthropic ClaudeOpenAIGoogle GeminiLocal
Graphneclaude-sonnet-5-5gpt-6-solgemini-3.5-flash-liteQwen 3.8 27B, or a similar instruction-tuned model of about 30B parameters
Alert triageclaude-haiku-5-5gpt-6-lunagemini-3.5-flash-liteThe same local model
Investigationclaude-opus-5-5gpt-6-solgemini-3.8-flashThe same local model
Remediationclaude-opus-5-5gpt-6-solgemini-3.8-flashThe same local model
Batch analyticsclaude-haiku-5-5gpt-6-lunagemini-3.5-flash-liteThe same local model
Knowledge searchAnyAnyAnyAny
  • Larger models. You can try a larger model, such as gpt-6-astra or a Gemini Pro preview, for investigation and remediation. Check that it finishes within the 45-second limit. A timed-out attempt is still billed, and the chain falls back. Avoid preview models for production routes.
  • Fallbacks. Add a fallback to every route, ideally on a different provider or the local model. Then a provider outage or a declined request still gets an answer. Claude's safety checks can decline some security questions, and a fallback on another provider answers those.
  • Gemini. In our testing, gemini-3.5-flash-lite answered Graphne questions about as accurately as gemini-3.8-flash, at about a ninth of the cost. Gemini's implicit caching didn't apply to Graphne's prompts, so gemini-3.8-flash paid its full input price on every call. Gemini bills its thinking as output tokens, and GraphnAI counts them in the cost.
  • One provider for everything. One provider can serve every task. Mixing tiers within one vendor is the simplest setup.
  • Typical cost. On our lab, a Graphne question cost about $0.03 on claude-sonnet-5-5 and about $0.002 on claude-haiku-5-5. An investigation cost about $0.04 on claude-opus-5-5. An Operational Safety Metric explanation cost under $0.001 on claude-haiku-5-5. On Gemini, a Graphne question cost about $0.006 on gemini-3.5-flash-lite and about $0.06 on gemini-3.8-flash, and an investigation about $0.04 on gemini-3.8-flash, at Google's standard rates. These are typical figures, not guarantees. Your costs depend on your environment. Track them on the Cost tab (see Viewing AI Cost Reports), and set a limit in Setting a Monthly Budget.

Configuring Routing via the Admin UI ​

The Task routing table on the Providers tab has one row per task. Each change saves at once.

  1. In a task's Primary column, choose a provider.
  2. In its Fallbacks column, select other providers in the order to try them. A selected provider turns green. Select it again to take it out.

Choose the primary before any fallback. A fallback chosen for a task with no primary fails to save.

With one provider, the Fallbacks column reads "Add another provider to set fallbacks."

Fallback Transparency ​

Fallback is automatic. When a provider fails and the chain moves on, the server log records a warning that starts "LLM provider failed". It names the provider, the task and the error. The Evidence tab shows the provider and model for every call, and marks a call a fallback answered.

Circuit Breaker Health ​

Each provider has its own circuit breaker. It pauses a provider that keeps failing, so requests go straight to the fallbacks.

How the Circuit Breaker Works ​

  • Healthy: Requests flow through. Failures in a row are counted.
  • Paused after errors: After three failures in a row within a minute, the breaker pauses the provider for 30 seconds. Requests skip it and go to the next provider in the chain.
  • Testing recovery: After 30 seconds, one request goes through. If it succeeds, the provider is healthy again. If it fails, it's paused again.

These don't count as failures:

  • A request you or the browser cancelled
  • A malformed tool call, an empty answer or an answer cut off at its length limit. The provider answered, so the request is retried instead.

A timeout counts as a failure.

Viewing Circuit Breaker Status ​

Open the Health tab on Admin > AI Provider. Each provider's card shows:

  • Its state: Healthy (green), Testing recovery (yellow) or Paused after errors (red)
  • Requests, with a bar of how many succeeded and how many failed
  • Uptime: the share of requests that succeeded since the server last started

When a Provider Is Paused ​

Paused after errors means the provider failed three times in a row: an HTTP 429, a 5xx response, a timeout or a network failure. Common causes:

  • Rate limit reached: The API key hit its rate limit. Check usage in the provider's console.
  • Invalid API key: The key was revoked or rotated. Select Edit on the provider's card and enter the new key.
  • Provider outage: Check the provider's status page.
  • Model unavailable: The model may have been retired. Edit the provider and enter a current model.

The breaker recovers by itself once the cause is fixed. There's nothing to reset.

Viewing AI Cost Reports ​

GraphnAI records the cost of every AI call, in USD. Records are held in memory and written to the database every 30 seconds.

Cost Data Collected ​

Each AI call records:

FieldDescription
ProviderThe provider type that served the call, and the name you gave the provider
ModelThe exact model identifier
Task typeThe task the call was routed as
AgentThe agent that made the call
Input tokensTokens sent to the model
Output tokensTokens the model returned
Cached tokensInput tokens read from the provider's prompt cache
Cost (USD)Worked out from the price table
UserThe user whose request made the call
TimestampWhen the call happened

Cached tokens bill at each model's cached input price, usually 10% of its input price. Some newer models charge less, and some older OpenAI models charge 25% or 50%. Anthropic also charges 1.25 times the input price for tokens written to its cache, and GraphnAI includes that in the cost.

Reading the Cost Tab ​

Open the Cost tab on Admin > AI Provider. It covers the current calendar month, in UTC:

  • Cost this month, Budget, Input tokens and Output tokens at the top
  • Cost by agent: a bar and a table with each agent's cost, requests and tokens
  • Cost by provider: a bar and a table with each provider type's cost and requests. Every local or self-hosted provider is grouped as OpenAI-compatible.

With no AI calls yet this month, the tab says "No AI usage this month."

Setting a Monthly Budget ​

  1. In the Budget section of the Cost tab, enter a Monthly budget (USD). Enter 0 for no limit.
  2. Enter Alert at (% of budget), from 1 to 99. The default is 80.
  3. Select Save budget.

The Budget card shows "$X of $Y" with a bar. It turns orange at the alert level and red at the budget.

Over budget, the Investigation, Remediation and Batch analytics tasks pause. That stops investigations, finding explanations, Select Fire's risk analysis and score explanations. Graphne's answers and alert triage keep running.

Setting a Hard Spending Ceiling ​

The monthly budget pauses only some tasks. The hard spending ceiling stops all of them.

  1. In the Budget section of the Cost tab, enter Hard spending ceiling (USD per month). Leave it empty to use 1.5 times the monthly budget.
  2. Select Save budget.

The ceiling can't be lower than the budget. A meter shows this month's spend against both.

When spend reaches the ceiling, every AI call stops, including Graphne and alert triage. AI resumes next month (UTC), or when you raise the ceiling. See AI Safety Controls.

Supported Models and Pricing ​

GraphnAI carries a price table for current Anthropic, OpenAI and Google Gemini models. To see it, open the Providers tab and scroll to Model prices. The table shows each model's input, cached input and output price per million tokens, in USD.

The built-in prices follow these rules:

  • Standard rates. An introductory or promotional price isn't used. The table lists the standard rate that follows it, so an estimate runs high during a discount rather than low after one ends.
  • Long prompts. Some models charge a higher rate once a prompt passes a size, for example above 100K, 200K or 272K prompt tokens. claude-haiku-5-5 charges a higher rate above 100,000 tokens. The table shows that rate under the model name, and GraphnAI applies it.
  • Retired models are removed. A model that has shut down, or shuts down within 90 days, isn't listed.

A model that isn't in the table, and has no price you set, records its tokens at a cost of $0.00. Add a price for it as the next section describes.

The built-in prices reflect provider rates at the last GraphnAI update. Check current rates with your provider.

Setting Your Own Model Prices ​

Set your own price when you run a local model, use a provider the table doesn't list, call a model newer than the table, or pay a negotiated rate.

When one of your providers calls a model with no price, Model prices says so above the table: "No price for ollama-gpu (llama3.3:70b). Its calls are recorded as $0 until you add a price." Add price then opens on that provider.

  1. Open the Providers tab on Admin > AI Provider.
  2. In Model prices, select Add price.
  3. Choose a Provider:
    • One of your providers, listed under Your providers by the name you gave it. The price applies only to calls through that provider. Use it for a local model, or for a provider the table doesn't list, such as an OpenAI-compatible service you added as a Local provider.
    • A vendor, listed under Vendors: OpenAI, Google Gemini or Anthropic Claude. The price applies to every provider of that type that calls the model.
  4. Enter the Model exactly as your provider is set to call it. When you choose one of your providers, its model is filled in.
  5. Enter the Input and Output prices per million tokens. Enter Cached input only if the provider publishes a cached price. Leave it blank otherwise, and cached tokens bill at the full input price. A price of $0 is allowed. It records a free local model's token use without a cost.
  6. Select Save price.

To change a built-in price, select the pencil icon on its row. The row then shows a Custom badge, with the default price below the model name. To go back to the default, select the reset icon on the row. A price you added shows an Added badge. To delete it, select its remove icon. A price you set to exactly the default is dropped, so the model keeps following future updates to the table.

When more than one price could apply to a call, the most specific one wins: a price on your provider, then a price you set on the vendor, then the built-in price.

A custom price can carry its own long-prompt rate. Turn on Higher rate for long prompts, then enter Above (tokens), Long input and Long output. When you edit a model whose built-in price has a long-prompt rate, the dialog opens with that rate filled in, so changing the base price keeps it. When you add a price for such a model, the dialog says the built-in price rises above a prompt size, and turning the rate on starts from the built-in one. Turn it off only if your provider charges one rate at every prompt size.

A row marked Not used can't apply to any call. The line below the model says why: no provider has that name, or that provider calls a different model. Rename the provider or fix the model to match.

Configuring OTel GenAI Tracing ​

GraphnAI can send a trace of each AI call to any OTLP-compatible backend. Spans follow the OpenTelemetry GenAI conventions that Langfuse, Phoenix, Datadog and Grafana read.

With no endpoint set, tracing is off and nothing changes.

What Gets Traced ​

Each AI call produces one span, named gen_ai.chat, with these attributes:

AttributeValue
gen_ai.operation.namechat
gen_ai.systemrouter for every call. Use gen_ai.response.model to see which model answered.
gen_ai.request.temperatureThe temperature setting
gen_ai.request.max_tokensThe output token limit
gen_ai.usage.input_tokensInput tokens used
gen_ai.usage.output_tokensOutput tokens used
gen_ai.usage.cache_read.input_tokensTokens read from the cache, when there are any
gen_ai.usage.cost_usdThe cost in USD
gen_ai.response.modelThe model that answered

A failed call records the error on its span.

Enabling Tracing via Environment Variables ​

Tracing is set with environment variables on the server. Set your OTLP HTTP endpoint URL and, if your backend needs one, the full Authorization header value to send with each export. Restart the server after you set them.

Tracing stays off while no endpoint is set.

Example: Langfuse ​

Set the endpoint to your Langfuse cloud or self-hosted OTLP traces URL. Set the authorization value to Basic followed by your Langfuse public key and secret key, joined by a colon and base64-encoded.

Example: Self-hosted Phoenix or OTLP collector ​

Set the endpoint to your collector's OTLP HTTP traces URL. Port 4318 is the standard one. An internal collector usually needs no authorization value.

Spans are exported over OTLP/HTTP in batches. The service name in traces is graphnai-agent.

Understanding Prompt Caching ​

Prompt caching needs no setup. It lowers the cost and latency of repeated calls by reusing tokens the provider has already processed.

How It Works per Provider ​

Anthropic: GraphnAI marks the end of the system prompt for caching. The first request writes the system prompt to the cache, at 1.25 times the input price. Later requests within the cache lifetime read it at the model's cached input price, 2.5% to 10% of the input price. Graphne sends the same large system prompt with every question, so the cache serves most of it.

OpenAI: The provider caches repeated prompt prefixes by itself. Nothing needs setting up.

Gemini: GraphnAI relies on the provider's implicit caching. In our testing it didn't apply to Graphne's prompts, so each call paid the full input price. Explicit cache management isn't available yet.

Viewing Cache Savings ​

Costs on the Cost tab already include the cache discount. The tab doesn't show cached tokens.

Troubleshooting ​

Provider shows "Paused after errors" but the API key is valid ​

The circuit breaker paused the provider after three failures in a row. It tries again after 30 seconds. If it keeps pausing:

  1. Check the provider's status page for an incident.
  2. Check that the model hasn't been retired.
  3. Check whether the API key has reached its rate limit or monthly quota.
  4. Watch the provider's card on the Health tab after its next requests.
  5. Search the server log for "LLM provider failed". The entry names the provider and the error.

"One of the provider settings isn't valid" ​

Cause: The save was refused. A routing rule names a provider that doesn't exist, two providers share a name, or a cloud provider has no key. On the page, choosing a fallback for a task with no primary causes it too.

Fix: Choose the task's Primary first, then its fallbacks.

Cost report shows $0.00 for all entries ​

The model has no price. Either it isn't in the price table (a local model, a provider the table doesn't list, or a model newer than the table), or its name doesn't match a table entry, such as claude-sonnet instead of claude-sonnet-5-5. The server log names the provider and model with the message "No price for this model". For a Local provider, the message is logged at debug level only. A model on your own hardware has no per-token price, so $0.00 is correct for it.

To fix it, add a price for the model in Model prices on the Providers tab (see Setting your own model prices). Or correct the model name in the provider's settings. Costs recorded before the fix stay at $0.00.

OTel traces don't appear in your backend ​

  1. Check that the tracing endpoint variable is set, and that the server was restarted after you set it. The server log says "GenAI OTel tracing enabled" at startup.
  2. Check that the server can reach the endpoint URL through any firewall.
  3. If your backend needs authentication, check that the authorization variable holds the full header value, with its scheme (Basic ... or a token scheme).
  4. Check the server's output for OpenTelemetry export errors. A failed export doesn't affect Graphne.

"The AI settings changed in another window" ​

Someone saved the AI provider settings, in another window or through the API, after this page loaded. Saving from the older page would have replaced their change, so GraphnAI refused it and reloaded the page with the saved settings. Check them, then make your change again.

All providers fail at once ​

When every provider in a task's chain fails, the request fails. Graphne shows "Query failed. Please try again." Give each task at least one fallback of a different provider type than its primary. A local provider (LM Studio / Ollama) can be the last fallback on an air-gapped or isolated network.

Role Requirements Summary ​

ActionRequired role
View providers and their healthAdmin
Add, edit or remove providersAdmin
Set task routing and fallbacksAdmin
Set model pricesAdmin
View AI cost reports and set the budgetAdmin
Configure OTel tracingWhoever deploys the server (environment variables)