返回资讯中心
外部精选
软件工程
#开发者实时流

Designing LLM failover as a route change, not a retry loop

Every request depends on two identifiers: the exact model ID in the body and the routing group on the key. Ordinary keys add ordered fallback groups that take over when a route cannot serve the request.

Hacker News 最新Matthewmodelfla2026/10/19 · 2026/10/196 分钟阅读

以下正文同步自 Hacker News 最新,版权归原站所有,已转换为易读排版。

Every request depends on two identifiers: the exact model ID in the body and the routing group on the key. Ordinary keys add ordered fallback groups that take over when a route cannot serve the request.

Modelflare routes each request through a routing group, a set of service provider channels. This article explains how model IDs and groups interact, how group failover works on ordinary keys, which errors stop a request immediately, and how billing follows the group that served the request.

Both have to agree. A valid JSON body still fails when the key's group has no channel for that model. Create the key in a non-default group that lists the model; there is no single required group. The live catalog, with prices, groups, and protocol labels, is Models & Pricing.

curl https://modelflare.dev/v1/models
-H "Authorization: Bearer YOUR_MODELFLARE_API_KEY"

Use the exact id from the response in the model field.

An ordinary API key names one primary group and, optionally, fallback groups in priority order. Configure the chain in API Keys when you create or edit a key. Order matters, and the console lets you move entries up and down.

A group exhausts its own channel priorities before the next group is tried, so a fallback is a route change rather than an extra retry of the same provider.

Failover covers route and connection problems only. The request stops immediately on:

Heartbeats alone do not prevent a switch, but once content has been written, the request is never replayed on another route. A client that streams should handle a mid-stream failure itself.

Image and video requests sent to the wrong protocol return 400 with unsupported_endpoint and name the required API.

After a failover, the request is billed at the multiplier of the group that served it, not the primary group. The consume log records which group that was, so a route change is visible after the fact.

Limited-time routes are temporary by design. When a key's primary group is a campaign route, the key cannot be saved without at least one concrete fallback group, so it keeps working after that route ends.

A Smart API Key does not take a manual fallback list. It walks every group the account can use, ordered by its routing strategy (balanced, stability first, or low price first). Choose an ordinary key when you want to control the order yourself and a Smart API Key when you want the gateway to resolve it. See Smart API Keys for details.

Further reading: Models and groups, Group failover, and Errors. For higher limits or volume arrangements, contact business@modelflare.dev.

正文由 探How 从来源站点 RSS 同步,内容未经改写;遇到排版缺失或需要图片、视频时请以原文为准。

打开原文
去社区讨论 →