Pi: Virtual Models

Code

Issues231 (231)

Pull requests40 (40)

Discussions

Actions

Projects

Security and quality4 (4)

Insights

More items

feat(coding-agent): Virtual models - #10035

#10035

Merged

mitsuhiko merged 13 commits intomainearendil-works/pi:mainfrom virtual-modelsearendil-works/pi:virtual-modelsCopy head branch name to clipboard

Sep 28, 2026

ConversationCommits13 (13)ChecksFiles changed

Merged

feat(coding-agent): Virtual models#10035

mitsuhiko merged 13 commits intomainearendil-works/pi:mainfrom virtual-modelsearendil-works/pi:virtual-modelsCopy head branch name to clipboard

Conversation

mitsuhiko

commented

Sep 25, 2026

•

edited

Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link

Copy Markdown

Member

This adds experimental support for virtual models. A virtual model is a catalog entry that an extension registers with pi.registerVirtualModel(). It does not talk to a provider itself; for every request it picks a physical model and thinking level. The motivation is to make routing policies pluggable: plan on a strong model and implement on a cheap one, pick a model with a classifier, or move to a larger window when the context grows. Today this needs model switching by hand or hacks in prepareRequest.

The selection stays virtual: model_change records it, resume restores it, and each response records the physical model and thinking level that produced it. Router state can be stored on the session branch, so it follows /tree and forks and survives compaction. Context limits, compaction and image sizing use the physical model a request was routed to. examples/extensions/jev-router.ts shows a full router that uses the Jev classifier to pick Sol or Terra for planning and hands off to Luna after the first edit. docs/virtual-models.md describes the API.

There was an error while loading. Please reload this page.

mitsuhiko

changed the title Virtual models

feat(coding-agent): Virtual models

Sep 28, 2026

mitsuhiko

added 4 commits September 28, 2026 18:22

feat(ai,coding-agent): add virtual models that route each request to …

…

277de57

…a physical model

Extensions register virtual models with pi.registerVirtualModel() (SDK code with createVirtualProvider()). The selection is recorded in model_change entries and restored on resume; assistant messages record the physical model and the new AssistantMessage.thinkingLevel. Agent-loop requests route in prepareRequest; summaries route before they are sized, and other direct requests route in ModelRuntime.streamSimple. Limits for compaction and context usage come from the physical model of the latest response.

Interactive mode shows the routed model in the footer, a notice when the responding model changes, and the selected and routed models in /session. The jev-router example plans on Codex Sol or Terra and switches to Luna after the first edit.

refactor(coding-agent): simplify virtual model routing

…

0595626

Detect virtual models by api, share latest-response lookup and routing helpers, drop the unused physical pass-through in resolveModel, remove the redundant /session model block, and tighten tests and the jev-router example.

fix(coding-agent): route virtual model retries and resume selections …

…

dd6395c

…correctly

• Route auto-retries and overflow compact-and-retry as retry, reporting the failed request's model as previous, so a retry stays on the model routed for the turn instead of the previous turn's model.
• Resolve the branch selection with getBranchSelection(): a virtual model_change holds until the next one, otherwise the latest physical response wins. Resume uses it and skips failed routing messages, so an unregistered virtual model falls back to the last physical responder.
• Record the selection before a run when the branch implies another one, e.g. after tree navigation left the latest model_change on another branch.

fix(ai,coding-agent): record thinking level for all simple streams

…

708614b

Move thinking level recording into pi-ai so Models.streamSimple and the compat streamSimple record it, not just the coding-agent runtime. Defer recording a resume model override to the next prompt instead of writing on open. Warm the prompt cache only for requests to the selected model, which excludes routed virtual model requests and extension redirects.

mitsuhiko

force-pushed the

virtual-models

branch from 462b447 to 708614b Compare

September 28, 2026 16:28

mitsuhiko

added 9 commits September 28, 2026 20:32

feat(coding-agent): register virtual models on any provider with rout…

…

6c37603

…er state

Virtual models are kept apart from providers and overlaid onto any provider's catalog, so a provider can list several virtual models next to physical ones. pi.unregisterVirtualModel() and ModelRuntime.registerVirtualModel() replace createVirtualProvider.

Routers receive failed retries as request.failed, separate from previous, and can return state that Pi stores on the session branch as pi.virtual-model-state entries. Requests routed to a model with a smaller context window compact first. The jev-router example keeps its phase as router state.

Merge remote-tracking branch 'origin/main' into virtual-models

9080c76

fix(coding-agent): route the first request of a prompt as a user turn

…

36176ad

The route reason came from the last message, so custom messages added after the user's prompt (before_agent_start, nextTurn) made the first request a continuation. Track whether a user message was added since the last routed request instead.

refactor(coding-agent): simplify virtual model request routing

…

5cc8577

Derive the user route reason from the context instead of a session flag, inline routing into prepareRequest, always check compaction against the routed model, give direct requests no router state, and drop the redundant unrouted stream guard in ModelRuntime.

refactor(coding-agent): remove virtual model change notices

…

db47f72

The footer already shows the routed model.

refactor(ai,agent,coding-agent): record the thinking level in the age…

…

3aa13ab

…nt loop

Stamp the requested thinking level on the final assistant message in the agent loop instead of wrapping three streamSimple paths in pi-ai. Every stream function the agent uses now records it.

fix(coding-agent): keep caller credentials off other providers and st…

…

014b48b

…op redundant model_change entries

Direct virtual requests no longer forward apiKey, headers and env to a routed model of another provider. Physical selections are no longer compared against responses, which appended a model_change on every prompt while prepareRequest redirected requests.

fix(coding-agent): restore virtual selections of virtual-only provide…

…

6f0792c

…rs on resume

registerVirtualModel only scheduled an availability refresh, so a provider of only virtual models (e.g. jev/auto) was not configured yet when session restore checked auth, and resume fell back to the default model.

fix(coding-agent): compare router state by identity and stop re-proje…

…

8826eef

…cting for compaction checks

Router state was compared with JSON.stringify, which depends on key order and throws on values it cannot serialize. Returning request.state or undefined now keeps the state; any other value is stored.

Compaction checks under a virtual selection projected the session again on every request. The check between turns is skipped for virtual selections, since prepareRequest checks the routed model, and that check reuses the projection the request was built from. A two-request run now projects 4 times instead of 8.

Hide details View details

mitsuhiko

merged commit 540e174 into

main

Sep 28, 2026

3 checks passed

Uh oh!

There was an error while loading. Please reload this page.

Successfully merging this pull request may close these issues.

There was an error while loading. Please reload this page.