Every time Pixie's agent is about to think (a chat reply, a brief, a huddle turn), it first has to answer a deceptively simple question: which model answers this one? This article is about the one function that decides, the settings that feed it, and the rule that makes the whole thing trustworthy: no matter what is misconfigured, expired, or unplugged, the agent always lands on a model. It just might be a smaller one.
This is a mechanism deep-dive under Choosing Brains: the Model Bench, which explains why Pixie offers several brains at all. Its sibling, Mechanism: On-Device Curation, covers the on-device half: how local models are vetted and why the built-in Apple model is always available. Here we take the other half: the external arms (a runtime on your own computer, a cloud key you bring, or a friend's shared Homebase) and the fallback discipline that keeps them safe to depend on.
Pixie's agent code never talks to a model directly. Every call site, whether the chat view
model (the code behind the chat screen), the brief generator, or the planner, asks for an
LLMProvider: a small protocol (Swift's word for an interface, a named list of
functions a type promises to provide). The protocol has a complete() (and a streaming
variant) that takes
messages, tool definitions, and options, and returns a response. Concrete providers
exist for each brain: AppleFoundationProvider (on-device), OllamaProvider (your own
runtime), OpenAIProvider and AnthropicProvider (your own cloud keys), and two
relay-routed ones (providers whose requests travel through Pixie's relay, the forwarding
server that only ever sees encrypted bytes) that we will meet shortly.
The thing that hands out providers is LLMRouter: a single class, and deliberately
the only place in the app where "which model?" is decided. Call sites either hold a
reference to it or reach it through one app-wide pointer (LLMRouter.current, set once
at app startup), and ask:
let provider = router.provider(model: modelName)
Two design choices in that one line carry most of the weight.
First, the decision is made at every call, not at construction. The router does not
cache a provider when a view model is created; it re-reads the user's settings each time
provider(model:) runs. Add an API key, flip the model radio, verify a runtime, and the
very next message routes differently, with no view-model rebuilding, no restart, no
"apply" button.
Second, there is exactly one decision point. When routing lives in one function, you can state invariants about it (rules that must always hold) and actually mean them. The invariant this article builds to: every arm of the router, when its prerequisite is missing or its remote end fails, lands on the on-device provider, never on an error. That is only a checkable claim because there is one place to check.
Inside provider(model:), the first step is a call to resolve(), which reduces all the
settings to one of four routes:
enum Route {
case onDevice // Apple system model or a local MLX model
case external // a model on an Ollama runtime (yours, or a shared Homebase)
case openai // your own OpenAI key
case anthropic // your own Anthropic (Claude) key
}
The rest of the article is how resolve() picks, and what each route does when reality
disagrees with the settings.
In Settings → Agent there is a radio group (the model-source radio, a pick-one selector) with three positions: Apple Intelligence (the built-in system model), Community (a locally downloaded MLX model, MLX being Apple's framework for running models on its own chips; see Mechanism: On-Device Curation), and External. Whatever the radio says is the routing truth. There is no heuristic that second-guesses it, no "smart" auto-upgrade to a bigger model behind your back. You chose a brain; the router honors the choice, up to availability.
External is not a single brain but a category. Inside its pane, a runtime dropdown
picks which external brain: Ollama (a model server running on your own computer, and
the default), OpenAI, or Claude. So resolve() is a short
decision list. Apple Intelligence and Community route .onDevice outright.
External consults the dropdown: OpenAI and Claude route to their APIs exactly when a
key is on file, and Ollama routes when that arm is available right now. Every missing
prerequisite lands .onDevice. A radio that was never set falls to the legacy
derivation below.
Notice the shape of every branch: the user's intent is one input, but a prerequisite check
is the other. Selecting Claude with no Anthropic key on file does not produce a "please
configure" error at send time; it produces .onDevice. The radio expresses what you want; the
router only routes there when the arm can actually carry the call.
The agent itself never refuses to answer. Where Settings does speak up is about a stale arm rather than an unconfigured one: a Homebase that has stopped advertising (stopped announcing "I'm here, and here are my models") is flagged as not heard from recently, with inference falling back on-device until it returns.
Two legacy branches keep old installs working. Early builds had a four-position radio with
OpenAI as its own top-level choice; a stored "openai" radio value still routes (key present →
.openai, else .onDevice).
And if the radio was never set at all (an install predating the radio entirely), the router derives a sensible answer from older settings: a working Ollama arm wins, then an explicit local-model preference, then a stored OpenAI key, then on-device. The lesson generalizes: a settings migration is a routing problem. If an update makes yesterday's stored preferences meaningless, users silently lose the brain they chose. Cheap aliases avoid that.
A word on why cloud keys exist at all. Pixie ships no platform key: there is no
"Pixie cloud inference" the app quietly bills or proxies through. If your words go to
OpenAI or Anthropic, it is because you pasted your key, under your account and
rate limits. The .openai and .anthropic routes are strictly bring-your-own.
People who bring keys often bring several: a personal key, a work key, one nearing its
quota. CloudKeyStore manages each provider's keys as a ring: an ordered list
stored in the device Keychain (the OS's encrypted secret store) as a JSON array under
<legacyKey>.list. The rules are small and strict:
That mirror trick is the interesting part. The router and the providers were written when there
was one key per provider, and they still read exactly one: hasBYOKey() (BYO:
bring-your-own) and hasAnthropicKey()
check the legacy slot, and each provider pulls the same slot at call time.
Rings were added around them: reorder the ring in Settings and the new first key lands in the slot everyone already reads. The consumers never changed. When you can add a feature by changing what a storage location contains rather than who reads it, the blast radius stays tiny.
Two more details make keys pleasant to live with. Each provider takes its key as a
deferred closure (apiKeyProvider): a little function that fetches the key fresh,
evaluated per request. So rotate a key and the
next call uses it, mid-conversation. And the Settings pane's "Test" button is backed by
a zero-token probe: AnthropicProvider.validateKey issues a GET /v1/models, which
answers 200 (success) for a good key and 401 (unauthorized) for a bad one, without billing a
single token (the word-fragment unit these APIs meter and charge by). In the
UI, keys render masked. The conventional prefix stays legible, the secret does not:
sk-ant-•••••.
The providers behind these routes are thin, faithful API clients.
AnthropicProvider speaks the Messages API (Anthropic's own HTTP surface for chat)
directly: it sends x-api-key and anthropic-version headers and carries tools as input_schema.
Tool calls come back as tool_use blocks whose arguments are a JSON object (unlike
OpenAI's, which arrive as a JSON string to parse). OpenAIProvider speaks Chat
Completions, OpenAI's equivalent surface. Both plug into the same agent loop as every
other provider, tools and all.
The dropdown's default, Ollama, is the most interesting arm, because "a model on your own computer" turns out to mean three different network shapes.
The anchor concept is the Homebase: the Mac in your account that acts as a full peer and
hosts big models (the whole story is The Homebase). On it runs
Ollama, an open-source local model server. Pixie's ExternalRuntimeStore
records where that runtime lives, with a test-before-persist gate.
An endpoint (the URL where that runtime listens) is stored only after it has answered a version probe, listed its installed models, and, if any completion-capable model is installed, actually served a trial completion. A URL that never proved it can do inference (actually generate text) never becomes a route. And a previously verified endpoint is not forgotten just because a later probe fails (maybe Ollama isn't launched right now); configuration and availability are deliberately separate facts.
One storage rule matters more than it looks: the endpoint is device-only and never
rides Pixie's cross-device settings sync. The default endpoint is
http://127.0.0.1:11434: the loopback address, the address a computer uses to talk to
itself. If the Mac synced that string to your
phone, the phone would try to reach an Ollama server on itself. "My machine runs a
runtime" is a machine-local fact, so it stays machine-local.
The Ollama arm also needs a selected model tag (like llama3.2:3b): no selection,
no route. With a selection in hand, provider(model:) walks the shapes in order. First: a selection
that points at a joined Homebase (someone else's) routes cross-account while that
Homebase is fresh, and on-device once it goes stale. Next: a verified local endpoint
routes to loopback Ollama. Then: a handset holding a fresh sibling advert routes
over the relay. Anything else lands on-device.
Shape one: the model is next door. On the Homebase itself, the verified endpoint is
loopback HTTP, and the provider is OllamaProvider. It is deliberately a thin wrapper:
Ollama exposes an OpenAI-compatible surface at /v1/chat/completions, same streaming
format, same tool-call shapes. So OllamaProvider simply configures the battle-tested
OpenAIProvider engine with a different base URL, the model tag, and a dummy key Ollama
ignores. We could have written a native client against Ollama's own /api/*; instead
that API is used only for management (probing, listing, pulling models), and the
inference path reuses code that already survived production. Fewer codepaths, fewer
bugs.
Shape two: the model is across the room. Your phone has no local runtime, but your
account's Homebase does. RemoteInferenceProvider ships the request to the Homebase
over the sealed sibling channel: the same end-to-end-encrypted sync_delta frames
your devices already use to sync with each other. End-to-end-encrypted means only your
devices hold the keys, so the relay in the middle carries
only ciphertext, encrypted bytes it cannot read. The request/accept/chunk/done round-trip gets its own article,
Mechanism: Inference over the Relay.
Shape three: the model is in someone else's house. A friend can share their
Homebase with you (see Mechanism: Sharing a Homebase).
The external selection records not just the model tag but whose Homebase serves it:
a field that is either "own" or the host device's public key, its shareable
cryptographic identity. A joined-Homebase
selection routes through CrossAccountInferenceProvider over the
sealed 1:1 contact channel: the end-to-end-encrypted lane
you and that friend already message over, so the relay forwards ciphertext it cannot
read. As the tree shows, this shape is checked first, so choosing a friend's model
wins even on a machine that has its own runtime. If the stored host value is corrupt,
meaning not valid hex (the letters-and-digits format keys are written in) or the wrong length,
it silently degrades to "own": a damaged preference
must not strand routing.
One platform asymmetry: shape two is fenced off on Mac Catalyst (the Mac build). The Homebase does not route to a sibling advert, because the Homebase is the host: if its own runtime is gone, the honest answer is on-device, not a network hop to itself. The guest side of shape three has no such fence; any device can be a guest.
Shapes two and three share a failure mode: the desktop at the far end can be asleep, offline, or powered off, and the router cannot cheaply ask "are you there?" before every message. Its evidence is adverts: small signed announcements a Homebase sends ("here are my models"), remembered on the receiving device with a timestamp.
ExternalModelsAdvertStore holds the last advert from your own Homebase, and the router's gate
is blunt: the advert must be less than 10 minutes old (and non-empty). The number is not
about model lists changing. It is about a specific bad experience. The remote-inference
protocol gives the Homebase 10 seconds to accept a request before timing out.
Without a freshness gate, a phone whose desktop went to sleep an hour ago would eat that 10-second wait on every single send before falling back. A stale advert means "probably unreachable," so the router falls back before the doomed wait, and chat stays snappy.
JoinedHomebaseStore gates shape three the same way but with a 24-hour window, and the
difference is principled. Sibling adverts are heartbeat-like: they arrive on a regular pulse,
so silence quickly means something. Cross-account adverts are
event-driven: sent on grant, on model changes, on runtime re-verification. There is no periodic
pulse, though every inbound frame from the host refreshes the clock (noteHostAlive).
A 10-minute window would mark a perfectly healthy shared Homebase stale most of the day. The
generous window only exists to skip hosts that are clearly gone; the in-call fallback (next
section) absorbs everything subtler. A joined record must also actually be in the joined state
(an invitation you have not accepted never routes) and advertise at least one model.
Freshness gates are a bet, and bets can be lost in both directions: a Homebase can die seconds after advertising, and a "stale" one might actually be reachable. That is fine, because freshness is only the outer of two safety layers.
Here is the deep reason the router is shaped the way it is: a call site cannot re-route mid-call. By the time a view model holds a provider and has started a completion (a model reply in progress), there is no seam to say "actually, that desktop is dead, please redo this elsewhere." So failure handling cannot live at the call site. It lives in two layers below it.
Layer one: routing time. Everything this article has described so far:
resolve() and provider(model:) check each arm's prerequisites (key on the ring,
verified endpoint, fresh advert, joined state, valid selection), and any miss returns
the on-device provider instead of the fancy one. There is even a re-check between the
two functions: resolve() may say .external, and in the instant before
provider(model:) finishes, the runtime could be forgotten, the advert could expire,
the Homebase grant could be revoked. Each of those races (cases where the world changes
in the instant between two checks) lands on-device, not on a trap.
Layer two: inside the call. The two relay-routed providers are constructed with an
injected on-device fallback provider baked in. RemoteInferenceProvider arms a
watchdog ladder per request, a stack of timers that each abort a hung step: no accept
within 10 s, no first output within 90 s, a
30 s mid-stream stall. Failures of the ordinary kind (timeout, model unavailable,
host error, an interrupted stream) are answered by silently running the same request on
the fallback. The user sees an answer; at most they notice it reads like the smaller
model.
Both relay-routed arms carry exactly one protocol-specific repair before that fallback,
and they are different repairs because the two arms fail differently. RemoteInferenceProvider
retries once (after first kicking off a sync reconcile, a catch-up pass that brings the two
copies back in line) when your Homebase reports that its replica of
the conversation has diverged: its copy no longer matches yours. That state is recoverable,
so it is worth one round trip
before giving up. CrossAccountInferenceProvider retries once with the full conversation
inlined (sent whole, rather than as just the newest messages) when the host's cache cannot
serve an incremental request. Anything else, and any
second failure of either repair, goes straight to on-device.
One more rule rides layer two: tool-using calls never cross the wire. If a request carries tools, the relay-routed providers hand it straight to the on-device fallback. The agent loop (tool execution, memory, files) stays on the handset by design (see Mechanism: CallTool, One Tool to Bind Them), and shipping a tool round-trip across the relay is future work, not a silent half-feature.
We could have surfaced these failures instead: an alert, a retry button, a "your Homebase is unreachable" sheet. Plenty of software does, and for a file sync that honesty is right. We decided an agent is different: it is a conversation, and a conversation partner who sometimes goes silent because of your network topology (which machines can currently reach which) is one you stop talking to.
Degrading to a smaller brain is visible in the answer's quality and correctable in Settings; failing to answer teaches the user the feature is flaky. The one thing Pixie never silently does is the reverse: quietly upgrading you to a cloud you did not choose. Fallback always falls toward the device in your hand, never toward someone else's computer.
Follow any arm of the tree to its end and you reach AppleFoundationProvider. Even
that has a fallback inside it: if your chosen community MLX model is not downloaded or
not ready, it runs the built-in Apple system model, which is always present on a
supported device (that last step belongs to
Mechanism: On-Device Curation). The recursion
bottoms out at a brain that cannot be missing.
That is why LLMRouter has no error path and no "unconfigured" state at all.
The moral of this mechanism is the same one that shapes the whole harness: routing is a promise about the worst case, not the best one. The best case was never hard: any demo can call a big model when everything is configured and online. The engineering is in guaranteeing that every misconfiguration, every expired advert, every sleeping desktop, and every mid-stream stall resolves to the same place: a model that is physically in your hand, answering.
/v1/chat/completions surface.OpenAIProvider and AnthropicProvider speak.