The L1–L2–L3 Harness

A model small enough to run on your phone can hold a perfectly good conversation. Ask that same model to also decide when it needs a tool, follow a multi-step procedure, and write a well-formed tool call (all at once, in one context window, the single span of text it can see at a time) and it quietly falls apart. This article is about the harness that fixes that: the scaffolding of prompts, checks, and loops built around the model. The harness splits one impossible job into three small ones, gives each the tiniest possible view of the world, and wraps the whole thing in a loop that catches failure without ever trusting the model to notice it failed.

This is the second article of the Agent series. Agent explains why a relationship-care app ships an agent at all, and Choosing Brains: the Model Bench explains which model ends up answering you at any given moment.

Here we take the hardest case that bench can hand us (the small, on-device model) and show how the harness makes it act like a competent agent anyway. As everywhere in this series, we are talking about harness engineering: prompts, sessions, loops, and signals. Nothing here is about training a model. Every model in this story arrives frozen (its weights never change), and all the reliability comes from structure.

1. One turn, three jobs

First, the thing everybody builds, because with a large model it works.

The classic agentic loop (a loop in which the model can act, not just talk) looks like this. Put two things in front of the model: a system prompt (the standing instructions it reads before the conversation starts) and a catalog of tool schemas (machine-readable descriptions of functions the model may call: name, arguments, types). The model either answers in text or emits a structured tool call.

The harness executes the call, appends the result to the conversation, and asks again. Repeat until the model answers in text. Pixie runs exactly this loop for the big external models: the chat view model streams the provider's output, executes each tool call through a dispatcher (the code that routes each call to the right function) and loops. The loop is capped at five round-trips per user turn, so a confused model cannot spin forever.

Look closely at what that one loop quietly demands of the model, though. It is really three different jobs:

Routing. Is this message small talk, or does it need a tool at all? Which one?
Instruction-following. Carry out a multi-step procedure (check this source, then that one, then combine) without skipping, repeating, or wandering off.
Call-crafting. Emit an exactly-well-formed call: right tool name, right argument names, right types, a query string that will actually match something.

And it demands them simultaneously, inside a context that now contains the persona, the full schema of every tool (a couple hundred tokens each; tokens are the word-pieces a model reads, and the unit its context is budgeted in), the procedure text, and the whole conversation. A frontier-scale model (one of the giant models running in a data center) shrugs at this. The on-device model (Apple's built-in system model, the floor of the bench, reached through the FoundationModels framework, Apple's programming interface to it) does not. Its failure modes are specific and repeatable, and we hit all of them:

It narrates instead of calling. Asked to find a document, it replies, as prose to the user, "I'll use search-info to locate your PAN card" (an Indian tax-ID card), and stops. That sentence looks like progress, but nothing ran. The user gets an empty promise.
It drowns in context. Every schema you bind (every tool made callable in that context) is text the model must wade through on every step. Stack a dozen schemas on top of procedure steps and history, and the small model latches onto the wrong thing or loses the thread entirely. (This is also why the agent's Notebook is never injected wholesale into context: retrieval happens through a tool call instead; see Mechanism: Memory & the Notebook.)
It does not know when to stop. Left to free-run, it re-calls tools it already called, or keeps going after it has the answer.

None of this is a moral failing of small models. It is a budget fact: attention and context are scarce, and the flat loop spends them on three jobs at once. The fix is the same one systems people always reach for: decompose, and give each part the least it needs.

2. The decomposition

When the user has skills enabled (a skill is a user-editable instruction bundle; more on that in a moment), the on-device provider stops running the flat loop and runs a three-level decomposition instead.

Three levels, and one rule repeated three times: every level sees the least it can get away with. The planner (L1) sees a one-line menu, not a single schema. The skill-follower (L2) sees the chosen skill's steps and one-line tool signatures (just a name, a key argument, and a purpose), never full schemas. Each tool-agent (L3) sees exactly one full schema and nothing else: not even the conversation. No level is asked to do more than one job, so no level needs the context that the other jobs would drag in.

Let's walk down the stack.

3. L1: the planner picks from a menu

The planner's session (one self-contained conversation with the model) is deliberately spartan. Its instructions are the agent's base persona plus a short piece of plan guidance plus a menu: one line per enabled skill, like

Your skills:
- @search-info: The skill to use whenever you need to find some information the
  user asked for — it checks every place that information could live.
- @check-schedule: Check the user's calendar — meetings, appointments, free/busy —
  for a specific day or a relative one like today, tomorrow, this weekend, ...

Your tools:
- @current-date: You do NOT know the current date/day on your own — use this for
  ANY question about today's date or day of the week.

Each @handle names a skill or a standalone tool in one shared @-namespace (skills and tools draw from one pool of unique slugs, short hyphenated names, so every mention is unambiguous). The detailed steps behind each skill, and every tool schema, stay out of this context entirely. The planner's whole job is a multiple-choice question: given this message, which line of the menu applies, or none?

And here is the first genuinely load-bearing harness trick. The planner binds exactly one tool, called plan. The model deciding to call plan is the "I need tools" signal: a structured tool call the harness receives directly, not a sentence of prose it would have to parse and hope it parsed right.

The plan argument is one sentence of free text naming the chosen skills with @: "I'll use @search-info to find their PAN number." The harness extracts the @handles from that sentence; the sentence itself is kept as the rationale and handed down to L2 as its roadmap.

There is a subtle mechanical problem hiding in this design. FoundationModels' respond() call runs the model until it produces a final text answer. But the planner has no real tools bound. After calling plan, the model has nothing to conclude with, and the call would never terminate.

The fix is blunt: the plan tool records the plan text and then throws a sentinel Swift error (an error raised on purpose as a signal, not a real failure) named PlanSignal, straight through the framework's loop. The provider catches PlanSignal (plan captured, session halted, move to execution) while any real error keeps propagating. We will see this trick again: when a model cannot be trusted to stop, the harness stops it structurally, by throwing through the machinery that runs it.

Around this sits a small amount of babysitting, all deterministic (plain code, no model judgment):

Handle validation. If the plan names a handle that does not exist, the harness replies to the planner ("Your plan named handles that don't exist: @foo. Use ONLY these handles: …") and lets it retry, up to three corrections.
Narration salvage. Remember the model that narrates instead of calling? If the planner's "direct answer" turns out to mention a valid handle in prose ("I'll use search-info to…"), the harness treats it as a mis-emitted plan and executes it anyway, so the user never receives an empty promise.
No plan, no ceremony. If the planner answers directly and names nothing, that text simply is the reply: most messages are conversation, and the guidance says so explicitly. (That guidance line was not written once and admired. It was chosen from a 28-variant sweep against an evaluation harness (a fixed battery of test messages, each with the routing it should get) and tuned to over-call on lookups rather than ever guess at a personal fact.)

One more deliberate subtraction: there exists a planner-side tool for reading a skill's detailed steps before planning, and we chose not to bind it. The planner routes on the menu alone; the steps are followed at the level where the work happens. A single-tool surface keeps "call plan" unambiguous for a small model; two tools would reintroduce a routing decision inside the routing decision.

4. From a sentence to a toolbox

Between L1 and L2 sits a purely mechanical step: turning the plan's @handles into the set of real tools to make available. The SkillRegistry resolves each handle against the shared namespace (skill or tool) and expands skills recursively, because a skill's body may itself mention other skills.

@search-info, for instance, says: check @check-notebook first, then @fetch-file (itself a skill that wraps the file-finding tools), then @grep-memory. The expansion walks all of it (guarded against cycles, so a skill that loops back on itself cannot expand forever), removes duplicates, and produces a flat, ordered toolset. That toolset, not the whole catalog, is what L2 gets.

How skills are authored, shared, renamed, and why this menu doubles as the user's main alignment surface is its own story: Skills, Tools, and the @-Namespace.

5. L2: the skill-follower executes one procedure

L2 is a fresh session whose instructions contain: a lean preamble, the planner's rationale ("Your plan: I'll use @search-info to…"), the chosen skill's detailed steps (now, finally, in context, because now they are the job), and one compact line per available tool:

Your available tools:
- check_notebook(query) — search your distilled private Notebook (facts about the
  user, a page per person in their life, ...); empty query browses recent notes
- find_file(query) — find one of the user's own files (docs/scans/photos/recordings
  across their devices) by description and attach the best match
- grep_memory(query) — last-resort search of your full long-term memory log ...

Those are signatures, not schemas: name, key argument, terse purpose. The roughly 200-token JSON schema behind each tool (its full machine-readable argument spec) never enters L2's context. That is the entire context saving, and it is what lets the toolset grow without L2's per-step reading load growing with it.

But if L2 has no real tools bound, how does it call anything? Through the second one-tool surface of the stack: a single meta-tool named call_tool, a tool whose only job is to reach other tools. To use a tool, L2 calls call_tool with the tool's name and a plain-language instruction, not arguments:

call_tool(tool: "find_file",
          instruction: "the user's resume, to read their email address")

L2 is explicitly told: do not write tool arguments yourself; just say what you want. Crafting the real call is somebody else's job: the L3 agent below.

Two hard-won details live at this level:

The loop must be the framework's own. An earlier version drove L2 manually: execute the call, rebuild the conversation with the result appended, ask again. On the small model, that rebuilt turn reliably degenerated into narrating the next call as prose instead of making it. Running L2 inside FoundationModels' native tool loop (where call_tool's handler dispatches L3 inline and returns the result into the same live session) is what makes the model keep genuinely calling, iteration after iteration. This seam, and why one meta-tool beats binding many real tools, is the subject of Mechanism: CallTool, One Tool to Bind Them.
The dispatcher owns discipline, not the model. Before any L3 run, the dispatch code resolves the named tool (a typo gets back a corrective message listing the real options), rejects an exactly-repeated call ("You already made that exact call. Use what you have, or do something different."), and enforces a per-tool budget of three calls per turn. Tripping the budget throws the second sentinel error, ToolLoopHalt, which unwinds L2's respond() the same way PlanSignal unwound L1's. A small model will happily search for the same file five times; the harness simply makes the fourth attempt impossible.

One curation note: not every bound tool is even shown to L2. The device-listing tool, for example, is plumbing (L3 defaults file searches to all of the user's devices in code), so L2 never sees it and never wastes a step reasoning about device scoping.

6. L3: one tool, one call, fresh eyes

Every call_tool dispatch spins up an L3 tool-agent: a brand-new, ephemeral session (it lives for one call, then is thrown away) bound with exactly one real tool and its full schema. Its instructions are almost insultingly narrow: you have exactly one tool; your only job is to call it once, with well-crafted arguments, to satisfy the instruction you were handed; then briefly state what it returned; never answer from your own knowledge.

This is where call-crafting quality comes from, because this is where the per-tool craft rules live: out of L2's sight, loaded only when relevant.

The file-finder's L3 is told to build its query from several ranking keywords (document type plus the specific detail wanted), never a single bare word. The calendar L3 is told to resolve "this weekend" into concrete YYYY-MM-DD bounds before calling. Each rule rides with its tool, costs context only when that tool is used, and is applied by a session whose entire world is that one schema. A small model that flails when juggling twelve tools is startlingly good at filling in one.

Two details make L3 safe and honest:

Serialization. FoundationModels may run tool handlers concurrently (at the same time), and there is one shared on-device model; two overlapping L3 generations would collide on it. An async gate in the harness (a lock that admits one job at a time) runs L3 dispatches one after another.
Side channels carry the truth. When a tool finds something real (a located file, gathered notebook entries), the tool itself records that payload in a bridge the harness reads later. The text L3 returns to L2 is only a compact control signal ("found the resume; it lists an email address"), never the authoritative data. When the turn ends, answer selection runs unconditionally. If a file was located, a focused synthesis pass answers from the actual file contents; if not, it answers from the gathered notes; only if neither exists does L2's own prose stand as the reply. Exact values (an email address, a policy number, a date) are copied from the source, never from a small model's paraphrase of a paraphrase.

7. The loop around it all: validate, replan, and the assessor we deleted

So far this reads like a straight pipeline. It is actually a loop, because plans fail: the planner picks a plausible skill and the tools come back with nothing usable.

Our first instinct was to add a fourth model: an LLM (large language model) assessor that read the executor's reply and judged whether the plan had succeeded. It was a mistake worth reporting. The assessor systematically misclassified grounded failures (honest answers backed by a real check, like "I checked your files and couldn't find a passport") as capability failures, and sent the planner back to try again.

The result was futile triple replans and roughly a minute per question, to arrive at the same honest "couldn't find it" the first attempt had already produced. The assessor was judging the one thing a model adds judgment to, and judging it wrong.

What replaced it is almost embarrassingly simple, and that is the point. The validate step is pure code: an empty reply fails; any substantive reply, including a grounded not-found, passes and ships. No model call, no latency, no misclassification. The lesson generalizes: put model judgment where only a model can judge (routing, crafting), and put deterministic code everywhere a rule can be written down.

When validation does fail, the harness replans, and this is where the loop earns its keep:

The failure is fed back as a planner note: "Your last plan (@check-schedule) produced no usable answer… Plan differently, using tools that would retrieve what's needed." The planner sees its own miss and routes around it.
The per-turn tool guards (the ledger of already-made calls and the per-tool caps) are reset, so the next attempt may legitimately re-issue calls the failed attempt burned.
Everything is budgeted: up to three plan attempts, plus two plain regenerations for the transient empty-output case, plus the three handle-correction retries inside each planning pass. Budgets nest; nothing is unbounded.
If every budget exhausts, the floor is honest: the best non-empty reply seen so far, or a one-line apology. Never a blank bubble, and never a hallucinated answer (one the model simply invented).

A last piece of hygiene guards the seams: anything streamed to the user is scrubbed of leaked plan syntax (a stray "Plan:" preamble, a bare @handle), so the machinery never shows through in the chat.

Step back and count the signals holding this together: PlanSignal halts L1 the instant a plan exists; ToolLoopHalt halts L2 the instant a budget trips; validate-plus-replan bounds the whole turn. Each is a structural stop (an error thrown through the framework, or a branch in plain code), not a request politely made to the model. That is the recurring shape of this harness.

8. Where this sits, and the moral

Two placement notes. First, the L1–L2–L3 machinery engages on the on-device path when skills are enabled. With no skills selected, the on-device session runs flat with a handful of native FoundationModels tools, and the big external models run the classic loop from §1; their capacity is exactly what makes the decomposition unnecessary there. The harness is scaled to the brain it is harnessing.

Second, this article described one turn of cognition. Which of your devices is allowed to run the agent's autonomous turns at all is governed by a single seam called AgentRuntime, which gates all generative agent behavior to the one device holding the "driver's seat". A direct chat, by contrast, runs wherever you are. That story belongs to Agent and The Agent in Your Day.

The moral of this article is the series' moral in miniature. Between the flat loop that failed and the harness that works, the model did not change at all: the same small, frozen, on-device model runs every level of the stack.

What changed is structure: three one-tool surfaces instead of one many-tool surface; menus and signatures instead of schemas; structured signals instead of parsed prose; deterministic validation instead of model judgment; budgets and thrown halts instead of hope. Small models are not made reliable by asking more of them. They are made reliable by asking less, more precisely, more times.

The single most consequential seam in the stack (the one meta-tool that lets L2 drive any number of real tools through fresh L3 sessions) deserves its own close-up.

Next: Mechanism: CallTool, One Tool to Bind Them, the meta-tool seam in full detail: one-line signatures, inline L3 dispatch, and why the native loop is non-negotiable.