HomeTechnical ArchitecturePhilosophyContactRequest access
All articles

GroupThink

Mechanism: The Driver's Seat

22 min read

An election with no election service, decided by a two-byte payload riding a message your devices already send each other. The interesting move is not the vote — there is no vote — but the decision to stop treating "holds the seat" and "is driving" as the same fact, which is what lets a device hand over control by first giving up the half it can retract on its own authority and keeping the half whose absence would be irreversible.

Prerequisites: Who Acts for why exactly one device may run autonomous work, and The Relay for what a sealed envelope hides from the server carrying it.

1. Two failures, and why one phase cannot dodge both

Who Acts named the two ways this goes wrong. Orphaned control: nobody drives, every device politely deferring to a sibling, and the symptom is silence with no error anywhere. Sustained overlap: two devices drive, so notes are written twice, nudges fire twice, and your agent turns up twice in one huddle in front of other people. They pull in opposite directions, and the one available knob — how long you wait before presuming a device is gone — slides you along a single curve between them.

Now make it concrete. Device B holds the seat and device A should take it. B stops holding at some instant, A starts at some other instant, and the two are separated by a message crossing a network — so one of them is first. If B is first there is an interval with no holder, and if A then crashes or never receives the message, that interval never ends. If A is first there is an interval with two holders. There is no third ordering, so a one-phase handoff — a single release, or a single take — can only choose which window it gets, never avoid both.

ONE MESSAGE CROSSES THE NETWORK — SOMEONE IS FIRST ORDER 1 · B STOPS FIRST B HOLDS A HOLDS NOBODY ORPHANED CONTROL IF A THEN CRASHES, IT NEVER ENDS ORDER 2 · A TAKES FIRST B HOLDS A HOLDS BOTH SUSTAINED OVERLAP TWO DRIVERS UNTIL A'S TAKE REACHES B A ONE-PHASE HANDOFF ONLY CHOOSES WHICH WINDOW IT GETS

The mechanism's response is to make one of the windows harmless rather than to try to close it. It splits the token from the authority to act, so a device can be the nominal holder while having already stopped doing anything. That is why there are two phases and not one: quiesce stops the acting without dropping the token, and release drops the token only against evidence.

2. An order nobody has to negotiate

An election normally means negotiation: propose, vote, break ties, and repeat when the round is lost. Every one of those steps is a message that can go missing, and the thing being negotiated — who deserves to lead — is a moving target.

DeviceRoleService refuses the negotiation by fixing a total order in advance. Rank is chronological: the earliest-registered device is Primary, then Secondary, Tertiary, and after that rankWord says "Device 4". The order comes out of a query in identity-svc never written for this purpose:

SELECT pubkey, mldsa_envelope_pubkey FROM identity_records
 WHERE user_id = $1 AND revoked_at IS NULL ORDER BY created_at DESC

DESC — newest first, described in its own docstring as "a cheap most-recently-added proxy" for the device-sync path that actually needed it. So refreshOrder reverses it:

let chronological = Array(newestFirst.reversed())   // oldest=primary fallback

That single .reversed() is the whole election's source of authority. Note also that the list includes this device — the field's comment says so explicitly, "INCLUDING self (identity-svc raw)" — because the index of self in the ranked list is the only thing reconciliation reads.

Now the payoff. Because the order is total, stable and identical on every device, a device's decision reads exactly one thing: the slice of the list above it.

let higher = rankedOldestFirst[..<myIdx]

Nothing below myIdx is ever consulted. So the devices never have to agree about who should win — the ranking already says — only about who is present. And the two-driver failure becomes describable in one sentence: if A and B both drive with A ranked above B, then B concluded that nothing above it was online, which means B believed A was offline. A double drive is always a disagreement about presence, never about rank. That collapses the election into a failure-detection problem, which is the part you could not have avoided anyway.

One escape from the ranking, and it fails open: a device that cannot find itself in the list — a brand-new device, or, more commonly, a roster fetch that returned nothing — takes the seat unconditionally so the agent never stalls. Who Acts treats that as the largest hole in the design; it is the same hole here.

3. A two-byte claim on a channel contacts cannot see

The rank is half the answer. The other half is liveness, and liveness needs a channel.

The obvious channel is the server that already holds the roster, and it is the one the design refuses: a presence table on identity-svc is a minute-by-minute record of when you pick up which device, held by a service that knows whose account it is. The next-most obvious is a new message type on the relay — new subject, new handler, one more thing the relay can tell apart from everything else.

What shipped is neither. The seat assertion rides the sealed same-account channel that already carries notebook and memory deltas:

let delta = Payload.syncDelta(SyncDelta(
    kind: "heartbeat", payload: Data([holdsSeat ? 1 : 0, requestReply ? 1 : 0]),
    updatedAt: UInt64(Date().timeIntervalSince1970)))
_ = try? await compose?.sendSealed(delta, to: sib)

Byte 0 is the seat assertion. Byte 1 is a request that the sibling beat straight back — the on-demand ping that §7 depends on. A one-byte payload from an older build decodes as requestReply = false, so the frame grew a byte without a version negotiation.

Three things fall out of that choice.

No election service. Nothing in the deployment knows the seat exists — no lease to renew, no lock row, no endpoint that has to be told you are alive.

No new wire type. The receive path already had a syncDelta case; the seat is a new kind string inside it, dispatched in InboxService beside memory and the reconcile rounds.

No contact-visible presence, which matters most and is least obvious. The envelope goes to pair.<subject>, where the subject is HKDF-SHA256(channelKey, "pixie-pair-subject:v1", "epoch" ‖ recipientPubkey ‖ epoch) — the same derivation your conversations with other people ride, with the shared sibling key standing in for a contact's channel key. No pubkey, phone number or account id feeds the routing value, and it rotates hourly. On the receive side, InboxService applies a syncDelta only when the sender's userIDHash equals your own.

So the asymmetry is structural rather than a permission check: whether you are online is readable by anything holding your sibling channel key, and nothing else holds it. A contact cannot learn it — their key derives different subjects, and their frames fail the self-gate. The relay cannot either: it sees an opaque 32-byte subject and a blob, and the blob is two bytes of plaintext under a layer it cannot open.

SIBLING CHANNEL KEY — BOTH DEVICES HOLD IT DEVICE A RELAY DEVICE B RIDES SYNCDELTA · KIND: HEARTBEAT BYTE 0 — ASSERTS THE SEAT BYTE 1 — ASK FOR A BEAT BACK SUBJECT = HKDF(SIBLING KEY, EPOCH) ROTATES HOURLY · NO ACCOUNT ID OPENS: SEAT · REPLY? WHO CAN READ THE HEARTBEAT SIBLINGS HOLD THE CHANNEL KEY RELAY ∅ OPAQUE SUBJECT + SEALED BLOB CONTACT ∅ DIFFERENT SUBJECTS · SELF-GATE PRESENCE IS READABLE ONLY UNDER THE SIBLING CHANNEL KEY

4. Quiesce: evidence of life without evidence of driving

The state the first phase exists for is narrower than it sounds.

There are two inbound liveness signals. noteSiblingHeartbeat(_:holdsSeat:wantsReply:) records both freshness and the sibling's seat assertion. noteSiblingHeard(_:) fires for any other self-gated frame — a self-echo, a memory delta — and its comment is careful: it "proves liveness, but carries no seat assertion, so leave the last-known assertion as-is." Quiesce is exactly the gap between those two facts: I know you are alive, and I have no evidence you are driving.

That gap is not exotic. Your Mac has been driving because your phone was asleep. The phone wakes, and before its own reconciliation has run, the Mac's ping arrives; pong replies immediately with whatever holdsSeat currently is, which for a device that just launched is false. The Mac now knows the phone is alive and is not driving — as it would from a plain memory delta arriving from a sibling that has not beaten yet.

What the Mac does then:

} else if higherOnline {
    if !yielding {
        yielding = true
        await onYieldSeat?()
    }
}

holdsSeat is untouched. Every subsequent heartbeat still asserts the token. But isDriving is holdsSeat && !yielding, so it goes false on the same line — nominal driver, no longer acting. That is the lame duck.

Why this is the safe intermediate is an argument about what a device can do unilaterally. Stopping work needs no confirmation from anyone: the instant I have evidence that a better-ranked device is up, I can stand down on my own authority, and overlap is prevented immediately. Dropping the token is the opposite — it is the only act that can create an orphan, and its safety depends entirely on a fact about someone else. So the device gives up the half it can retract by itself and keeps the half whose absence would be irreversible.

One honest wrinkle. The header describes quiesce as stopping work and flushing context, and onYieldSeat is a real hook for exactly that. In the shipping app it is never assigned, and the reason is recorded where the wiring would go: memory events already emit a live sync_delta, notebook and saga edits converge through the digest-reconcile round in Staying in Sync, and the takeover path verifies the shared state root before it drives — so it pulls anything not yet converged rather than depending on the yielding device having been prompt about pushing it. The hand-over-with-context step was designed, built as a seam, and then deliberately left unplugged because the sync layer made it redundant.

5. Acquire: a device only ever looks upward

The other branch is three lines:

if !higherOnline && !higherAssertsSeat {
    holdsSeat = true
    yielding = false
    if let onAcquireSeat { Task { await onAcquireSeat() } }
}

Read the two conjuncts against their definitions. higherOnline is "some device above me is fresh"; higherAssertsSeat is "some device above me is fresh and asserts the seat." The second implies the first, so !higherOnline already implies !higherAssertsSeat, and the test reduces to !higherOnline. The redundancy is the interesting part: it writes down, in the condition itself, that an assertion only counts when it arrives with liveness. A dead holder's last claim is a fact about the past, and the code is shaped so no later rearrangement can accidentally let it block anybody.

Now the case that matters. A lower-ranked device still asserting the token — the lame duck from §4 — does not block acquisition, and this is not a special case anywhere in the code. It falls out of rankedOldestFirst[..<myIdx]: a lower-ranked device's assertion is invisible to the acquiring device by construction. The new driver takes the seat on top of the old one, and there is no instant in between where nobody holds it.

The steal is the same branch with time doing the work. If the holder dies, nobody will ever confirm anything, so the confirmed path cannot complete. Instead its lastHeard timestamp ages past the presence window, isFresh goes false, higherOnline goes false, and the next device down acquires. That timeout is the mechanism's only one, and it exists solely for the case where positive evidence is impossible.

Release — phase 2 — is the mirror of quiesce:

if holdsSeat {
    if higherAssertsSeat {
        holdsSeat = false
        yielding = false
    }

Not a timer. The token is dropped against a specific observed fact: a higher-ranked device, fresh and asserting, in a frame this device decrypted itself. And the third branch of that same if makes quiesce reversible — if no higher device is online any more, yielding goes back to false and the lame duck resumes driving. A device that stood down for a sibling that then vanished does not stay stood down.

6. What the invariant does and does not promise

The invariant is one sentence: the yielding device never drops the token until it has seen the new driver assert it. Two consequences, one comfortable and one not.

PHONE RANK 1 MAC RANK 2 AWAKE · NOT YET ASSERTING HOLDS + DRIVES ACQUIRE — SEAT TAKEN ON TOP DRIVES LAME DUCK — HOLDS ONLY QUIESCE — STOP ACTING, KEEP TOKEN RELEASE — ASSERT SEEN PONG: ALIVE · SEAT = 0 HEARTBEAT: SEAT = 1 · ONE HOP BOTH HOLD — NO ORPHAN A LOWER-RANKED ASSERT IS INVISIBLE UPWARD — IT CANNOT BLOCK THE ACQUIRE THE TOKEN DROPS ONLY WHEN THE NEW DRIVER IS SEEN ASSERTING

The comfortable one: no window during a handoff has nobody holding the seat. The old holder is still holding at the moment the new one takes, so a taker that crashes mid-handoff leaves the old holder in place — quiesced, but present, and §5's third branch resumes it once the taker stops looking alive.

The uncomfortable one: the double-hold window is real by design, and the double-act window is narrowed rather than eliminated. Device A acquires and begins driving the moment its own reconciliation runs, which is before anything it sends has reached B; B stops acting when A's frame lands. So both devices genuinely act for as long as one heartbeat takes to cross the relay, and no local decision can shorten that, because B cannot react to a fact it has not received. What the mechanism removes is sustained overlap: the two-driver state ends on a positive event — the assertion arriving — rather than on a timeout, and the window is one relay hop rather than one presence window.

That is the trade §1 said you cannot escape, taken deliberately: the design accepts a bounded overlap window because overlap has per-activity backstops — watermarks, per-contact cooldowns, day stamps — while an orphaned seat has no symptom at all.

7. Asking the question is what produces the answer

isActiveAgent() looks like a predicate. It is not; it performs I/O and can take a second and a half.

if let last = lastHandshakeAt, now.timeIntervalSince(last) < Self.handshakeDebounce {
    await refreshOrder()
    await reconcileSeat()                            // reuse fresh presence
} else {
    lastHandshakeAt = now
    await beat(requestReply: true)                   // ping every sibling
    if settleDelay > 0 { try? await Task.sleep(for: .seconds(settleDelay)) }
    await reconcileSeat()                            // decide on the pongs
}
return isDriving

The first call in a handshakeDebounce window (5 seconds) pings every sibling with the reply-request bit set, waits one round trip (settleDelay, 1.5 seconds in the app and 0 in tests), and only then reconciles; calls inside the window reuse the presence the first one paid for. Pongs never request a reply of their own, so a ping cannot loop, and replyThrottle (2 seconds) stops a chatty sibling from making this device beat in a storm.

There is no periodic heartbeat anywhere. An idle account generates no liveness traffic at all, which means presence is manufactured on demand at the moment a decision needs it. That inverts the property that made the naive race unavoidable: because one inbound envelope fans out to every device at once, all of them handshake on the same event and converge exactly when there is work to converge about.

What the seat gates is narrow. AgentRuntime is the single funnel — run(activity) { … } returns nil when this device is not driving, isAgent(activity) serves call sites that prefer an early return, and both call straight through to DeviceRoleService. The activity value is not an input to the decision; it documents intent and anchors handoff for the resumable activities.

The line it draws is generation, not action. Sending a message from a non-driving device is completely untouched: it goes to the recipient and self-echoes to your siblings exactly as it would from the driver. Filing an inbound message, acking it, writing the memory event, applying a thread title a sibling chose — all ungated, on every device. What the gate covers is work the agent invents: a note-taking distillation, a thread name (an LLM call), a proactive nudge, a huddle turn. One feature can land on both sides of the line, and the side is decided by where the non-determinism sits.

8. Overriding the order, and showing the work

Registration age has nothing to do with fitness — it knows nothing about battery, thermal headroom, which device can run which model, or which one is in your hand. So Settings → My Devices lets you drag the list, and Save writes SettingsKey.deviceOrder: a comma-separated list of hex pubkeys, top priority first.

That setting has to sync, and §2 says why: the guarantee there was that devices never disagree about rank, so all disagreement reduces to presence. A purely local reordering breaks precisely that invariant, and the breakage is two drivers, each top of its own private list. So the CSV rides the synced-settings CRDT and every sibling elects the same order.

applyUserOrder then has to survive a preference older than the device set:

let ordered = userOrder.filter { present.contains($0) }
let appended = chronological.filter { !orderedSet.contains($0) }
let result = ordered + appended
return result.isEmpty ? chronological : result

Keep the user's order for devices that still exist; append devices the preference has never heard of at the end; drop devices no longer present; and if any of that leaves nothing, fall back to chronological. Every clause preserves a total order over the live set in the face of a stale list.

The result is rendered back by deviceStatuses(reachable:fallbackRoster:), whose most interesting behaviour is what it does when it cannot vouch for anything: if the roster handshake failed, this device shows .offline and every other shows .unknown — the enum has a third case specifically so the UI can decline to guess.

One deliberate exception to the ordering. forceClaimSeat() takes the seat out of band, for mid-huddle failover: a sibling that has confirmed the driver dropped needs the seat now, not after the predecessor times out. It sets the token and then beats — and because beat() reconciles before it sends, a claimer that is wrong about the predecessor being gone still steps down for a device that is verifiably online right now. The tests pin both halves. The point of claiming loudly is that a returning ex-driver sees the assertion and hands off through §4 and §5 instead of double-driving. The reverse hook, onAcquireSeat, closes the loop: acquiring the seat is itself the trigger to check whether a huddle needs taking over.

9. The residual

Partition is not handled, and it is the honest hole. Acquisition is a local belief about a remote fact. If two devices cannot exchange heartbeats but both remain able to do work, the lower-ranked one's isFresh for the higher goes false after the presence window and it acquires, while the higher never sees anything that would make it release. Both drive, and nothing here bounds how long that lasts — it ends when the partition ends. This is availability chosen over exclusion, and the per-activity backstops are what make it survivable. The layer above does solve its version of the problem: before seizing a dropped driver's live huddle, attemptWitnessedTakeover asks the other members' devices whether anyone still sees it, and a single "I still do" vetoes the takeover — external witnesses, unanimous absence required, no responders at all meaning you may be the isolated one. The seat election has no such witness, because the only witnesses available to it are the devices whose reachability is in question.

Presence lags, and the lag is a round-trip budget, not a heartbeat interval. Since there is no periodic beat, "online" means "answered our most recent handshake within presenceWindow" — 8 seconds — and the gate waits settleDelay, 1.5 seconds, for pongs before it decides. A sibling whose relay round trip exceeds that budget reads as offline and the seat may move to a device that should not have taken it. Eight seconds is aggressive on purpose; 1.5 seconds on a cellular hop is the number actually at risk.

The gap after a hard death is measured from the next piece of work, not from the death. The confirmed release cannot fire when there is nobody to confirm, so the steal path runs — but only when something calls the gate. In exchange, an idle account whose driver has died costs nothing, because there is nothing that wanted driving.

Instantaneous overlap remains, bounded by one relay hop, as §6 derived. A reordering has a propagation window, since the override is a synced setting, so two devices can briefly hold different orders — reintroducing, for a few seconds, the one condition §2 ruled out. And the seat is a filter, not a lock: holdsSeat alone is not the exclusion property, isDriving is, and code that reads the token believing it asked "am I the agent" asked a different question.

References & further reading

  • Wikipedia, Bully algorithm — the classical shape of an election decided by a fixed priority order rather than a vote; §2 and §5 are a quiescent, gossip-driven variant of it.
  • Wikipedia, Failure detector — the abstraction §2 reduces the election to; Chandra and Toueg's 1996 paper on unreliable failure detectors is the standard treatment of what one can and cannot promise.
  • Wikipedia, Split-brain (computing) — the §9 partition case by its usual name.
  • RFC 5869, HKDF — the key-derivation function behind the rotating pair subject that makes a heartbeat unreadable and unlinkable to the relay.
  • Who Acts — the two failure modes, the alternatives that were rejected before this mechanism, and the backstops that make a bounded overlap window survivable.
  • Staying in Sync — the convergence loop that made the yield-time context flush in §4 unnecessary.
  • Algorithm: Who Gets to Put You in a Group — why a sibling taking over a live huddle has to be added to the group mid-session, and who is allowed to add it.
  • Agent — what the gated activities actually do once a device is allowed to run them.