HomeTechnical ArchitecturePhilosophyContactRequest access
All articles

GroupThink

Mechanism: Taking Over a Live Huddle

12 min read

The driver's seat decides which of your devices does autonomous work, and it decides it privately: get it wrong and you get duplicate notes nobody else ever sees. A huddle is the one case where the decision is visible to other people. Your agent is mid-conversation with your friends' agents when the phone holding the seat goes under a tunnel — and now the failure mode is not a duplicate note, it is a group that stalls waiting for a participant who is never coming back.

Prerequisites: The Driver's Seat, which establishes the seat and the two-phase handoff this extends, and Who Acts for what a huddle is.

1. A commitment other people can see

The seat problem has two failure modes, and The Driver's Seat solves both for the private case: never orphaned (no device driving, work silently not happening) and never sustained overlap (two devices driving, duplicate work). Both are recoverable. A duplicated nudge is noise; a missed one arrives late.

A huddle changes the stakes because the commitment is external. Your device is a member of a group whose other members are other people's devices, taking turns in a structured round. Those peers are waiting on you. If your driver drops:

  • The group does not know whether you are thinking or gone.
  • The round cannot close, because a turn that never arrives is indistinguishable from a slow one.
  • And you cannot simply re-join from another handset, because membership is cryptographic — the group's key schedule knows your device, not your account.

So the private seat protocol is necessary and not sufficient. Taking the seat makes a sibling willing to drive. It does not make it a member of the group it now has to drive.

2. A beat your siblings can hear and the cohort cannot

The first requirement is that a sibling knows a huddle is happening at all. It cannot learn this from the cohort — it is not in the cohort, and putting it there would leak your device fleet to your friends.

So a live driver advertises to its own account. The marker carries what a successor would need:

struct ActiveHuddleMarker: Codable, Equatable {
    var cohortID: Data
    var memberPubkeys: [Data]   // each member's in-huddle device pubkey
    var memberHashes: [Data]    // each member's user_id_hash (parallel array)
    var convener: Data
    var lastBeat: UInt64        // unix seconds — freshness for drop detection
    var stateRoot: Data?        // the driver's shared progress at this beat
}

It rides the sealed sibling channel, so the cohort roster never leaves the account. That is the asymmetry that makes the whole mechanism work: your siblings learn the roster of a group they are not in, and the group learns nothing about your siblings.

The beat is deliberately unconditional — a live driver emits it every few seconds whether or not the round is progressing. That decoupling is what makes silence meaningful. If the marker stream were tied to turn progress, a driver that was merely thinking hard would look identical to one that had dropped, and the cutoff would have to be set to the worst-case turn length. Instead:

static let driverDropCutoff: TimeInterval = 8
static let markerFreshness: TimeInterval = 20

Eight seconds of silence from a stream that beats every few seconds is a real drop, not a slow turn. That is what lands failover in roughly ten seconds — inside the window the round model tolerates, rather than waiting on the much slower seat clock.

3. Silence is not evidence, so ask the room

Eight seconds of silence is evidence of a drop, but it is evidence gathered from one vantage point — yours. The sibling watching the marker stream is on a different network from the driver. The thing it actually observed is "I stopped hearing beats," which is consistent with the driver being gone and equally consistent with the sibling's own connectivity having degraded.

Seizing on that alone would produce exactly the failure the seat protocol exists to prevent, only now in public: two of your devices driving one huddle, both taking turns, both visible to your friends.

So before it seizes, the watching sibling asks every other member of the cohort a single question: have you heard from this driver recently? The peers are in a better position to answer than the sibling is — they are the ones the driver has been taking turns with.

A single sighting vetoes the seizure. That asymmetry is deliberate: "somebody still sees the driver" is a positive fact from a witness with direct evidence, while "nobody answered" is consistent with the poll itself having failed. One yes is conclusive; the absence of yes is not.

The probe is answered only by a live in-huddle member, and only for a requester that owns a member account — the same authorisation primitive the takeover itself uses, so the poll cannot be turned into an oracle for whether a given cohort is live.

4. Exactly one peer serves the takeover, and it is usually a joiner

Once the sibling decides to take over, it publishes a KeyPackage and asks the cohort to readmit it. It asks every member, because it does not know which of them are reachable — but only one of them may act, or two concurrent MLS adds would collide on the group.

The serving peer is therefore deterministic: the lowest user_id_hash among members other than the requester's own account. Every member computes the same answer from the roster it already has, with no negotiation and no round trip.

Now the observation that made this fail in practice for a long time. The deterministic serving peer is not the convener. It is whichever member happens to sort lowest — which, most of the time, is someone who joined the group via a Welcome rather than creating it.

That matters because a member's ability to admit another member is not automatic. It depends on the group configuration that member persisted when it joined. A joiner whose stored configuration omitted the ratchet tree can hold a perfectly valid membership, take turns normally, and be unable to add anyone — the Welcome it emits is missing the very structure the next joiner needs to build the group. Every ordinary test adds from the group's creator, so the defect is invisible until exactly this path runs: the one where the admitting member is, by construction, not the convener.

The fix is that a joined group carries the ratchet-tree extension, and that a group already persisted without it is repaired when it is next loaded — because a configuration written at join time outlives the code that wrote it.

5. The add happens inside the turn loop

The serving peer does not admit the new driver on the spot. It hands the pending add to the running session, which services it between turns.

This is a concurrency decision, not a stylistic one. An MLS commit mutates the group's key schedule. Applying one from a network callback while the session is mid-turn — encrypting, decrypting, advancing its own epoch — is a data race on the group state with an outcome no test would reproduce reliably. Servicing it at a known point in the loop makes the add ordered with respect to everything else the session does.

The receiving member still validates before it acts: the requester must own an account that is a current member of the cohort, proven by the sealed sender certificate rather than by anything the requester asserts. The relay never sees this check; it happens after decryption, against the account-bound identity in the envelope.

6. What is handed over, and what is deliberately rebuilt

The snapshot the serving peer returns carries only what every device in the huddle holds identically: the deterministic round machine, the broadcast transcript, the floor lines, the per-slot discussion, and the session-isolation nonce.

It deliberately omits three classes of thing:

  • Self-relative machine views — how many turns I have taken on this slot, which lines are mine. These are rebased to the new driver rather than transferred, because they describe a position in the round that the new device does not occupy.
  • The retrieval context, which the taker rebuilds locally from its own synced Notebook.
  • Per-round private notes, regenerated at conclusion from the shared transcript.

The distinction is what makes a peer able to serve a takeover at all. A snapshot containing your private state could only be served by another of your devices — and your devices are precisely the ones that might be offline. Restricting it to shared state means the person who hands your phone the huddle back can be someone else's phone entirely, holding nothing of yours.

7. The check that warns rather than refuses

The marker carries a stateRoot — a deterministic hash of the driver's shared progress at its last beat. The taker compares the peer-served snapshot against it.

A mismatch does not stop the takeover. It logs, and the session resumes anyway.

That is worth arguing rather than asserting, because "verify then proceed regardless" reads like a check that does nothing. What a mismatch actually means is that the serving peer is behind the dropped driver's last published position — or that the driver advanced past its own last marker before dropping. Neither is evidence of an attack; both are ordinary consequences of an asynchronous group where the marker and the snapshot are produced by different devices at different moments.

And the alternative is worse. Refusing to resume on a mismatch strands the huddle: the peers keep waiting, and the one device willing to continue has declined to. The snapshot is the best state any peer holds, so the choice is between resuming from it and abandoning the group. The check earns its place as a signal that reconciliation is needed, not as a gate.

What it is not is a repair mechanism. The taker does not pull the missing progress before it drives. It proceeds on what it was given.

8. Coming back is not the same as taking over

When the original device returns, it does not stay away. It reclaims — and a reclaim is a different protocol from a takeover, because the returning device is still a member.

Its membership was never revoked; it simply stopped participating. So there is no MLS add. The returning device syncs its existing membership forward through the commits it missed, and the sibling currently driving hands over the live state inline and steps out.

The request looks identical on the wire to a takeover request, and is distinguished by a single comparison: the requester's user_id_hash equals the server's own. Same account, therefore a reclaim rather than a peer-served admission. It also bypasses the deterministic serving-peer gate, which would otherwise exclude the very device being asked — that gate selects among members other than the requester's account, and in a reclaim the requester's account is ours.

9. Removing a leaf you cannot name

The step-down leaves the group holding two leaves for one person: the sibling that was driving, and the one that just reclaimed. The leave model here is that a member never commits its own removal — the sponsor posts the Remove — so the reclaiming driver has to prune the leaf of the device that stepped down.

The obvious way to do that is to name the leaf by the device's public key. The obvious way does not work, and it failed silently for a long time.

MLS leaf credentials in this system are minted per plan inside the FFI. A leaf identity is an opaque per-group credential, unrelated to the device's envelope key and not derivable from it. A removal keyed on a device pubkey therefore matches no leaf at all and fails unconditionally — while looking, from the outside, exactly like a removal that had nothing to remove.

Only the leaving device can read its own credential. So it stamps it into the handover snapshot as it steps down, and the reclaiming driver removes precisely that leaf. The field is device-relative rather than shared, which is why it sits outside the shared-state root: folding it in would break the root comparison on every honest reclaim, since the marker root is published from a snapshot that never carries it.

One more scoping rule, easy to miss and load-bearing on an account with several devices. The handed identity is honoured only when it arrives from the exact sibling being reclaimed from — the verified sender key must match. Account scope alone is not enough: a second sibling racing a snapshot into the same window would otherwise have its leaf pruned, while the device that actually stepped down kept its own.

10. The residual

A failed prune has two shapes the code cannot tell apart. Either the leaf was already gone — a stale handed identity, harmless, and the resume must continue — or the Remove merged locally and the commit never reached the group, in which case it is this device that is now out of step with the cohort. The error surfaces the same either way, so the resume proceeds and says so, on the argument that aborting on the benign case would kill an otherwise healthy reclaim. The alternative is a session that keeps running while every peer frame lands as "cannot decrypt", which is the worse silence.

Takeover is bounded by the poll, not by the seat. If the cohort is unreachable — everyone offline, or a network partition that isolates all of them — the sibling cannot gather the witness evidence it needs and will not seize. That is the correct trade for a group commitment, but it means a huddle can stall in a partition rather than migrating.

The resumed state is the best any peer had, not necessarily the driver's last. §7 is a warning, not a repair. In the mismatch case the huddle continues from a position slightly behind where the dropped driver actually got to, and the lost progress is whatever that driver had advanced past its final marker.

None of this makes the group aware of your fleet. Which is the constraint that shaped every other decision here: the peers see one participant whose device key changed mid-conversation, which is exactly what a re-admission looks like, and they never learn that a second device existed.

References

  • RFC 9420, "The Messaging Layer Security (MLS) Protocol" — Welcome messages, the ratchet tree extension, and why a member's ability to admit another depends on state it persisted at join time.
  • L. Lamport, "Time, Clocks, and the Ordering of Events in a Distributed System," CACM 1978 — the reason a missing beat is not an event and has to be corroborated by other observers.
  • Wikipedia: Split-brain (computing), Failover.

Back to The Driver's Seat, whose two-phase handoff times the step-down this article hands state across, or across to Who Gets to Put You in a Group, which governs who may admit a device to a cohort in the first place.