The turn

One turn, four processes

A turn begins when the browser sends stop_listening and ends when the gateway emits response_complete. Between those two events the gateway allocates a turn identity, transcribes, opens a socket, streams, chunks, synthesises and cleans up — and at four separate points re-checks whether the turn it is working on is still the current one.

Fresh per turn, then pooled forever

The gateway opens a new unix socket for every single turn (always_assistant/worker_client.py): connect, write one JSON line, read newline-delimited events until {"event": "final"}, close. That is simple and cheap and has no state to corrupt.

The worker cannot do the same thing, and the docstring of voice-daemon/jcode_server_client.py explains exactly why:

So the worker keeps an OrderedDict pool keyed by session id, with LRU eviction at eight entries and a 1,800-second idle TTL. A connection that has never been used is stored under the literal key "__new__" until the agent reveals which session it was actually given, at which point _remember_under_id() re-keys it. Session discovery is opportunistic: any event carrying a session_id field wins, and failing that, a swarm_status event with exactly one member is treated as authoritative.

Two ordering details in chat() are worth reading, because both are comments recording a bug that was actually hit:

# resume_session emits an ack/error and possibly history.
# Wait for it to finish before sending the message.
# Otherwise a fresh jcode-server can route the message
# into its default session before the requested session
# is attached, silently forking browser chat context.

# jcode-server applies set_model in-order but does not
# emit a done frame for it; waiting here forces a
# timeout and falls back to the slow subprocess path.

The five-part prompt

The user's words are not sent alone. chat_with_worker_stream() assembles between three and five blocks, joined by blank lines, and the transcript arrives last:

1. The persona and format prefix. Configurable via ALWAYS_PROMPT_PREFIX; the default is quoted below.
2. A runtime-context directive naming the harness and the active model alias, ending with instructions on how to answer “what model are you” — the model is told to give the alias and to mention that llama.cpp is only the backend runtime.
3. A visible-reasoning directive, only when the browser has thinking mode on. It demands exactly two XML-ish sections, <always_reasoning> and <always_final>, which the gateway then splits apart with two compiled regexes.
4. A web-search directive, only when search is enabled, whose last clause is “Do not say you lack internet before trying the available … web tools.”
5. User request: <transcript>.

“1 to 3 short spoken sentences” is the single most consequential line in the configuration. Everything downstream — the sentence chunker, the 220-character speech split, the 320-character ambient buffer cap — is sized on the assumption that it is obeyed.

The chunker starts talking before the answer exists

Waiting for a complete answer before synthesising it adds the whole generation latency to time-to-first-audio. So the gateway maintains a growing foreground_tts_buffer and, on every text_delta, asks _pop_streamable_spoken_chunk() whether enough has arrived to be worth speaking. It is a four-rule decision:

Reconciliation at the end of the turn is deliberately blunt. _remaining_tts_text() compares normalised strings by prefix only — no diffing, no alignment. If the streamed prefix matches, only the tail is spoken; if the final answer is shorter than what was already said, nothing is spoken; and if the model rewrote its answer mid-stream so that neither is a prefix of the other, the entire final answer is spoken again. That last case is a real duplicate-audio path, accepted in exchange for a function that is four lines long and cannot itself fail.

Five verbs

The voice worker is a newline-delimited JSON server on an AF_UNIX stream socket, chmod 0600, listen(8), one daemon thread per connection. Its entire vocabulary is five operations. Anything else gets a single, very specific error string:

$ SOCK=~/.hermes/voice-daemon/jcode-worker.sock

# op 1 of 5 — liveness. also lazily ensures the agent process is up.
$ printf '{"op":"ping"}\n' | socat - UNIX-CONNECT:$SOCK
{"ok": true, "pid": 3184177, "rpc_pid": 3184240}

# the op the README and ARCHITECTURE.md both still document:
$ printf '{"op":"subscribe_ambient","session_id":"…","model":""}\n' | socat - UNIX-CONNECT:$SOCK
{"ok": false, "error": "unknown op: subscribe_ambient"}

# the streaming path the gateway actually uses:
$ printf '{"op":"chat","prompt":"say hi","session_id":"","stream":true}\n' | socat - UNIX-CONNECT:$SOCK
{"event": "connection_phase", …}
{"event": "text_delta", "text": "Hi"}
{"event": "text_delta", "text": " there."}
{"event": "message_end", …}
{"event": "tokens", "input": …, "output": …}
{"event": "final", "ok": true, "answer": "Hi there.", "session_id": "…"}