31: Voice and Conversation - transcript
Jessica: The QUILL Cast, episode thirty-one. I'm Jessica. Today: voice and conversation, the part of the product that lets you talk to QUILL and have it talk back, safely, on your own machine.
Liam: I'm Liam, and a quick honesty note before we go anywhere. The brief for this episode said it would recap the accessible vault basics from episode thirty, but the actual episode thirty on disk is dictation. So we'll recap dictation, the offline speech pipeline, and then build on it into voice commands, conversation mode, and the wake word. The vault basics is episode forty and we'll see it later in the series.
Jessica: The recap, in two breaths. Episode thirty covered Locked Dictation, the Whisper-based, fully local flow that turns speech into text at the cursor. The gesture is a single, dependable one: control F9 to start, speak freely, control F9 again to finish, and the words land as one undoable edit. Pause and resume with control shift F9, status with alt F9, escape stops and keeps the audio for review, shift escape discards it. Everything in dictation is private because the audio never leaves the machine. The Whisper model, the model manager, the audio recovery folder, all local.
Liam: And now we step from dictation, which is dictation, into voice, which is control. Dictation types at the cursor. Voice runs commands. The boundary is sharp, and the safety boundary is even sharper. Today's episode covers three surfaces, plus a routing trick: voice command, the push-to-talk gesture for a single command. Voice conversation mode, the hands-free back-and-forth. The Hey QUILL wake word, always listening for the wake phrase. And finally, how a spoken question gets handed to Ask Quill instead of being forced to look like a command.
Jessica: Let's set the frame. All three voice surfaces ride on the same offline engine dictation uses, the same Whisper-family model, the same local microphone capture, the same on-device transcription. Nothing new in the privacy story: the audio is captured, transcribed, and consumed on your processor. The audio is not sent to a cloud speech-to-text service, the transcript is not sent anywhere unless a command you speak explicitly asks the AI assistant to do something, and even then the consent is yours, in the Ask Quill composer.
Liam: The other frame is the safety frame, and it deserves the headline. Voice cannot run arbitrary commands. It can only run commands on a curated, non-destructive list, the same list the AI agent uses when it decides to run a command. That list lives in quill core ai agent, and it's called SAFE_TOOL_IDS. Twenty-four commands today: file save, file save all, file new, undo, redo, select all, bold, italic, uppercase, lowercase, title case, sentence case, bullet list, numbered list, word count, spell check, read aloud start and pause, read aloud stop, next heading, previous heading, outline navigator, find, toggle soft wrap, and command palette. That's it. Voice cannot delete a file, cannot close a document, cannot run a Quillin, cannot fire off an email. "Saying the wrong thing" cannot run a dangerous action because the wrong thing literally cannot reach a dangerous action.
Jessica: And the rule is enforced in code, not in a prompt. After a transcript is resolved to a command, the UI checks the command id against the safe allowlist one more time before running it. If the id is not on the list, the command is dropped, and the status line says so. That's a defense-in-depth: the matcher only proposes safe commands, but the dispatcher double-checks anyway. We verify it line by line because safety is the kind of thing you verify.
Liam: Now the surfaces, one at a time. First, voice command, the simplest one. The menu entry is Tools, Speech, Voice Command, Offline, and there's a dedicated command id, tools dot voice command. The default keybinding is empty, meaning the gesture is whatever you bind it to in the keymap, because voice surfaces are deliberately off until you opt in. The setting is voice commands enabled, and it defaults to false. The same setting gates all three voice surfaces, because they share the same trust model.
Jessica: Press the key, and QUILL announces, listening for a command, run the command again to stop and act. Say one phrase. Press the key again. The recording is transcribed, the transcript is matched against the safe-allowlist catalog, and the matching command runs. The full round trip: your voice to your model to your machine, no network. If nothing matches the phrase, QUILL says so and nothing happens. If the phrase sounds like a cancel, cancel, never mind, nevermind, stop, dismiss, the turn is dropped and you're back where you started.
Liam: The matcher's design is honest about how voice works. It is not a hard-coded if-then tree. It builds the voice-invokable catalog from the command registry, then matches a transcript against it. Exact phrase equality scores one, a phrase appearing as a whole sub-sequence of the transcript scores zero point nine, and token overlap is graded on a fraction of the phrase's words, with a threshold of zero point six. That is a confident number, and the threshold is the floor: ambiguous matches do not run.
Jessica: And every voice-invokable command has aliases, the spoken forms the matcher recognizes, beyond the command's own title. A few favorites: save or save file or save document, undo, redo, select all, bold or make bold, italic or make italic, uppercase, lowercase, title case, sentence case, bullet list, numbered list, word count or count words, spell check or check spelling, read aloud or start reading or pause reading, stop reading, next heading, previous heading, outline or outline navigator, find or search or find text, soft wrap, command palette or commands or show commands. You don't have to memorize the registry titles. You can say any of the aliases.
Liam: Second, voice conversation mode, and this is where the design story hides. Tools, Speech, Voice Conversation Mode, command id tools dot voice conversation. The mental model is that voice command is a walkie-talkie, press, talk, release. Voice conversation is a phone call. You turn it on once, you speak many things, and the loop runs in your hands, with audio cues between every state.
Jessica: The implementation is a five-state machine, and it's in quill core speech conversation. The states are off, idle, armed, review, and busy. The flow: start moves from off to idle and arms the first listen. On the armed state, the microphone is open for one utterance. When a transcript arrives, the state moves into review, which is a brief cancel window, a beat, the matched command's title is announced, and a timer runs for review ms, nine hundred milliseconds by default. If the timer fires, the command dispatches and the state becomes busy. If the user says a cancel phrase, the command is dropped and we re-arm.
Liam: And the timer windows are the part most people don't notice, but they should. They're all user-tunable in settings, with sensible defaults grounded in WCAG timing guidance. Silence ms, two thousand, how long a pause ends an utterance. Review ms, nine hundred, the cancel beat before a matched command dispatches. Follow-up ms, three thousand, stay armed after acting so a follow-up needs no re-arm. Thinking ms, two thousand, the spacing of the still working tick, so a slow-running command stays perceivable. Every window can be set to zero, which disables it. The defaults are not arbitrary, and the escape hatches are not hidden.
Jessica: There's also spoken cue support, off by default, opt in, called voice conversation spoken cues in settings. When you turn it on, and a screen reader is not currently running, QUILL speaks the prompts aloud through the read-aloud voice: a warm welcome the first time, a varied listening prompt each subsequent time, an acknowledgement while working, and a follow-up nudge. The phrases are time-aware, good morning, good afternoon, good evening, and personalized with the optional user name from voice conversation user name. If a screen reader is detected, the spoken cues stay quiet, so QUILL never talks over the reader. That is exactly the right default.
Liam: And the cancel handling has a real piece of craft in it, worth pausing on. The cancel phrases are the same set, cancel, never mind, nevermind, stop, dismiss. A bare stop, just the word, exits conversation mode entirely, by design. The other cancel phrases, cancel, never mind, nevermind, dismiss, only abort the current turn and re-arm, they keep the conversation going. Same surface, two different intents, and the rule is in code. The brief honesty note: the wake word and the push-to-talk surface have the same rule, with the same intent.
Jessica: Third, the wake word, the always-listening surface. Tools, Speech, Listen for Hey QUILL, command id tools dot voice wakeword. The default keybinding is empty. The setting that gates it is voice commands enabled, the same as the others, and the persistence flag is voice wakeword persist, with voice wakeword enabled sitting behind it. Off by default. Always off in Safe Mode.
Liam: The mechanics deserve a moment. The microphone is open in short windows, transcribed on-device, checked for the wake phrase, and discarded. If the wake phrase is not heard, the window closes and the next one starts, and every twelfth idle window, by default, a reminder is played, so a live microphone stays perceivable. If the wake phrase is heard, the trailing words are extracted. Two cases: a bare wake, "Hey QUILL" alone, opens one command turn. A wake plus a command, "Hey QUILL, save file," runs the command straight away. After either case, the loop resumes listening.
Jessica: The wake phrases the controller accepts are Hey QUILL and QUILL. The matcher is a simple prefix-strip, and there is a cooldown, one window by default, to prevent the tail of the wake utterance from immediately re-triggering. That is the only double-trigger protection. It is enough, and it is in the test suite.
Liam: And the inline command runs through the same safe-allowlist gate as push-to-talk and conversation mode. A wake word does not bypass safety. It cannot. A user could shout, "Hey QUILL, delete everything," and the result would be a status line that says so, because there is no delete everything on the list.
Jessica: A do this now beat, and this is the one we'll ask you to pause the audio for. Open Settings, find the speech section, and locate Voice Commands Enabled. Turn it on. Then decide which surface you actually want to try first. If you want the simplest possible introduction, bind a key to Tools, Speech, Voice Command, Offline, and use the key. If you want the conversation feel, bind a key to Tools, Speech, Voice Conversation Mode, and use it once, with a quiet room, and a headset if you have one. If you want the always-listening experience, bind a key to Listen for Hey QUILL, and accept that the microphone will be live in short windows, with a periodic reminder that it is live. Resume the episode when you have a setting flipped and a key bound.
Liam: The reason that one beat is worth the pause: every later conversation in this episode assumes voice is on and a key is bound. If we kept talking, you'd be listening to a tour of a feature you couldn't reach, and that's a waste of your time.
Jessica: Phase four, and this is the small but important routing trick. When conversation mode or push-to-talk receives a transcript that the safe-allowlist matcher cannot resolve to a command, the route is, by default, to play an error cue and re-arm. But if the transcript sounds like a question, the whole hands-free session turns itself off and Ask Quill opens with the question pre-filled. The classification lives in quill core speech voice routing. Three kinds, cancel, question, command. A cancel is the cancel set. A question is either an explicit ask prefix, ask QUILL, ask, question, hey QUILL ask, or a leading question word, what, who, where, when, why, how, which, can, could, would, should, is, are, do, does, did, will, explain, define, summarize, tell. Everything else is a command.
Liam: The hand-off is the part to remember. Voice never fires the AI request on its own. It opens the composer, pre-filled, and turns itself off so the microphone is free for the chat's own voice controls, if you have any. A person presses send. The consent stays with the user. That is the contract.
Jessica: Now a code-verified walkthrough, because we said we would verify. Press your voice command key. The handler in main frame speech is voice command toggle. The first thing it does is check voice commands available against settings and safe mode. The function is in voice commands, and it returns false in safe mode, true only when settings dot voice commands enabled is true. If unavailable, the handler announces that voice is off and exits. If available, it checks whether the mic recorder is already running. If yes, it stops and dispatches. If no, and the conversation mode setting is on, it delegates to conversation mode instead. Otherwise, it starts the push-to-talk recording.
Liam: On the stop path, the recording is handed to the offline provider, transcribed on a background task, and the result is fed to dispatch voice result. That function calls resolve transcript, looks up the outcome, checks the command id against SAFE_TOOL_IDS one more time, and runs the command via the registry. If the outcome is a no match, the status line is updated. The audio is unlinked from disk at the end of the work, on purpose, so no temp WAVs accumulate.
Jessica: The conversation mode path is similar but richer. The conversation controller is created with a Timing object, built from settings, plus a user name and a flag for varied prompts. The first thing start does is sound the conversation on earcon and arm. The arm effect plays a listen earcon, announces a prompt, and starts the capture. The capture uses voice activity detection, a silence detector, to know when a turn is over, with a hard cap as a backstop. The transcript is then resolved and routed, with the question rule from Phase 4 applied.
Liam: The wake word path is the simplest loop. The wake controller lives in quill core speech wakeword. Start transitions from off to listening and announces the listening prompt. Each transcribed window is fed to on window, which checks the cooldown, then extracts the transcript body. A body that is none means no wake phrase, so the controller increments idle windows, schedules a reminder if due, and listens again. A body that is the empty string means a bare wake, so the controller sounds the wake cue and emits an arm effect, opening one command turn. A body that is non-empty means a wake with an inline command, so the controller dispatches it. The result: the same offline pipeline, with the same allowlist, and the same cleanup.
Jessica: Honest corrections, because the brief asked us to call out drift when we find it. One: the previous episode is not vault basics, it's dictation. We recapped dictation. The vault basics lives at episode forty, which we'll meet when we get there. Two: the actual menu labels are slightly more specific than the brief assumed. The push-to-talk entry is Voice Command, Offline, not just Voice Command. The conversation entry is Voice Conversation Mode, not just Conversation. The wake entry is Listen for Hey QUILL, with the Wake Word in parentheses in some surfaces, like the help topic, and without it in the menu. Three: the safe allowlist has twenty-four commands today, and it can grow, but every addition has to clear the same safety gate the agent itself uses. We will update this number in the script as it changes.
Liam: One more correction we want to be honest about. The wake word feature is on disk and tested, and it works, but it is opt in, off by default, and not advertised in the welcome tour. We think the right call is to keep it that way until the always-listening experience is rock solid for screen reader users, because the price of a live microphone is the highest of any voice surface. If you are a power user, the toggle is there, and the help topic explains the affordances.
Jessica: The privacy story across all three surfaces, in one line. The audio is captured, transcribed on your machine, and consumed by an allowlisted command or dropped. The transcript is not sent to a network unless a command you speak explicitly invokes the assistant, and even then, the assistant's own consent flow runs. That is the whole story.
Liam: Now a quick contrast, because contrast teaches. Dictation turns speech into text. Voice turns speech into action. Dictation inserts at the cursor, voice runs a command. Dictation's gestures are control F9, control shift F9, alt F9. Voice's gestures are whatever you bind, plus an optional wake phrase. Dictation is always on as a capability, the model is the prerequisite. Voice is opt in, because the cost of misrecognition is a wrong command, not a wrong word.
Jessica: And the layered design inside voice itself: push-to-talk is the floor, simplest, most predictable, one phrase at a time. Conversation mode is one level up, with state, with timers, with the warm cues, with the ability to chain commands. The wake word is the top of the stack, always listening, but with a periodic reminder and a hard rule that bare stop turns the surface off. Three surfaces, one engine, one allowlist, three trust postures, and you choose which one fits the moment.
Liam: A craft note that matters more than it sounds. The audio cues are not decorative. They are the affordance that lets a hands-free session be perceivable to a screen reader user, a sighted partner in the room, or a recording workflow. The conversation on cue, the conversation listen cue, the conversation wake cue, the conversation review cue, the conversation ready cue, the conversation idle cue, the conversation thinking tick, the conversation error cue. Nine cues, one per state, all in quill core speech conversation. You can customize each, or quiet them all, but the default is intentionally audible.
Jessica: A settings tour, briefly, and the names are the names in settings specs. Voice Commands Enabled, the master switch, default false. Voice Conversation Enabled, when true, makes the push-to-talk key behave as a conversation toggle. Voice Conversation Silence Ms, default two thousand. Voice Conversation Review Ms, default nine hundred. Voice Conversation Followup Ms, default three thousand. Voice Conversation Thinking Ms, default two thousand. Voice Conversation Spoken Cues, default false. Voice Conversation User Name, optional, used in the welcome. Voice Wakeword Enabled, default false, gates the wake word surface. Voice Wakeword Persist, default false, lets the wake word survive a restart.
Liam: And the safety gate, one more time, in case anyone has been skimming. The voice commands available function returns false in Safe Mode, no matter what the settings say. That is the same guarantee the AI assistant's run command path uses, the same SAFE_TOOL_IDS, the same floor. If you turn on Safe Mode, all three voice surfaces turn off. The brief was emphatic about this and the code agrees.
Jessica: Alright, a do this now beat for the second half. Open Tools, Speech, Speak Voice Status, or press the key you bound to tools dot voice status. QUILL announces what voice is doing right now. If the wake word is on, you hear, listening for Hey QUILL. If conversation mode is on, you hear, listening, or working, or got it, depending on the state. If push-to-talk is recording, you hear, listening for a command. If nothing is on, you hear, voice is not listening right now. This is the moment-of-truth status check, and it is the same gesture that satisfies the mic-live perceivability requirement.
Liam: Homework, four steps. One: turn on voice commands in Settings, bind a key to Tools, Speech, Voice Command, Offline, and run three commands: save, find, and bold. Notice the latency, the success rate, the error announcements, and the zero network traffic. Two: turn on Voice Conversation Mode, bind a key, and have a small back-and-forth, two or three commands in a row, and observe the audio cues, the cancel window, and the follow-up window. Say, cancel, mid-thought, and notice the difference from saying, stop. Three: turn on Listen for Hey QUILL, bind a key, and try the two wake shapes, a bare wake, "Hey QUILL," and a wake with an inline command, "Hey QUILL, outline." Notice the reminder, and notice that a bare, stop, turns the surface off. Four: try a question, "what is the next heading," and watch the conversation surface hand off to Ask Quill with the composer pre-filled, and the session off.
Jessica: One more thought before we close. The first version of this surface was much less safe. We removed a press-and-hold mode for the same reason dictation did, because a held key repeats and announces itself endlessly. We made a wake word opt in, and a voice command opt in, and a conversation mode opt in, because the cost of a live microphone is real, and the audience is using screen readers. Honest subtraction is a feature.
Liam: And the brief asked us to set up episode thirty-two. Next episode: transcription and the Listening Companion, the pipeline that takes whole recordings in and finished documents out. We already touched the offline engine today, and on Tuesday we'll show the rest of the path, capture, transcribe, transform, with the transcript actions that turn a wall of spoken language into the document you actually needed. The voice surface in this episode was one of three doors into the speech stack. Transcription is the second.
Jessica: That's the close. I'm Jessica.
Liam: I'm Liam. Say the words, and let QUILL answer.
Jessica: Fifty-four episodes total, twenty-three to go, and a wake phrase to keep an eye on.