32: Transcription and the Listening Companion - transcript
Liam: The QUILL Cast, episode thirty-two. I'm Liam. Today: transcription and the Listening Companion, the pipeline that takes whole recordings in and finished documents out, with the local engine doing the heavy lifting and the AI doing the shaping.
Jessica: I'm Jessica. Quick recap of episode thirty-one. That was voice and conversation: the push-to-talk voice command surface, voice conversation mode, the Hey QUILL wake word, and the offline Whisper-family engine they all ride on. Three things to carry into today. One, the audio stays on the machine, captured, transcribed, consumed locally. Two, voice only runs commands on a curated safe allowlist, twenty-four commands, with the allowlist re-checked in the dispatcher after the matcher proposes. Three, when a spoken turn does not look like a command, it can route to Ask Quill with the composer pre-filled, so the consent stays with the user. Today we step from voice, which is the door that listens, into transcription, which is the door that records, and into the Listening Companion, which is the part that turns the raw transcript into the document you actually needed.
Liam: Before we go anywhere, an honesty note about file names. The brief for this episode references a file called quill core bits whisperer, and the previous script referenced it too. That module does not exist on disk. What exists is quill core ai transcript actions, and what it carries forward from the old BITS Whisperer project is the spirit and the action catalog, not the file. The PRD calls this out under section five point eighty-four b. The Listening Companion is the name, transcript actions is the menu, action builder is the customization surface, and the code is in quill core ai and quill ui. We are not going to keep calling it by a module that no longer ships.
Jessica: Let's set the frame for the day, in three parts. The pipeline has three stages: capture, transcribe, transform. Capture is whatever produces audio or video: a meeting recording, a lecture, an interview, a voice memo, a screen-captured webinar, the dictation holdover from last week, anything that has sound and a beginning. Transcribe is the part we will spend most of our time on, because it is the engine, and the engine runs locally. Transform is the Listening Companion itself, the part that takes a wall of spoken language and gives you back the document you needed in the first place. Three stages, one tab, no upload to a third-party speech-to-text service unless you choose the cloud path.
Liam: A do this now beat before we get into the code, because the rest of the episode assumes you can reach the feature. Open QUILL. Open the speech menu under Tools. Find the entry called Transcribe Audio or Video, Offline, with the parenthetical. Click it. The dialog that opens wants an audio or video file, a transcript format, and a model. If you have not installed a model yet, the dialog offers to open Manage Speech Models and download one, with a progress dialog that has a Minimize to status bar button so a long download does not pin the screen. If you do not have an audio file handy, that is fine, the rest of the episode works with the description, but the install piece is worth doing now, because the cloud path will not work without a key and the offline path will not work without a model. Resume the episode when you have either a working Transcribe Audio or Video, Offline command, or a clear plan to come back to it.
Jessica: One clarification on what the offline path will and will not do, because the brief is full of marketing language and we want to be exact. The offline path transcribes on your machine, using one of the Whisper-family models in the catalog. It can identify different speakers, when you have a speaker-detection model installed, and the heuristic is in main frame speech: it prefers a model that the catalog flags as a diarization model and uses that, otherwise it picks your default. It supports plain text, SubRip, WebVTT, Markdown, and HTML output formats. It does not translate. The translate-to-English feature is the cloud path, because it is OpenAI Whisper translation, and the offline path is intentionally the private one. If you need to transcribe a non-English recording on your own processor, you get a transcript in the original language, and you can still run transcript actions on it, and the actions are language-tolerant, but the result document will be in the same language as the input. That is a real boundary, and it is the right one for a privacy-first feature.
Liam: Now the file format story, because people hit it and the answers are not always obvious. The offline transcribe dialog accepts these formats: wav, mp3, m4a, aac, flac, ogg, opus, wma, mp4, m4v, mov, mkv, webm, avi. The comment in the code at quill ui main frame speech line fifteen says, transcribe and captions accept these, ffmpeg transcodes them to sixteen kilohertz mono WAV. ffmpeg is the audio format helper, and it transcodes whatever you give it into the sixteen kilohertz mono WAV the model actually wants. ffmpeg is GPL and LGPL, and QUILL does not bundle it. There is a separate command, download ffmpeg, about one hundred and ten megabytes, fetched from the official Gyan.dev build on an explicit user action. Install it from Tools, Speech, and the offline dialog will start accepting anything ffmpeg can decode, which is essentially everything. If you do not install ffmpeg, you are limited to the formats the model engine reads natively, which is a smaller list. The magic is in the on-demand ffmpeg.
Jessica: And the speaker diarization story, which deserves a paragraph. There are two diarization paths in QUILL, and they are not the same. The cloud path, the OpenAI Whisper one with the Translate checkbox, uses Deepgram for speaker identification, and it needs a Deepgram key in the AI Hub. The offline path uses a Whisper-family model that has speaker detection built in, like the pyannote-style models, and the catalog flags it as a diarization model, and the UI prefers it when one is installed. The transcript text in the offline path comes out with labels like Speaker zero, Speaker one, plus the speaker's name if diarization can identify one. The transcript has speakers detection in the action recommender, transcript has speakers in transcript actions, looks for two or more distinct speaker labels in the text and routes the chooser accordingly. Cloud gets you more accurate labels, offline gets you labels and the privacy story. Both are honest, both are real.
Liam: Two things we want to correct from earlier episodes, because we verify and we want to be honest. One: the previous script and the brief describe a feature called translate during transcription in the offline path. It does not exist. The translate-to-English checkbox lives on the cloud path, AI, Transcribe Audio, Transcribe Audio File, and it routes to OpenAI's translation endpoint, quill core ai transcription translate file. The offline path does not translate. Two: the previous version of this episode listed nine transcript actions, Meeting Minutes, Action Items, Executive Summary, Interview Notes, Study Notes, Q and A, Follow-Up Email, Key Quotes, Clean Up and Draft. The list has grown. There are now ten built-in actions, and the one we missed before is Decisions Log. The code defines ten: meeting-minutes, action-items, executive-summary, interview-notes, study-notes, qa-extraction, clean-draft, follow-up-email, key-quotes, decisions-log. The test suite covers all ten and the watch folder profile lists all ten in the Then make dropdown. We are calling that out because the previous episode got it wrong, and the test in tests unit core ai test transcript actions would have failed against the old script's claim. Honest subtraction and honest addition are both features.
Jessica: Stage three: the Listening Companion. This is the part that earns the name. Nobody actually wants a transcript. A transcript is a wall of spoken language, full of filler, false starts, half-finished thoughts, and no structure. People want what is in it: the minutes, the action items, the executive summary, the study notes, the quote, the decision log, the follow-up email. Enter the transcript actions. When a transcript is ready, the post-transcription flow in main frame calls offer transcript actions, and a single-choice dialog appears with the question, what would you like me to make of this. The choices are the built-in actions, ordered for this transcript, and one opt-out, Just keep the transcript. Pick one, and the action's plain-language instruction is wrapped around the transcript and sent to your AI provider through the unified ProviderChatBackend, the same backend the AI Hub uses for Ask Quill and the AI Library. The result opens in a new buffer, with the file name prefixed Created action-name from, so the original transcript is never overwritten, and the action is reviewable and editable before you commit.
Liam: The ordering, since we said we would verify. The recommender in transcript actions looks at the transcript and asks two questions: does it have speakers, and is it question-dense, where question-dense means at least three question marks and more than one percent question marks per word. Four cases. Multi-speaker and question-dense, like an interview: lead with Interview Notes, Key Quotes, Q and A Extraction, Meeting Minutes. Multi-speaker not question-dense, like a board meeting: lead with Meeting Minutes, Action Items, Decisions Log, Follow-Up Email. Question-dense single voice, like a Q and A webinar: lead with Q and A Extraction, Study Notes, Key Quotes, Clean Up and Draft. Plain single voice, like a lecture or a voice memo: lead with Clean Up and Draft, Study Notes, Executive Summary. Every action stays available, the user is never boxed in, the order just puts the most useful action first. The test suite asserts all four cases with real transcript snippets, and the same test that locks in the multi-speaker case also locks in the new Decisions Log and Follow-Up Email leading the meeting list.
Jessica: And the actions are adjustable, which is the part that makes them a tool and not a toy. Every action's build prompt accepts an extra instruction, the user's own plain-language adjustment, focus on the budget decisions, keep it under one page, list only the items with a deadline. The action's canned instruction is prepended, the extra instruction is appended as additional instructions from the user, and the transcript is appended. That is the whole prompt. The same action, same transcript, two different extra instructions, two different results. The post-transcription chooser does not show this lever in the simplest form, because it is a single-choice dialog, but the AI, Transcribe Audio, Transcript Actions menu item, which runs on the current selection or document anytime, is the door to the same magic from any document, not just a fresh transcript. If a long report is open, you can ask for Key Quotes on it, and that is excellent. The status line always tells you when there is nothing to work on, when AI is off, and when a provider is not reachable, because for a screen-reader user a silent menu item is the worst possible failure.
Liam: The customization tier: the Action Builder. The built-ins cover common cases. The Builder covers yours. It is a no-syntax form, in quill ui action builder dialog, with five fields. A name, a one-line identifier for your action. A Start from dropdown, with the option Blank, write my own, and the ten built-in actions as starting points, so you can clone the Meeting Minutes instruction and edit it. A plain-language instructions box, multi-line, the body of what you want. An Attach Reference button, with a file dialog that accepts txt, md, and docx, where docx goes through the optional markitdown converter when it is installed, and the reference text is read and capped at eight thousand characters, because a template or a prior good example only needs to fit in a screenful of context. And a Preview button when the host supplies a preview callback, so you can run the action against a test transcript and see what comes back before you save. Save, and the action becomes a real Skill, written through action to skill source, and installed into the AI Library, where it shows up alongside the built-in transcript actions in the same chooser, with Run, Edit, Enable, Export, and Promote available for free.
Jessica: How you reach the Action Builder, because the menu path is easy to miss. Open the AI menu. Open AI Library. The AI Library is the unified manager for prompts, skills, and agents, and it has a verb set that includes Build. The Build verb opens the Action Builder, and on Save, the new skill is installed and the library reloads so it is immediately visible. If you are already in the AI Library, you do not have to leave it. There is no separate Build Action button on the main AI menu, because the PRD consolidates the verb set into one manager. That is a design choice, and it is the right one, because a separate menu item would scatter the surface. From the AI Library you can also reach the existing actions, including the transcript actions, and the watch folder profiles, and the engines, and the sessions. The AI Library is the front door. The Transcript Actions menu item is the shortcut. The Action Builder lives behind the Library door.
Liam: A small thing worth saying out loud. The reference attachment is the part most people skip and most people should not. The instruction is the what, the reference is the look. An agenda, a house style guide, a prior good example, last month's minutes, all of these can be attached. The action is saved with the reference text baked in, and the model is told, use the following reference as the template or example to match in style, structure, and terminology. This is how a support lead builds Complaint Summary that looks like last quarter's. This is how a study group builds Flashcard Prep that follows the professor's outline. The Builder is not a general-purpose prompt editor. It is a transcript-action editor with a reference slot, and the slot is the lever that turns a generic action into your action.
Jessica: The automation crown, the part that ties transcription to the watch folder story from episode twenty. The watch folder profile, the per-folder settings dialog in main frame watch profile, has a Then make dropdown on its transcribe action. The dropdown is populated with None, just the transcript, and the ten built-in transcript actions. Pick one, and a recording that lands in the watch folder gets transcribed and then has the chosen action run on the new transcript, and the result document is saved next to the audio. Recorder syncs to the folder, minutes await you. The committee secretary's whole job shifts from typing while listening to reviewing a draft, which is the part that actually needs a human. The implementation is in watch transcribe, underscore maybe make action document, and it is Do-Not-Disturb-aware, so a watch folder never fires off an action while you are presenting or recording. AI off, no provider, or a failed action skips the document step with a clear note and always keeps the transcript. The transcript is the floor. The action is the ceiling.
Liam: The honest story about the AI connection for this whole stage. The transform tier, the Listening Companion, the transcript actions, the action builder, all of it runs through the AI provider you have configured in the AI Hub. If you have not set AI up, the post-transcription dialog does something better than a dead end: it offers the AI Setup Wizard, the same wizard that Ask Quill and the AI Library offer, with a tailored reason, to turn this transcript into minutes, action items, or notes, QUILL needs AI. The wizard has two real paths, an on-device Ollama path, which is private and free, and a cloud account path, with Claude, OpenAI, Gemini, OpenRouter, or Ollama Cloud, with a key pasted once and stored in the OS secure store. There is a Get API key button on every keyed provider. Setting AI up once makes the AI Hub, Ask Quill, the AI Library, and the Listening Companion all work, because they all share the same backend. The transcription itself, stage two, does not need AI. It runs locally. So the experience is, transcribe any recording on your machine with no key and no internet, and only when you want a document made of the transcript, you turn AI on. That is a kind permission ramp.
Jessica: One more thing to call out, because it matters for the screen-reader audience. The offer transcript actions chooser is a wx single-choice dialog, and single-choice dialogs are well-behaved for screen readers. The labels are formatted as name, dash, description, and the descriptions are short. The opt-out is the last item, Just keep the transcript. There is no reviewable text inside the dialog, because the action is the reviewable thing. The status line announces, Creating action-name, when the action starts, and the result opens in a new buffer with a clear file name. The progress dialog has a cancel button for long runs, and a minimize to status bar option so a long generation does not pin the screen for someone who would rather keep working. These are small details, but they are the difference between a feature you can use and a feature that uses you.
Liam: A code-verified walkthrough, because we said we would verify. The offline path starts in main frame speech, transcribe audio offline. The provider is the same offline Whisper-family provider that powers dictation and voice. The model selector prefers a diarization model when one is installed, otherwise the user's default model. The format chooser is a single-choice dialog with text, SubRip, WebVTT, Markdown, HTML. The file dialog accepts the audio and video wildcard we listed earlier. The work is dispatched on a thread behind the AI progress dialog, with cancel supported. On completion, the result is handed to open transcription result, which formats it through the speech formatters module, plain text, markdown, or html, and opens it in a new document tab. The announcement says, Transcription complete, with speaker labels if any, the word count, and a review the draft transcript prompt. That is the offline path end to end.
Jessica: The cloud path starts in main frame, ai transcribe audio file, which calls underscore ai open transcribe dialog with translate to English set to false. The dialog, AI transcribe dialog, has a file picker, a supported formats string built from SUPPORTED AUDIO EXTENSIONS, a language chooser with thirty-eight options plus auto-detect, a diarization checkbox that gates Deepgram, a maximum speakers spinner, and a translate checkbox. The two checkboxes are mutually exclusive in the dialog, so you cannot ask for translation and diarization at the same time, which is the right call. On submit, the handler asks for the OpenAI key, and if there is none, it offers a path to the AI Hub and points you at the offline path. With a key, the work runs on a thread, the transcribe file or translate file function from quill core ai transcription is called, and the result is handed to show transcription result, which calls offer transcript actions first and only falls back to the plain transcript dialog if the user opts out or AI is not ready. That is the cloud path end to end.
Liam: The Listening Companion. Show transcription result calls offer transcript actions from quill ui transcript actions. That function checks the provider backend, returns False if AI is off, and on False offers the AI Setup Wizard. With a backend, it calls recommend actions on the transcript, builds the chooser labels, and shows a wx single-choice dialog. The opt-out returns False. A real pick is dispatched to underscore run action, which spins up a worker thread, opens the AI progress dialog with the action's name, and calls generate action text from transcript actions with the backend. The result is opened in a new buffer through the power-tools open-text-in-new-buffer path, with the file name Created action-name from file-name. The worker thread pattern is GATE-40-OK compliant, the UI thread is never blocked, and the work survives a screen reader dialog dismissal. That is the Companion end to end.
Jessica: The Action Builder. The verb lives in ai library dialog, skill build, which constructs an Action Builder dialog. The dialog reads the BUILTIN TRANSCRIPT ACTIONS list for its Start from preset dropdown, positions the Blank option first, and waits. The user enters a name, picks a preset or Blank, writes instructions, optionally attaches a reference. On preset change, the instructions box is pre-filled with the preset's instruction, and the name box is pre-filled with My preset-name if it is empty. Preview runs the action against a test transcript if the host supplies a callback, otherwise it is hidden. Save calls action to skill source with the name, the instruction, and the reference text, and adds the resulting source to the Skill Store. The saved skill is a real Promotable Skill with Run, Edit, Enable, Export, Promote. That is the Builder end to end.
Liam: Honest corrections, one more pass, because we are going to live with this script and we want it to be true. One: there is no quill core bits whisperer module. The transcript logic is in quill core ai transcript actions. The PRD calls BITS Whisperer the ancestor, the docstring in action builder calls it the descendant, and we are not going to keep using the old file name. Two: the offline transcription path does not translate. Translation is the cloud path. The previous script implied it was a flag on the offline path, and it is not. Three: there are ten built-in transcript actions, not nine. The previous script missed Decisions Log, which is the action that extracts just the decisions made, each with its rationale and owner, and it is a real action with a test. Four: the action builder is reached through the AI Library's Build verb, not from a top-level menu item. Five: the magic for the post-transcription chooser only shows up if the dialog is in the path, and it is, in show transcription result, and the AI Setup Wizard is the on-ramp, not a dead end.
Jessica: One more honest thing. Live and diarized streaming transcript actions were deliberately not built. The PRD says so. The value is already delivered by file transcription plus watch folder automation, and the one genuinely unique kernel of a live version, real-time captioning, is a separate accessibility feature that would need real-time audio capture infrastructure. If it is ever wanted, it should be scoped on its own merits, not bolted onto this Companion. So when you ask, can I get live captions of this meeting as it happens, the answer today is no, the path is record, transcribe, action, and that is a known decision. The watch folder plus the Then make dropdown is the closest analog, and for most workflows it is the better one, because you can review the action result before it ships.
Liam: A do this now beat for the second half. Pick one audio file you already have, even a voice memo, and transcribe it. Tools, Speech, Transcribe Audio or Video, Offline. Pick the default model, the text format, and let it run. When the transcript opens, look at the chooser. Notice that the lead action is the one we said it would be, and that all ten are still there. Pick two different actions, Meeting Minutes and Key Quotes, and compare what each kept. Notice the difference between a structured document and a curated list of lines. Resume the episode when you have a transcript and two action results, and a sense of which one fits this recording.
Jessica: A craft note, and the kind of thing that makes the difference between a tool you tolerate and a tool you trust. Microphones. A cheap conference microphone improves transcripts more than any setting. We say this in every episode that touches audio, and we will keep saying it, because it is true. Name your recordings meaningfully, because the filename rides the whole pipeline, into the transcript, into the action result, into the watch folder's next-side document. For long recordings, run Executive Summary first, read it, and only then decide which deeper actions the recording deserves, because ten seconds of summary can save ninety minutes of processing the wrong file. And keep your AI provider working, the way you would keep your printer working, because a transcript action without a working provider is a chooser that offers you Just keep the transcript as the only honest option.
Liam: Homework, four steps. One: transcribe any recording you have, even a voice memo, just to see the pipeline run, and notice the announcement, the chooser, and the result buffer. Two: run two different transcript actions on the same transcript and compare what each kept, in tone, in structure, in length. Three: open the AI Library, find the Build verb, and sketch your own custom action, the one you would build for a document you produce monthly, save it, and run it on a transcript of your choice. Four: if you use watch folders, attach a transcribe action to one and a Then make action, and let a real recording flow through, and notice that the result document lands next to the audio without you being there. That is the pipeline running itself.
Jessica: Next episode: the audio studio. We step from text into sound, from the editor into the player. We will cover the chapter workbench, the speech engine picker, exporting chaptered audiobooks, the DAISY talking book format for accessible libraries, and the on-demand download of the audio extras, mpv for gapless playback and MP3 chapter markers. The same care we took with transcription carries forward, the audio is yours, the consent is yours, the result is reviewable.
Liam: A reminder, since this is the end of episode thirty-two. The QUILL Cast is fifty-four episodes total, twenty-two to go after this one. We have done the first thirty-two: the foundations, the speech stack, the safe subtractive design, the offline engines, and now the pipeline that turns recordings into documents. What is ahead is the audio studio, the accessible vault, the trust and community finale, and the episodes in between. The transcript of this episode is on the site, and the source for every claim we made today is in the code, in the PRD, and in the test suite.
Jessica: That's the close. I'm Jessica. Stop taking minutes. Start taking review passes.
Liam: I'm Liam. Fifty-four episodes total, twenty-two to go, and a transcript that knows what to do with itself.