22: Word, EPUB, PDF and Friends - transcript
Liam: The QUILL Cast, episode twenty-two. I'm Liam. Today, the bridge between QUILL and the rest of the world, the in and the out for Word, EPUB, PDF, and every other file format your day hands you. Other people's formats, treated with fidelity on the way in and dignity on the way out.
Jessica: I'm Jessica. Before the new material, a quick recap of episode twenty-one. That was the deep dive on rich formatting and hidden codes, the screen-reader-first answer to the oldest problem in word processing. The headline was the separation. The editor buffer is clean plain text, exactly what a screen reader was designed to read. The formatting you set, bold, italic, fonts, colors, alignment, named styles like Quote and Title, rides along as invisible codes inside the canonical QUILL markup. Pandoc attribute spans for runs, fenced divs for paragraphs, pagebreak as its own marker line. Describe Formatting at Cursor, Ctrl plus Shift plus D, reads the codes in effect at your caret. Reveal Codes, Alt plus F3, shows the whole stream in a synchronized pane. None of the codes show up in the buffer, none are read aloud, none count as characters. The editor stays clean so every other feature still works on the same text the screen reader already understands. The Illumination sidecar carries the same codes into plain text files, with a sha-256 hash to detect when another program edited the text. One source of truth, four materializations.
Liam: That is the foundation. Today's episode is what happens when that source of truth meets the formats the rest of the world uses. The hidden codes you met last episode do not stay hidden forever. They are about to materialize into real Word styles, real RTF runs, real HTML attributes. And on the way in, every format the world throws at you, .docx, .doc, .pptx, .xlsx, .pdf, .epub, .html, .rtf, .odt, .csv, .tsv, .tex, even Apple .pages, is about to be turned into a QUILL tab you can actually read.
Jessica: The philosophy first, because it explains the design. QUILL's home base is plain text and Markdown, the most accessible format alive. But the world sends attachments. So QUILL meets the world where it is, opens nearly everything readable, and exports back to what recipients expect, while keeping your working copy in the home format where every QUILL power applies. The formats are dialects, and QUILL speaks them all politely. The conversion engine is open and honest about what it knew and what it guessed, and the home format stays the source of truth.
Liam: A code-verified tour of the menu, because the surface is wider than it looks. File menu, Import submenu, Export submenu. The Import submenu opens with a free-first tool called Import slash Convert Document, and underneath it the per-format menu items, Markdown, HTML, Word Document, OpenDocument Text, Rich Text Format, EPUB Book, CSV slash TSV Table, LaTeX slash TeX. Then a separator, then Batch Conversion, then Other Pandoc Format, the door to anything not on the curated list. We verified the curated list against quill-core-pandoc-formats, the file that owns the Tier-1 input and output frozensets, and the import and export menus read from those frozensets. The Tier-1 list is a QUILL product decision, not a reflection of what Pandoc itself supports.
Jessica: The Export submenu is similar but with one important addition: PDF. Markdown, HTML, Word Document, OpenDocument Text, Rich Text Format, EPUB Book, PDF Document, Plain Text, then DAISY Talking Book, which is its own non-Pandoc pipeline. Same separator, same Batch Conversion, same Other Pandoc Format. The asymmetry is honest: PDF is a Tier-1 export only, not an import. Issue two-sixty-two states it directly. PDF is export yes, import no, because PDF text extraction is unreliable enough to be its own feature, the one episode twenty-three covers, and the one we will set up at the end of this episode.
Liam: Here is the do this now beat, before the deep walkthrough. Pause the audio. Open QUILL. Open the File menu. Find Import. Tell me, out loud or to yourself, which entries are present and which are not. Then open Export, and do the same. If a Markdown and a Word Document entry are both there, the menu is in its standard state. If PDF Document is in Export but not in Import, that is correct, by design. If something is missing entirely, that is also a real answer, and we will explain what to do about it at the end of the segment. Take thirty seconds. Press play when you have looked.
Jessica: Welcome back. Whatever you saw is correct, and the rest of the episode will make more sense now that you have looked at the actual menu. If PDF Document was missing from Export entirely, the most common reason is that Pandoc itself is not installed. QUILL does not bundle Pandoc, it is an optional external tool, and the menu's PDF entry hides itself when the tool is missing. The check is one call: get_external_tool_status for the pandoc key, in quill-core-external-tools. If you want the PDF entry back, the door is Help, Download Optional Components, and the download itself is a single file fetched over HTTPS with a pinned fingerprint, no silent installs, no surprise downloads.
Liam: Let us walk one round trip, the most common one, against the code. The function is import_document in quill-ui-main-frame, the handler behind every per-format import menu entry. You pick a file, QUILL hands it to Pandoc with the Tier-1 format name and an output format of markdown, always, because the editor buffer is Markdown. The result text becomes a new Document, the new tab is created, focus is restored to the new editor, and the status line announces, Imported file-name-dot-extension as Markdown. The aria-live region announces the same thing. The original file is untouched. That is the design: import always brings the file in as Markdown text.
Jessica: The export round trip is symmetric, and it is where last episode's hidden codes pay their rent. The function is export_document, right next to import_document. The buffer is saved first, in its canonical QUILL Markdown form, then Pandoc is invoked with from-format gfm plus hard_line_breaks. The hard_line_breaks piece is load-bearing. The editor is line-oriented, one editor line is one paragraph, so bare gfm, where a single newline is a soft wrap, would join your lines into one paragraph in every exported format. The hard_line_breaks extension preserves the line-as-paragraph convention. Then Pandoc writes the target format, and your Markdown headings become genuine Word heading styles, your attribute spans become real Arial fourteen blue runs, your fenced divs become centered or double-spaced paragraphs. The same source of truth, materialized into the recipient's dialect.
Liam: Honest correction, because we promised to be honest. A few things in older docs need sharpening. The conversion is not magic. Pandoc is a real program on your machine, and a misformatted source produces a misformatted target. The fix is in the editor, not in the conversion. The hidden codes from last episode materialize only into the formats that actually carry formatting. Word, yes. RTF, yes. HTML, yes. EPUB, partially. PDF, only via Pandoc's LaTeX intermediate, so PDF is the one export where the round trip is not always lossless for every font. Plain Text, the eighth export entry, is intentionally not a full materialization. Plain text cannot hold a font or a color, so QUILL gives you a choice through the plain-text-with-formatting setting, and the Illumination sidecar keeps the codes in a separate file next to the text. That is the honesty the design called for.
Jessica: Now the formats, one by one, the way the menu lists them, with the design intent and the lossiness for each.
Liam: Word, both modern .docx and legacy .doc. Headings arrive as Markdown headings, so the episode-seven navigation, F6 to the structure pane, H to jump by heading, works instantly on your boss's report. Lists arrive as lists. Bold and italic ride along as spans, real attribute runs. Going the other way, exporting to Word, your Markdown headings become genuine Word heading styles, which means your document is navigable for other screen reader users too. You are producing accessible Word files by default. The Word import also has a special structured view, the WordDocumentSurface in quill-ui-word-view. The setting word_open_mode, with values prompt, structured, or text, decides which surface opens. Prompt is the default, and the prompt offers a special label, Open in structured Word view, and a remember this choice checkbox. The structured view is a real tree of headings and paragraphs, an accessible surface in quill-ui-word-view, fully tested and un-gated. For .doc files the legacy binary format, the same paths apply with the same code, no special case. For a structured Word file, the surface exposes the document outline, lets you arrow through it, and reads cleanly with NVDA or JAWS.
Jessica: PowerPoint and Excel, .pptx and .xlsx, plus their legacy forms .ppt and .xls. These are Tier-1 import formats. The conversion extracts slide text and cell values into a flat Markdown buffer. The structure is best-effort. A slide with a title and three bullets becomes a heading and a list. A spreadsheet becomes a pipe-delimited table, one row per line, which is rough but readable, and for anything beyond a quick read, the right tool is still Excel, not a text editor. The lesson: QUILL can open these, but the right home for a live spreadsheet is a spreadsheet. Use QUILL for the read, not for the work.
Liam: EPUB, the format that gets the most care. Open an .epub file and QUILL parses the zip, walks the NCX table of contents, and resolves each chapter to its HTML file. The chapter files are joined with form feeds so chapter boundaries survive into the buffer, and the chapter titles come from the TOC when one exists, falling back to file stems when the EPUB is tocless. The code is in quill-core-epub, the dataclasses are EpubBook, EpubChapter, and EpubHeading, the loader is load_epub_book, the renderer is render_epub_book. The reading experience is what makes EPUB special. The Navigator tool, bound under Navigate, EPUB Navigator, opens a tree of chapters, and inside each chapter, a tree of H1 through H6 headings. Pick a chapter and QUILL opens it in a new tab with the chapter title as a Markdown H1 heading. Pick a heading and the caret jumps to that heading inside the chapter. The method is open_epub_navigator in quill-ui-main-frame, the tree builder is _build_epub_navigator_nodes, and the chapter renderer is _render_epub_chapter_text. Reading an actual book becomes actual reading.
Jessica: The EPUB export is a Tier-1 Pandoc output, and it is the right target for finished long-form drafts. Pandoc produces a real .epub file with a generated table of contents from your heading structure, a proper OPF package, and a navigation file. The result opens in any EPUB reader. The round trip is good for text-heavy books. For image-heavy or richly designed books, expect to do some clean-up in a dedicated EPUB editor after the export. The honest limit: Pandoc makes a standards-compliant EPUB, not a designer EPUB.
Liam: PDF, the honest conversation. A PDF is a picture of a page that sometimes remembers its text. When it does, QUILL extracts it. The code is in quill-io-pdf, the function is extract_pdf_text, and the result is a PdfExtractionResult dataclass with five fields: text, quality_score, engine, page_count, and extracted_pages. Two extractors are tried in order, pdfplumber and pypdf, and the first one that returns non-empty text wins. The quality score is zero to one hundred, computed from characters per page, words per page, and pages successfully extracted. The function _score_pdf_text is a simple heuristic, no machine learning, just a sum of three bounded subscores, and a score below fifty triggers a fallback to MarkItDown. Pages are joined with form feeds, not blank lines, so real page boundaries survive into the editor and the page navigation code we built for EPUB works on PDFs too. The import menu offers Import slash Convert Document for the heavy case, but plain Open on a .pdf uses the same PDF extract path. The result lands in your tab with a header that says Engine colon pdfplumber, Quality score colon seventy-three out of one hundred, and the text below.
Jessica: The honesty feature, and this is the part that separates QUILL from a tool that says trust me. When the PDF has no text layer at all, the result is not a silently blank document. The result is a clear message in the buffer, No selectable text was found in file-name. It is likely a scanned or image-only PDF, use File, Import and choose OCR to read it. That message is the result, and it is the honest answer. The same is true for the no-extractor-installed case, which produces a different message that points the user at Help, Download Optional Components. Two different problems, two different remedies, kept distinct by the code in the early portion of quill-io-pdf. Issue nine-zero-nine is the originating ticket, and the fix is part of the on-demand unbundle of the PDF and Office text extraction.
Liam: The on-demand download. The PDF and Office text extraction is not bundled in the installer. It is downloaded the first time QUILL needs it, when the user picks Import slash Convert Document and the file is born-digital. The download is a single archive from a verified URL with a sha-256 fingerprint check. No silent install, no scheduled background fetch, no telemetry. The user clicks, QUILL fetches, the menu items light up. The same pattern covers the Whisper speech model, the Kokoro voices, the spell dictionaries, the assets-v1 unbundle program. The user is in control of every byte that lands on the disk.
Jessica: HTML and RTF, the in-between formats. HTML opens as a flat text view, scripts and styles stripped, structure preserved where the source had it. RTF opens through a similar path, and the rich-text hidden-codes storage we built last episode, the Pandoc attribute spans and fenced divs, is the same model RTF files are converted into on open. That is why a Word document with the same formatting imports cleanly into QUILL. The path is symmetric, the storage is shared, the round trip is lossless for the attributes we set. RTF exports land as real RTF, and the formatter engine in quill-io-rtf knows how to materialize the same attributes into real RTF runs.
Liam: OpenDocument, the .odt format from LibreOffice and OpenOffice. Tier-1 import and export. Headings, lists, emphasis, and the OpenDocument paragraph styles map cleanly. A document saved from OpenOffice Writer round-trips with its structure intact. The less common cases, .odp for OpenDocument Presentation and .ods for OpenDocument Spreadsheet, are not Tier-1 and are not in the menu, but Pandoc knows them. The door is Other Pandoc Format.
Jessica: CSV and TSV, the table formats. The default open mode is text, one row per line, pipe-delimited, readable in any screen reader. The setting csv_open_mode offers a second choice, grid, which opens the file in the CsvGridSurface, a real accessible ListView grid in quill-ui-csv-grid, fully tested. The grid is the right home for a real working spreadsheet, the text mode is the right home for a quick read. The two-mode prompt appears the first time you open a CSV in QUILL, and the choice is remembered per file via the source metadata.
Liam: LaTeX and TeX, the academic formats. Tier-1 import and export, the extensions are .tex, .ltx, and .latex. The conversion preserves section headings, emphasis, code blocks, and the most common math delimiters. Custom commands and complex packages will not round-trip cleanly, that is the honest limit, but the typical article or paper imports into a clean Markdown view and exports back into a workable .tex file. The LaTeX export is the input to Pandoc's PDF path, by the way, so when you choose Export as PDF, that is the route the conversion takes underneath.
Jessica: Plain text, the format that does not actually exist as a format. A .txt file is just bytes, and the only thing it can carry is the bytes themselves. The export flow uses the Illumination sidecar from last episode. The setting plain-text-with-formatting has three values, Ask, Illuminate, Plain. Ask, the default, offers a three-way choice every time you save a formatted document as text. Illuminate always writes the sidecar. Plain always saves plain and drops formatting. The sidecar lives at file-name-dot-txt-dot-illumination, next to the text file, and the model is in quill-io-illumination. The sha-256 staleness check means if another program edits the text, the Illumination is dropped, the file opens as plain, and the formatting is reported as lost in the status bar. That is the only honest answer.
Liam: Apple .pages, the format with two routes. Route A is pure-Python IWA parsing, the keynote-parser package, fast and offline, used when the package is installed. Route B is LibreOffice plus MarkItDown, the slower path with higher fidelity, used when keynote-parser is not installed. If neither route is available, the import returns a graceful error message that names the two remedies. The code is in quill-io-pages, and the heading-style map translates Pages paragraph styles to Markdown heading prefixes at the top of the file. Pages is not in the standard Import menu because it is a Mac-only format with low install base, but Other Pandoc Format plus a manual extension check covers the case for users with the package installed.
Jessica: The import menu's free-first sibling, the one that leads the submenu. Import slash Convert Document is the routing tool for every file the user might want to rescue, and it is its own episode, the one we will preview at the end. The short version for today: the tool always tries the free local converter first, the MarkItDown package, and the OCR is only offered when the result looks scanned. The tool is gated on free-first, local-first, and consent-first, the three laws every QUILL feature that touches the network has to honor.
Liam: Now the batch conversion, the one that turns one menu click into fifty clean files. The Import submenu has a Batch Conversion entry, the Export submenu has a Batch Conversion entry, and the wizard is the same wizard, the file is quill-ui-batch-wizard, the core is quill-core-batch-convert. The wizard asks for a source folder, a recursive checkbox, a source format, a target format, an output layout, and an overwrite policy. The four moves are the four decisions. The worker is run_batch, and it iterates the target files, calls convert_file_with_pandoc on each, and returns a BatchReport with totals: total, converted, failed, skipped, plus a per-file entries tuple. The conversion runs on the QuillTaskManager background pool, so the editor stays responsive, the progress dialog updates per file, and Cancel is honored between files.
Jessica: The output layout, the design point worth knowing. The default is subfolder, the conversions land in a new folder called Output next to the source folder, the original stem preserved, the extension replaced with the target format's canonical extension. The second choice is same_folder, originals and conversions side by side, the user's choice. The output extensions live in quill-core-pandoc-formats, the picker prefers the longer or more common form, so .html wins over .htm, .markdown wins over .md for screen-reader-friendlier filenames. The overwrite policy is one of ask, never, always, and ask is enforced at the batch level with a single confirmation dialog that lists the first ten conflicting files. That is the design, and the design is screen-reader-friendly by not prompting per file.
Liam: Fidelity rules that keep trust, the last design point before homework. When a target format cannot hold something you have used, plain text cannot hold your highlights, PDF cannot hold your editable links, the conversion warns you before it commits. The warning is the plain-text-with-formatting dialog for text, the source-metadata flag for any format that drops structure, and the quality score for PDFs. No conversion is a silent downgrade. Every conversion is a new file or a new tab, originals are never converted in place, the overwrite policy is explicit, and the report at the end names what was converted, what failed, and what was skipped. That is the contract, and the contract is what makes batch conversion safe to use on a real folder of real work.
Liam: Now the honest corrections segment. A few claims drifted between earlier show notes and the code, and we want to be straight with you.
Liam: First, the conversion engine is not a do-everything wizard. It is the Tier-1 list in quill-core-pandoc-formats, eleven input formats and ten output formats, frozen in the code. The Tier-1 list is a QUILL product decision, not a reflection of what Pandoc itself supports. Pandoc supports dozens of formats, the full list is longer, and the Other Pandoc Format menu entry is the door to the longer list. If a past claim said QUILL can convert any format Pandoc knows, it drifted. The curated Tier-1 is the honest scope.
Jessica: Second, PDF extraction is the on-demand component, and the engine is not the same as the OCR engine. The PDF text extraction in quill-io-pdf uses pdfplumber and pypdf, the OCR engine for scanned PDFs is Tesseract, the OCR engine for the lighter image flows is the native Windows runtime. Three different engines, three different install stories, all free and local, none the same. If a past claim conflated them, it conflated them. The code is precise about the engine behind each door.
Liam: Third, EPUB and PDF are not symmetric in QUILL. EPUB has a real chapter-aware navigator because EPUB carries its own table of contents. PDF does not, by design, the format is a print artifact, and QUILL's PDF reader is a faithful text extraction with page boundaries preserved, not a chapter-aware reader. If a past claim said QUILL navigates PDFs by chapter, it drifted. Pages yes, chapters no.
Jessica: Fourth, the Word structured view is a real surface, an accessible ListView tree in quill-ui-word-view, and it is the default for .docx and .doc when the user picks structured in the open-mode prompt. The setting word_open_mode persists the choice. The surface is not experimental, the surface is un-gated, the code is the door. If a past claim said the Word view was preview or beta, it drifted.
Liam: Fifth, the LibreOffice dependency is real for one path. The PDF and Office text extraction download includes pdfplumber and pypdf, those are the primary PDF engines, and they are pure Python. The Apple Pages route B uses LibreOffice for higher fidelity, but route A, the IWA parser, does not. If a past claim said LibreOffice is required for PDF import, it drifted. LibreOffice is one of three PDF paths, not the only one.
Jessica: Homework, four steps. One: open the File menu in QUILL, look at the Import submenu and the Export submenu, and tell me, in writing or out loud, which entries are present and which are not. If PDF Document is missing from Export, install Pandoc from Help, Download Optional Components. If Other Pandoc Format is in both menus, you have the full Tier-1 plus the long tail. Either way, the menu is honest about its state.
Liam: Step two: pick one of your real working Markdown documents, the one with headings and lists and at least one bold word, and export it to Word. Open the result, if you have Word, and confirm the headings show up as real heading styles. If you do not have Word, save the export and trust the round trip, the conversion is the same path Pandoc ships in its own tests. The same document exported to HTML, then opened in a browser, should preserve the headings as h1, h2, h3 and the bold as strong.
Jessica: Step three: open an EPUB you actually want to read, the door is File, Open, or drag and drop. After the file opens, hit the EPUB Navigator entry under the Navigate menu, pick a chapter, and confirm a new tab opens with the chapter title as a Markdown H1. Then pick a heading inside the chapter, and the caret jumps to that heading. Reading an actual book becomes actual reading, and the round trip is one click.
Liam: Step four: batch-convert a small folder of anything. A folder of .docx reports, a folder of .html exports, a folder of .tex drafts, any of them. Open the Batch Conversion wizard, pick a source folder, pick a source format, pick a target format, leave the defaults for output layout and overwrite policy, and run. When the report appears, the Status Page entry will read converted, failed, skipped. The originals are still in the source folder, the new files are in the Output subfolder, and the whole thing is reversible because the originals were never touched.
Jessica: The architecture lesson, the one we want you to carry out of this episode. QUILL does not pretend to be every format. It speaks the formats the world sends, it writes the formats the world expects, and it does both through a single canonical source of truth that lives in plain text. The hidden codes from episode twenty-one ride along on every export that supports formatting. The reader's experience is the same on the way in and the way out, the buffer is always clean, the screen reader always knows what to do, and the user's working copy is always in the most accessible format alive. The formats are dialects. QUILL is the translator that does not lose the words.
Liam: Next episode, episode twenty-three, the heavy document rescue pipeline, the import tool that handles every PDF and Office file and image when the first tier comes back empty. The OCR episode, the confidence-reporting episode, the escalation-prompt episode, the one that makes scanned paper editable on your machine without an upload. The free-first, local-first, consent-first design law applied to the hardest case. We will walk a scan that looks like a complete failure and watch it become readable text, page by page, with the per-page confidence reported out loud.
Jessica: The QUILL Cast is a fifty-four episode course, and we are twenty-two of the way through, past the first quarter, into the deep middle where the systems are real and the engineering is something you can rely on. The audio arc returns after the rescue episode, the AI hub follows, then the vault, then GLOW, then braille, then Quillins, then the finale. The road is long, the road is good, and we will walk it together.
Liam: I'm Liam.
Jessica: I'm Jessica. Speak the dialects, keep the home format.
Liam: The QUILL Cast is a fifty-four episode course on QUILL. Subscribe wherever you listen, and we will see you in episode twenty-three.