25: Files Everywhere - transcript
Liam: The QUILL Cast, episode twenty-five. I'm Liam. Today: files that don't live on your computer. FTP servers, SFTP, HTTPS with WebDAV, and S3-compatible object storage, plus the special case of GitHub, the edit-over-SSH path for live config work, and Open from URL for one-off links. Five doors in the File menu, all aimed at ending the download-edit-reupload shuffle.
Jessica: I'm Jessica. Quick recap of episode twenty-four, because we left off in a specific place. Last episode was OCR and AI image describe. We walked the five doors under Tools, Reading and Dictation, OCR Image, OCR Clipboard, OCR Screen Capture, the shared review surface with confidence flags, and Describe Image for the AI side with its vision prompt picker. The honest note we made last week still holds: the lighter-weight image flow uses the built-in Windows.Media.Ocr engine and works without any install, while the document rescue flow inside Import slash Convert Document uses Tesseract for whole PDFs. Both are local and free, but the install story is different for each door. Episode twenty-five is the other side of the locked-content coin: the words that exist on a server instead of on paper. Same philosophy, applied to a different kind of distance.
Liam: The frame first, because it explains the design. Most of the world's text, the words that matter, lives somewhere other than your desktop. On a client's SFTP box. In a company's S3 bucket. Behind a WebDAV share. In a GitHub repository. On a web server, the one you ssh into to fix a config file. Until now, the only honest answer was a tower of single-purpose tools: an FTP client for FTP, a terminal for SSH, a browser tab and a copy-paste for S3, a Git client for the repo. Every hop a different app with its own accessibility quirks, its own credential model, its own way of getting in your way. QUILL collapses the pilgrimage into one File menu, and the host of credibility for that collapse is that the security and consent posture is identical across every door.
Jessica: Five doors, same File menu. File, Open from URL, the one for a link someone sent you. File, Open over SSH, a submenu with Quick Connect and Site Manager, the dedicated path for editing live config on a remote host. File, Open from Remote, a submenu with Open from Remote, Save to Remote, Save Copy to Remote, GitHub Repository, GitHub File URL, Save to GitHub, and at the bottom, Manage Remote Sites and Manage GitHub Accounts. That's thirteen menu entries across three submenus, and they share two underlying protocols: the cross-protocol remote-sites engine for FTP, SFTP, WebDAV, and S3, and the dedicated GitHub path that uses PyGithub directly.
Liam: Let's start with the cross-protocol door, because it sets the pattern the rest will follow. File, Open from Remote, Open from Remote. A two-panel dialog appears. Left, your saved sites. Right, the directory browser for whichever site is selected. The dialog lands initial focus on the site list, the first actionable control, honors the QUILL modal contract (Escape cancels, Enter accepts), and every control has an accessible name. The list is empty the first time; the New site button is the front door.
Jessica: The site editor is the spot to take a breath, because the same dialog and the same password field are reused for every protocol, and the design here is worth understanding. Fields: friendly name, protocol, host, port, username, password, root directory, and one protocol-specific field labeled Bucket (S3 / base path WebDAV). One checkbox, Trust host key on first use, and it only matters for SFTP. The password is never written to the saved site file. The site file, remote_sites.json, holds name, host, port, username, root directory, the trust flag, and the protocol-specific extras. The password lives separately, encrypted with DPAPI on Windows or in the macOS Keychain on macOS, addressed by the site's id. That separation is deliberate: a plaintext password in a settings JSON is the kind of leak nobody notices until it ships in a backup.
Liam: Four protocols, each honest about its own risks. FTP is plain. The docstring in the transport spells it out: QUILL does not ship FTPS, because we have no out-of-the-box certificate store for it, and silently sending passwords in cleartext is a footgun. If you need encrypted file transfer, use SFTP, which is the SSH-based file protocol. SFTP, the protocol within File, Open from Remote, uses paramiko, defaults to a host-key reject policy, and is password-only; the docstring is explicit that users with key material should go through File, Open over SSH instead, which has a richer key-management story. WebDAV goes over HTTPS, period. S3, the S3-compatible object transport, is HTTPS too, and supports anonymous reads of public buckets via the AWS SigV4 signing path or via boto3 if you have it installed.
Jessica: A small but important distinction between the two SSH doors, because the menus look similar. File, Open over SSH, the one tied to issue #139, is for editing live config files on a server. It uses a separate site manager, ssh_sites.json, and it supports password, SSH agent, and key files, OpenSSH, .ppk, and SecureCRT and ssh.com key formats, with the load_private_key helper and putty_key module doing the format work. The host-key policy is paramiko.RejectPolicy by default, and you can opt into AutoAddPolicy with the trust_first_use per-site flag. File, Open from Remote, when you pick SFTP as the protocol, uses the cross-protocol sftp_transport, which is password-only and refuses the moment paramiko isn't installed. Two engines, two sites stores, same library, different feature surface. The choice is on you: dedicated editor for the server admin, convenience file retrieval for everyone else.
Liam: Now an interactive beat, because the next segment is the one you actually want to drive, not just hear about. Pause the audio, do this now, then come back. Step one: in QUILL, open File, Open from Remote, Manage Remote Sites. The two-panel dialog appears. Step two: click New site. Give it a friendly name, test site, host ftp.example.com, leave the rest empty, click Save. Step three: click the site you just made. The right panel populates with its root directory. Step four: press Escape to close. That's it. You now have a saved site. Press play when you're ready to use it.
Liam: Welcome back. Open the site you just made, and the directory browser wakes up. Each entry has a name, a size in bytes, a modified time, and a flag for directory. Directories are listed first, then files, both alphabetical, and the dialog is keyboard-first: arrow keys to move, Enter to descend into a directory, Backspace to go up. When you land on a file, its absolute path fills into the Remote file field at the bottom of the right panel, which is also a free-text input if you'd rather type the path directly. Click Open, and the work begins.
Jessica: And a code-verified honesty moment, because we want to be straight with you about what the Open from Remote flow actually does versus what you might assume. The status line announces "Downloading <path> from <site name>" before the bytes move, and the screen reader hears that announcement. The download then happens in the foreground on the modal dialog's worker, returns, and the file lands in a temp path, which becomes a new tab with source label "from <site name>:<remote path>". But the download is not gated by a pre-transfer confirmation dialog; the dialog itself is the gate, because Open is the consent. There is no separate yes-or-no prompt for size. If you want to be told the size before committing, you can read it in the directory listing; the bytes are right there in the column. The earlier short version of this script overclaimed that "every operation is explicit and announced, host and expected size before any transfer." That's not quite right. The host and the path are in the dialog and in the announcement; the size is in the listing you can already see. The trust architecture is real, but the gate shape is: choose the site, choose the file, click Open.
Liam: Another honesty moment, and this one corrects a real drift. The previous short version said "Save to Remote writes it back, with a tilde backup left beside the original on the server." That is true for the SSH edit-over-SSH path, where the SftpFileService has a write_file with a make_backup flag that renames the original to a tilde-suffixed copy before writing. It is not true for the cross-protocol door, File, Save to Remote. The FTP, SFTP-in-Open-from-Remote, WebDAV, and S3 transports all upload via a plain overwrite; there is no tilde backup. If you need a backup on the server, you must make it yourself, or use the SSH door. The asymmetry is by design, because the SSH door is the dedicated config-editing path and gets the editor-grade guarantee; the cross-protocol door is convenience file retrieval, and the assumption is that the server has its own backup story. We tell you this now, rather than have you discover it after losing a file. The two menus are not interchangeable.
Jessica: The save flow itself is straightforward. File, Save to Remote, or Save Copy to Remote. Both go to the same Remote Sites dialog in Save mode, so the layout is identical: pick a site, browse or type a target path, click Save. The difference is what the editor does locally. Save to Remote writes the file to the server with no local copy left behind; Save Copy to Remote keeps your local file and writes a copy to the server. The active tab, by the way, knows if it was opened from a URL: read_only_remote is set, and in that state, Save to Remote is grayed out and you get an explanatory message. Use Save Copy to Remote in that case, or save a local copy and then upload. Same dialog, different button label, identical wire behavior.
Liam: The trust architecture, code-verified, applies to every door. Let me walk the four pieces. First, credentials. Saved passwords are encrypted at rest; the sidecar store uses DPAPI on Windows, the macOS Keychain on macOS, and the site file itself never holds the password. Second, host-key policy. SFTP, both the cross-protocol door and the SSH door, defaults to paramiko.RejectPolicy, the strongest option, and trust_first_use is opt-in per site. Third, transport security. WebDAV goes over HTTPS via the verified SSL context helper; S3 is HTTPS only; Open from URL refuses non-http schemes and uses the same verified SSL context. FTP, the plain option, is the explicit exception, and the docs in the transport are loud about it. Fourth, response parsing. The WebDAV PROPFIND response and the S3 LIST response both go through a safe XML parser that refuses external entities and unsafe constructs, so a malicious server cannot turn the listing step into a parser attack. Yes, somebody thought of that. That's the house style.
Jessica: Now the GitHub door, because it is the crowd favorite and it earns the hype. File, Open from Remote, GitHub Repository. The GitHub sign-in is its own flow: the manage-accounts dialog lets you add a personal access token, which is stored in the Windows credential vault, never in a plain config. The consent dialog appears the first time, one time, and explains what QUILL will do with the token: read repositories, read and write files. After consent, you can browse your own repositories, browse any public repository anonymously, no token required for reading, and open files into a normal tab. The browse dialog is the same dialog shape as the rest of the remote flow: list on the left, directory on the right, accessible name on every control.
Liam: There is a second GitHub entry worth knowing: GitHub File URL. Paste a github.com URL of the form https://github.com/owner/repo/blob/branch/path/to/file, and the parser splits it into owner, repo, ref, and path. The parser strips query strings like ?plain=1, line anchors like #L10, and percent-decodes the path, all common when copying from the GitHub UI. If the URL doesn't match, you get a clear error: "Could not parse the URL. Expected format: https://github.com/owner/repo/blob/branch/path/to/file." If it matches, QUILL downloads the file and opens it. No browsing required, just paste the link. Both entry points land you in a tab with source_label "GitHub: owner/repo (ref)" and the file content in UTF-8 text, decoded with a Latin-1 fallback for any non-UTF-8 file rejected with a clear "not a UTF-8 text file" message.
Jessica: The save-back is the moment GitHub stops being a read-only convenience. File, Save to GitHub. The command looks up the file's origin in the per-tab origin map. If the active tab was opened from GitHub, the origin is there, and QUILL prompts for a commit message in a single-line dialog, prefilled with "Update <name>". You confirm, and the save-back worker thread sends the file content to the GitHub contents API, using the SHA of the file you opened as the expected blob, which is how the API detects a conflict: if someone else pushed to that file while you were editing, the write is rejected with an error, and the error message names the file. That's the right behavior. You don't want your edit to silently overwrite a teammate's change. The worker thread also releases the underlying GitHub session in a closing block, so a mid-save quit can't leak the connection.
Liam: A subtle accessibility note about GitHub that came up in the implementation. The GitHub dialogs and the underlying threading are in separate places: main_frame_github.py for the menu wiring, github_dialogs.py for the dialogs, and core/github/github_provider.py for the API. PyGithub is required, and the consent + dependency check happens before any PyGithub call. If PyGithub is not installed, the menu entry still appears, but the first attempt to open a repository shows an information dialog with the install hint. The same pattern protects the rest of the GitHub surface. This is also why the GitHub flow is its own door and not folded into the cross-protocol sites: PyGithub handles a lot of edge cases that a hand-rolled REST client would have to re-implement, and the cost of one extra optional dependency is worth it for the correctness gain.
Jessica: The SSH edit-over-SSH door, File, Open over SSH, deserves its own minute. Quick Connect, if you have a one-off host, prompts for host, port, username, and authentication method, and then connects. Site Manager is the saved version: a list of sites with name, host, port, username, auth, key path, and default directory. Three auth methods: password, key, agent. Password is prompted at connect time and never persisted. Key reads a private key file, optionally with a passphrase, and supports the OpenSSH format, .ppk via the putty_key module, and the SecureCRT and ssh.com key formats via the keys module. Agent talks to the running SSH agent if you have one. The site manager dialog is the dedicated control plane for the SSH path, and it is deliberately separate from the cross-protocol remote sites manager; the two stores don't share a file.
Liam: The connect worker runs on a daemon thread, the connection itself is held in a list on the main frame, and a tilde-backup aware SftpFileService is the read/write API. Browse, find a file, click Open, the file downloads to a local temp path that becomes a tab with the source label naming the remote file path. Edit it locally with every QUILL power. Save it. The mixin's maybe_upload_remote_on_save method intercepts the save, looks up the binding for the local path, and pushes the bytes back over the same connection, with a tilde backup left beside the original on the server. The newline style is preserved across the round trip: the file's original LF, CRLF, or CR is detected on read and re-applied on upload, with surrogateescape for non-UTF-8 content. So a Linux config file stays a Linux config file, byte for byte, except for the bytes you actually changed.
Jessica: The host-key policy is the part of the SSH door that matters most for trust. By default, the SSH client loads the system host keys and rejects anything new. An unknown host raises an SSHException and the connection fails, with a message that names the host. If you know the host is yours, you can opt into trust_first_use, which is the AutoAddPolicy behavior, and that flag is per-site, so the decision is granular. The default-reject posture is the one QUILL ships with, because the legacy "accept and cache" behavior is exactly the kind of default that turns into a quiet MITM. If you want a Trust This Host dialog instead, that's a known follow-up; the docstring calls it out. The current default is the right default, and the opt-in is right next to it.
Liam: The smaller doors next, because they round out the family. Open from URL, File, Open from URL. A single-line dialog with a default of "https://", an explicit list of allowed schemes (http and https only), and a transport that streams the response to a local temp file. The transport has a redirect cap of five to prevent a server from looping the editor, a hard byte cap of two hundred fifty-six mebibytes, and a filename inference from Content-Disposition, then the URL path, then "untitled". The downloaded file is then handed to the format detection pipeline, which means the URL door can open a PDF, a Word document, an HTML page, or a plain text file, whichever the server sends. The tab is tagged with source_label "name (from URL)", and Save is replaced with Save Copy to Local File, because the URL is the source, not a place you can write back to.
Jessica: Open from URL is not gated by a pre-download consent dialog in the way the Open from Remote flow is gated by the two-panel picker. The gate shape is: you typed the URL, you clicked Open. That is the consent. The redirect cap and the byte cap are the safety belts. If the server returns a 401 or 403, the dialog shows the HTTP status; if the URL is malformed, you get a clear message; if the download times out, you get a clear message. The error taxonomy is the same as the other transports: RemoteAuthError for 401/403, RemoteNotFoundError for 404, RemoteTransportError for everything else. The screen reader hears the error, the dialog is the gate, the URL is the consent.
Liam: And one door we have not yet covered, because it lives in a later episode. The Accessible Vault, episodes forty and forty-one, has its own sync story: Vault Sync, the file in core/vault/sync.py, an opt-in commit-plus-pull-plus-push flow over your own Git remote, with conflict detection surfaced as a spoken, itemized list. "These N notes changed both places, keep mine, keep theirs, merge." The vault is plain files, so any file sync already works; this adds the layer that handles the hard case, conflicts, with the spoken treatment. We'll come back to it. For now, know that the GitHub door you met today is the same machinery the vault sync runs on, and the consent model is the same: explicit, opt-in, and announced.
Jessica: The composite day, because the pieces stack. Morning, the client emailed you a draft. They put it on their SFTP server, in the same directory as last month's. Open from Remote, pick the client site, browse, pick the file, edit, save, and the file is back on their server before your coffee cools. No FTP client, no terminal, no second window. Afternoon, you maintain the company website. The text is in a GitHub repository. Open from Remote, GitHub Repository, pick the org's repo, navigate to the typo in the readme, fix it, Save to GitHub, the commit message is "Update README.md", and the change is live the moment the page rebuilds. No Git client, no command line, no ceremony. Evening, the web host's Nginx config needs a one-line change. Open over SSH, Site Manager, pick the host, browse to the config, edit, save, and the tilde backup is sitting next to the original in case you need to roll back. Three infrastructures, one editor, zero context switches. The tax every tool switch used to charge, QUILL stopped collecting.
Liam: And the long tail, the people this was actually built for. Writers collaborating with a freelance editor who keeps files on a personal SFTP server. Researchers pulling datasets from a lab's WebDAV share. Small newsrooms that store source documents in S3 because the buckets are free and the audit trail is built in. Documentation teams that live in a GitHub repository and need a path that doesn't require a developer to install Git. Anyone with a web host and a config file. Anyone who has ever said, I just need to fix this one file on that one server.
Jessica: Accessibility notes, because this audience matters. Every dialog in this family passes the QUILL modal contract: initial focus on the first actionable control, Escape to cancel, Enter to accept, every control with an accessible name. The site list and the directory list are both single-selection ListBoxes; the path input is a process-enter TextCtrl that submits on Enter. Status announcements use the same path as the rest of QUILL, with announce calls and status-line updates in the worker thread, marshaled to the UI thread via wx.CallAfter. The GitHub consent dialog is its own surface, with an explicit "I understand" gate before the token is saved. The SSH password prompt uses PasswordEntryDialog, so screen readers do not echo the value. The Open from URL URL field is a single-line TextCtrl with the scheme pre-validated, and a malformed URL is caught before any network call.
Liam: A few craft notes that pay off. One: when you save a site, give it a friendly name you'll recognize by ear; "Acme" is better than "site-1" when you're scanning the list with a screen reader. Two: the trust_first_use flag is a per-site decision, so you can leave your own hosts on the strict default and turn it on only for the dev boxes you spin up and tear down weekly. Three: the temp paths QUILL uses for downloaded remote files are cleaned up on close via the GitHub mixin's prune method and the SSH close method, but the cross-protocol temp files are not auto-cleaned; if you need that, close the tab and reopen. Four: Save Copy to Remote is your friend when you want to leave the original alone and upload a new version. Five: when an S3 bucket policy denies a public read, the SigV4 fallback needs an access key and secret key in the site's extras; the dialog labels the field "Bucket" but the key-value pairs are stored under s3_bucket, s3_access_key, s3_secret_key, s3_region, s3_endpoint, and you'll need to edit the JSON directly or add them in a follow-up dialog. The minimum for an authenticated S3 site today is the bucket, the access key, and the secret key; the others are optional with sensible defaults.
Liam: Homework, four steps, do them in order. Step one: save one remote site, even a test account, and round-trip a file. Use Open from Remote to fetch it, edit it, Save to Remote to put it back, and watch the status line confirm both directions. Step two: browse any public GitHub repository from QUILL and open its readme, no account needed for reading. Notice the source label on the tab. Step three: try Open from URL on any document link and listen to the status line as the download runs. Note the redirect cap behavior with a link that 301-redirects. Step four: pick one file on a server you actually own and SSH into it. Open over SSH, Site Manager, save the site, connect, browse, edit, save, and verify the tilde backup sits beside the original. That is the whole family in four steps.
Jessica: Next episode, episode twenty-six, we hand the editor a watch folder. QUILL starts working while you're not there: drop a file in, it opens; drop an audio file in, it transcribes; drop a PDF in, it runs the rescue pipeline. Automation with the same consent posture, the same screen-reader announcements, and the same Safe Mode guarantee that disables it entirely when you want a quiet room.
Liam: For a fifty-four-episode series, this is the end of the files arc. We've done Word, EPUB, PDF, the import-and-convert family, the rescue and OCR family, and now the cross-internet family. The next one is the foundation underneath all of it: background work, with the same explicit and announced posture that has defined every door so far.
Jessica: I'm Jessica.
Liam: I'm Liam. The world is your filesystem, and you can edit it from here.
Jessica: The QUILL Cast is a production of the QUILL project. Episode twenty-five of fifty-four. Thanks for listening.