Blog

Typing with your voice in any app

How text gets from a dictation app into the window you are looking at on macOS: accessibility APIs, the clipboard path, and why browsers differ.

The visible half of dictation is turning speech into text. The invisible half is getting those characters into the window you were looking at when you pressed the shortcut, and on macOS that second half is where most of the engineering goes.

A dictation app has no privileged relationship with your editor, your browser or your terminal. It is another process, and the only ways it can put text somewhere else are the ones macOS gives any app: write into the focused element through the accessibility API, post synthetic key events, or put the text on the clipboard and trigger a paste. All three work somewhere, none works everywhere, and the failures are not evenly distributed.

Three routes into a text field

These are less alternatives than rungs on a ladder, each tried only when the one above it has provably failed.

  • The accessibility write: ask the focused element to replace its selected text. Nothing else on the machine moves and the clipboard is untouched.
  • Synthetic keystrokes: post key events character by character, as though a very fast person were typing.
  • The clipboard: put the text on the pasteboard, send one Command-V, and let the target handle it like any other paste.

Why the app never types it out character by character

The middle rung is not used at all. Posting a paragraph one key event at a time takes seconds, reorders under load, and mangles anything with an input method attached, because the input method is midway through composing when the injected events arrive. For anyone dictating Korean, Japanese or Chinese that is the normal path rather than an edge case. The only synthetic keystroke in the whole design is a single Command-V.

Proving the direct write landed

The first rung asks for the focused element, checks that its selected-text attribute is settable, and writes. What matters is what happens next, because the obvious implementation, trusting whatever the write call returned, is wrong in the worst possible way.

So the code fingerprints the field before the write and compares afterwards. The fingerprint is deliberately not the document text but the character count plus the location and length of the selection: reading the whole value on both sides would copy the entire document on every dictation, and one terminal scrollback measured 62,115 characters. Two integers answer the only question being asked, which is whether anything moved. The comparison runs up to eight times at 25 ms intervals.

Unmeasurable is not the same as successful. If neither attribute can be read there is no fingerprint, so the write counts as unproven and the text goes to the next rung.

The proxy problem in browsers and Electron apps

In a browser, the accessibility tree another process can see is not the document. The document lives in a renderer process and the tree is a proxy that forwards requests across a process boundary. The architecture is public and reasonable, and it has a consequence that is easy to miss: a write can be accepted, delegated and acknowledged before anything has been applied. The reply reports delivery, not effect.

Measured, Chromium- and WebKit-based windows advertise the selected-text attribute as settable, accept the write, return success, and leave the document unchanged after 900 ms of polling, confirmed through the page title so a stale accessibility cache could not be the explanation. A native AppKit field is already done on the first read, at 0 ms, and nothing was observed in between, which is the only reason a short deadline is safe here.

The cost of believing that acknowledgement is not a failed insert, which would at least be visible. It is the ladder stopping on its first rung, so the fallbacks that exist so words are never lost never run: the text goes nowhere and the status says it was inserted.

Why a list of app names cannot decide this

The tempting shortcut is a table, with browsers and Electron apps sent to the clipboard and native apps given the direct write. It does not hold, because the behaviour is not a property of the app.

An Electron app whose accessibility tree has never been switched on refuses the write honestly at the settable check, and the direct rung works perfectly. The same binary with the tree switched on accepts the write and drops it. The deciding variable is a runtime state, so any name list is wrong for some of its entries at any moment, and wrong in a way that changes without a release.

The check therefore looks at the element instead. Chromium and WebKit both hang an accessibility DOM class list or a text-marker attribute on web nodes, and no AppKit control exposes either, measured against a plain text editor and a terminal.

Those attribute names are not public API, so a rename would make the check stop matching with no error at all. It survives that only because it is an optimisation rather than the safety mechanism: the fingerprint catches web content whether the attribute check does or not.

The clipboard rung, step by step

When the direct rung is skipped or fails, the text goes through the pasteboard, and the sequence is more careful than the summary suggests.

  1. Snapshot the pasteboard, capturing every representation of every item rather than just the string, because people keep images and rich text there and a plain-text approximation is a silent loss.
  2. Write the transcript as plain text, marked current-host-only so it is not eligible for Universal Clipboard. With Handoff on by default an ordinary pasteboard write pushes what you just dictated to every device on the same account, and since this is the rung browsers always take, for a browser user that would be every dictation. What does leave the machine is itemised in what goes over the network.
  3. Record the pasteboard change count, which now identifies this generation of the clipboard.
  4. Re-read the focused element and fingerprint it, rather than reusing the earlier one, since a failed write may have moved focus.
  5. Post Command-V to the HID event tap so it lands in the frontmost app rather than back in ours, with the app’s own hotkey listener suppressed.
  6. Watch the fingerprint for up to about 600 ms, because a paste is asynchronous and the target reads the pasteboard on its own schedule.

What happens to your previous clipboard

If the paste is observed to land, the snapshot goes back, after a 400 ms delay and only if the change count is still the one that was recorded. That condition earns its place: 400 ms is long enough for you to have pressed Command-C in another window, and if you did, restoring the snapshot would destroy what you just copied, with no error and no undo.

If the paste cannot be observed to land, or the key event never fired, nothing is restored and the result is reported as copied rather than pasted. Your previous clipboard is gone, and that is the deliberate trade: the dictated text is in the one place you can still reach it, and putting the old contents back would take it away while calling the operation a success.

Where there is nothing to measure against, with no focused element or no readable fingerprint, the restore goes ahead anyway, since the text is on the clipboard either way.

The always-copy-to-the-clipboard setting skips the ladder outright: nothing is written into any document, no paste is sent, and the transcript is left on the clipboard with the status saying so.

One risk is left visible rather than papered over. There is no later check asking whether a failed direct write landed after all, so an app that applied it past the deadline would receive the text twice. Duplicated text can be seen and undone, and silent loss cannot.

What this looks like from the outside

In use, the whole ladder collapses into two words in the status line. Inserted means the text went into the field, and copied means the field refused it or could not be verified and the transcript is on your clipboard, which is worth knowing before you assume a dictation vanished.

None of it works without the permission that lets one app write into another app’s window, and macOS keeps that separate from the permission that makes the shortcut fire; both are in the permissions article. For which editors and terminals take the direct path, see dictation for developers.

Related

  • macOS permissions for dictationMicrophone, Accessibility and Input Monitoring: what each is for, why macOS asks separately, and what to do when one looks granted but nothing happens.
  • Dictation for developersCode is a bad thing to dictate. The prose around code — commit messages, PR descriptions, review comments, docs — is a very good one.

All articles