On-device versus cloud speech recognition
What differs between recognition that runs on your Mac and recognition that runs on a server: latency, cost, offline behaviour and who holds the audio.
The choice between on-device and cloud speech recognition gets discussed as a values question — privacy on one side, convenience on the other. It isn’t, or at least not only that. It is an engineering decision about where a particular matrix multiplication happens, and four practical consequences fall out of it: how long you wait for text, how the product can be priced, what happens when the network is bad, and how many parties end up holding a recording of your voice.
All four are measurable, and none of them require you to have an opinion about surveillance. Below is what actually differs, including the places where sending audio to a server is the better engineering choice, because there are several and pretending otherwise makes the comparison useless.
The same computation, in two places
A speech recognition model is a neural network, usually an encoder-decoder. Audio goes in as a spectrogram, the encoder produces a representation, the decoder emits text tokens one at a time. That computation is identical whether it runs in a data centre or on the laptop in front of you.
In the cloud arrangement your microphone audio is encoded and sent over the network, either uploaded after you stop speaking or streamed in chunks while you talk; a server runs the model on a GPU shared with many other users and sends text back. In the local arrangement the model file lives on your disk, gets loaded into memory, and runs on your CPU, GPU or neural accelerator, and the audio never becomes a network packet. Everything else here follows mechanically from that one difference.
Latency and where it comes from
Cloud recognition pays transport costs that local recognition does not: connection setup, upload time proportional to how long you spoke, queueing behind other requests, then the return trip. On a good connection that can be small. On hotel Wi-Fi, on cellular in a lift, or behind a corporate VPN routing your traffic through another continent, it is not, and the variance matters more than the average — a tool that usually answers in 400ms and occasionally takes six seconds feels broken in a way a consistent one-second tool does not.
Local recognition pays no transport at all but is bounded by your machine, competing with whatever else you are running. So the honest framing is not that local is faster; it is that local latency depends on things you control, while cloud latency depends on things you can’t. Both sides can hide some of the wait by transcribing as you speak, so streaming is not a differentiator. Model size is the real local lever, and a genuine trade rather than a free win: picking a speech model size is a negotiation between accuracy, memory and patience.
Cost structure, which decides the business model
Server inference has a marginal cost. Every minute of audio consumes GPU time that someone pays for, and it scales linearly with usage. No cloud vendor serves a heavy user for free, so pricing has to recover it — either metered per minute, or a subscription priced high enough to absorb the people who dictate all day.
Local inference has a marginal cost to the vendor of approximately zero. The model is a file, and shipping it to a thousand more users costs bandwidth once. That is not a moral advantage, it is an accounting one, and it is the whole reason a one-time purchase is structurally possible for on-device tools and structurally difficult for cloud ones.
There is a cost you do pay locally, and it is disk. A useful multilingual speech model runs to hundreds of megabytes — Voicecape ships a quantised Whisper large-v3-turbo of about 547 MB inside the app bundle, which is why nothing downloads on first launch and also why the download is not small. A cloud client can be a few megabytes. On a machine where storage is genuinely tight, that is a real point against local.
Behaviour when the network is unavailable
This one is binary rather than a trade-off. Cloud recognition on a plane, in a tunnel, on conference Wi-Fi that has captive-portalled you, or during the twenty minutes your ISP is down does not degrade — it stops. Some clients queue the audio and transcribe later, which beats nothing but is not dictation, since the point is that text appears at your cursor while you are still holding the sentence in your head.
Local recognition in the same conditions is unchanged, because nothing in the path touched the network. Its failure modes are different in kind: a loaded machine transcribes more slowly, a memory-starved one may swap. Those are degradations, not outages. How much this matters is entirely a function of where you work — close to irrelevant at a desk on fibre, and the whole decision for anyone who writes on trains and planes.
Who ends up holding the audio
Locally, the audio exists in memory on your machine and then it doesn’t. There is exactly one party to the transaction, and nothing about that requires trust, because there is no second party to trust.
In the cloud arrangement the recording of your voice, along with whatever you said into it, is transmitted to and processed by an organisation you have a contractual relationship with, possibly using subprocessors you inherit, in a jurisdiction that may not be yours. Reputable vendors document retention windows, state whether audio is used for training, and list their subprocessors. That documentation is a genuine answer, not a dodge.
But it is an answer of a different type. "We delete audio after 30 days and do not train on it" is a promise about behaviour, verified by audit if at all. "The audio never left the machine" is a claim about architecture, and you can check it yourself in a few minutes without anyone’s cooperation — the procedure for testing any dictation app works on on-device products too, including ones that would prefer you took their word for it.
What the cloud genuinely does better
Any comparison that does not concede these is selling something.
Server-side models are not bounded by consumer RAM. A vendor running on data-centre GPUs can deploy a model several times larger than anything that fits comfortably on a laptop, and for speech recognition larger models really are more accurate, particularly on accented speech, noisy rooms, overlapping speakers and low-resource languages. If a small local model transcribes you badly, a big cloud model may simply do better, and no amount of architectural purity fixes that.
Server-side models also update without a reinstall. When a better model ships, a cloud vendor deploys it and every user has it that afternoon; a local model is a file inside an application, so improving it means shipping a new version and asking people to download several hundred megabytes again. The gap between the best available model and the model you have opens up over time in a way it does not for cloud users. Server hardware can also afford computation a laptop cannot spare — wider beam search, diarisation, reconciling several models — so compute-hungry features arrive on cloud products first, and sometimes only.
The trade, stated plainly
Cloud recognition offers you someone else’s larger model, kept current without your involvement, at the cost of a recurring bill, a dependency on the network at the exact moment you want to write, and a copy of your voice in someone else’s system.
Local recognition offers you a model you already have, which behaves identically at 35,000 feet, costs nothing per minute to run and produces no second copy of anything — at the cost of disk space, accuracy bounded by your hardware, and a model that only improves when you update the app.
Neither is correct in general. The deciding factors are unglamorous: how good your network usually is, how much you dictate, whether the language you speak is well served by a model that fits on a laptop, and whether anything you say into a microphone is confidential. If it helps to see the local configuration as a concrete thing rather than an abstraction, one fully on-device implementation shows what these constraints produce in practice.
Not a real differentiator: encryption in transit. Every credible cloud vendor uses TLS, so "your audio is encrypted" says nothing about who reads it at the other end. The question is not whether the transmission is protected but whether a transmission happens.