RustSTT

Speech recognition services are usually a Python process holding a model, wrapped in an HTTP API, with a queue in front of it. That works until the model needs to change, at which point the deployment, the memory profile and the response shape all change with it.

The requirement here was different: several decoder architectures, one integration, and a running service with no Python interpreter in it.

The four are not variants of one design. GigaAM is a Conformer encoder with a CTC head; Parakeet TDT is a FastConformer with a token-and-duration transducer; Qwen3-ASR feeds an audio encoder into a language-model decoder; Whisper large-v3 turbo is an encoder–decoder transformer. The usual answer to that spread is four services. This is one.

A single decoder trait keeps one frame shape across all four, so integrating against it is a WebSocket and a JSON contract rather than a deployment decision. Changing which architecture serves a stream is then a server-side concern: the client does not learn about it, because the frames it receives do not change.

The service is Rust on Candle. There is no Python interpreter in the running process, which removes an entire class of packaging, GIL and dependency problems from production, and weights are memory-mapped rather than loaded, so a process starts in the time it takes to open a file.

It runs on CUDA in production, and on CPU and Apple Metal as well, from the same binary and the same decoder abstraction.

Audio arrives as 16 kHz mono PCM in binary WebSocket frames. Partial transcripts are emitted while the stream is still open, because a transcription that only exists once the speaker has finished is a different product. Real-time factor is reported alongside the output rather than being an internal metric, so a caller can see whether the service is keeping up with the audio it is being sent.

Audio is transcribed in memory and never written to disk. Access is controlled by domain or IP, and the service is priced per audio hour.