# More voices per GPU

> CUDA graph, MPS and batching: how we doubled real-time voices on the same GPU.

Infrastructure · November 4, 2026 · Pedro Cormann, Founder & Machine Learning Engineer · https://lero.studio/research/en/gpu

Generating voice in real time for many conversations at once is an engineering problem, not only a model problem. Speech comes out in short steps, and each step launches dozens of kernels on the GPU. With one conversation per process, the GPU spends much of its time waiting for the CPU to send the next kernel.

A CUDA graph records the whole step once and replays it with a single call. MPS lets several processes truly share the GPU. The biggest gain came from batching: every conversation moves in the same step, each in its own KV cache row. The audio is identical to one conversation at a time, with twice the real-time voices on the same GPU.

_(Interactive on the page: https://lero.studio/research/en/gpu)_

_(Interactive on the page: https://lero.studio/research/en/gpu)_