Sound as numbers
Before any model, a voice is a list of numbers. Take a close look.
Sound is vibrating air. When you speak, air pressure rises and falls hundreds of times per second. The microphone turns that pressure into voltage, and the computer's converter measures the voltage at fixed intervals. Each measurement becomes a number between −1 and 1. Digital audio is just that: a list of numbers.
How many measurements per second
The sample rate says how many times per second the sound is measured: at 48 kHz, 48 thousand numbers per second. The highest frequency that fits is half the rate (the Nyquist rule), so 48 kHz holds up to 24 kHz, above what the ear can hear. The bit depth says how many steps each number can take: 16 bits gives 65,536. Zoom in until each measurement is a dot.
Read next
A new voice without touching every weight
Low-rank adapters: train under 1% of the numbers and keep the same speed.ReadAttention and the next token
A language model writes one token at a time, looking only backwards.ReadMore voices per GPU
CUDA graph, MPS and batching: how we doubled real-time voices on the same GPU.Read