Latency Budget

Counting Every Millisecond

In real-time video production, latency—the delay between an event happening in the real world and it being seen on a screen—is a critical factor that can make or break a workflow. A Latency Budget is a systematic accounting of all the delays introduced by each piece of hardware and software in a video signal path. It’s a fundamental concept in live broadcasting, virtual production, and remote collaboration, where production teams must engineer their entire system to stay within a strict, predetermined threshold for total delay.

Sources of Latency in a Production Chain

Latency is cumulative; every device in the chain adds a few milliseconds of delay. Understanding and measuring these sources is the first step in creating a budget. A typical signal path might include:

Component	Source of Delay	Typical Latency (ms)
Camera	The time the sensor and image processor take to capture and output a frame.	1-30 ms
Encoder	Compressing the raw video signal for transmission over a network.	10-100+ ms
Network	The ‘speed of light’ delay plus any congestion or routing delays.	5-200+ ms
Decoder	Decompressing the video signal at the receiving end.	10-100+ ms
Display	The time the monitor or projector takes to process the signal and illuminate pixels.	2-25 ms
Other Processing	Switchers, converters, and graphics systems all add their own small delays.	1-10 ms per device

’Spending’ the Budget

The ‘budget’ concept arises from the needs of the specific application. The total acceptable latency dictates how much delay can be ‘spent’ at each stage. Different applications have vastly different budgets:

• Virtual Production (LED Wall): This has the tightest budget, often requiring a ‘glass-to-glass’ (camera lens to LED panel) latency of under 25 milliseconds. If the delay is any higher, the movement of the virtual background will not match the movement of the physical camera, destroying the illusion of reality.
• Live Broadcast Interview: For a seamless back-and-forth conversation between two remote locations, the round-trip latency (there and back) should ideally be under 200 milliseconds to avoid awkward pauses.
• Remote Dailies Review: In this case, interactivity is less critical. A latency of up to 1-2 seconds (1000-2000 ms) might be perfectly acceptable, allowing for more robust error correction and higher-quality compression during transmission.

Engineering for Low Latency

Once a budget is set, engineers must select components and design the workflow to meet it. This involves making conscious trade-offs. For instance, they might choose a less efficient but faster compression codec, or opt for a more expensive, dedicated network connection over the public internet. They must use specialized tools to measure the latency of each individual component and ensure the sum of the parts does not exceed the total budget. Failure to manage the latency budget can lead to unusable systems, compromised performances, and costly production delays.