Proxy Latency Budgets: Setting and Enforcing Timeout Expectations

Slow Requests Are Expensive

A request that takes ten times longer than normal does not just waste its own time. In a batch pipeline, the whole job waits for the slowest item. Proxy latency budgets give every request a deadline: finish in time or fail fast so the job can move on. It is the simplest performance control you can add.

Without a budget, a few bad requests can idle an entire fleet while they crawl.

Set the Budget From Data, Not Guesses

Start by measuring. Look at the latency distribution of a healthy week and set the cutoff above the normal tail, not at the average. A budget that is too tight kills good requests; one that is too loose lets bad ones linger. The health baseline guide shows how to capture those percentiles to base the number on.

Review the budget as targets and pools change. A number that was right for one domain can be wrong for another.

Timeouts Should Fail Fast

A timeout is not an error to avoid; it is a decision to stop waiting. Set per-request timeouts that match the budget, and set a shorter connect timeout so a dead route fails before the read even starts. The gateway checklist covers where these belong in a new setup.

Fast failure keeps workers busy on requests that can succeed instead of stuck on ones that cannot.

Treat Timeouts as a Signal

A rising timeout rate is information. It can mean a target slowed down, a pool degraded or a geo route went bad. Log which domains and which routes time out, and alert on the trend rather than the single event. The pool diversity guide shows how to route around a bad path once you spot it.

The docs detail timeout configuration, and the proxies catalog shows the latency profile of each tier. A budget that is measured, enforced and watched turns slow requests from a drag into data.