HTTP 429 Retries: Respect Retry-After Without Losing the Deadline

Suppose your worker is importing a catalogue from an API. The next request returns 429, and the client library tries again immediately. Several workers do the same thing. The import stops making progress while the request count keeps climbing. The first correction belongs in the retry decision.

Read Retry-After when it is present, check whether the operation can safely be repeated, and keep both an attempt limit and an overall deadline. If the server’s requested wait would outlast that deadline, stop the interactive operation or defer the background job. A shorter timeout in your client doesn’t make an earlier retry useful.

RFC 6585 defines 429 Too Many Requests for rate limiting and permits a Retry-After field. It leaves the server’s counting policy open. The limit might apply to an account, credential, route or another scope. Check the API’s documentation and error details before assuming another worker has an independent allowance.

This illustrative response asks the client to wait 20 seconds after receipt before a follow-up request.

HTTP/1.1 429 Too Many Requests
Retry-After: 20

HTTP Semantics allows Retry-After to contain either a non-negative integer number of seconds or an HTTP date. Parse the whole value as one of those forms. A date needs a clock-aware calculation of the remaining wait. Keep your client clock synchronized and account for clock skew. Missing or invalid values need an explicit fallback policy.

For the hypothetical import, imagine the caller has 12 seconds left before its deadline. A 20-second requested wait cannot fit. Return a clear deferred or unsuccessful result, or persist a later attempt if this is a durable background workflow. Don’t clamp the 20 seconds down to 12 and send anyway. Reserve time for the next request itself when deciding whether an attempt fits.

When the API provides no usable wait instruction, a bounded backoff with jitter is a common fallback. AWS’s retry and jitter guidance explains how synchronized retries can amplify load and why retries need limits. When a usable server delay exists, a practical policy is to wait at least that long, apply any longer local backoff, and add randomized spread afterward. That policy is an implementation choice, subject to the API’s contract.

Choose one place to own the retry budget. If an SDK retries internally and your job runner retries the entire operation, the combined request count can exceed what either setting suggests. Record the actual attempts, the chosen delay, the remaining deadline and the reason for stopping. Keep credentials and sensitive response content out of logs.

A safe repeat is a separate decision from how long to wait. HTTP’s idempotency rules describe which methods have repeatable intended effects. For a write, use the API’s documented idempotency or reconciliation mechanism. Keep the same operation identity where that contract requires it. A fresh identifier on every retry can create fresh work.

If the allowance is shared, coordinate admission across the workers that use it. Starting more workers can consume the allowance faster. Keep the stop or defer decision visible to the job-status endpoint so the caller knows whether the catalogue import is waiting, failed or complete. The Krova HTTP 202 guide covers that asynchronous status contract.

Rehearse against a local stub before changing the live integration. Cover an integer delay, a future date, a past date, a missing or malformed field, and a wait longer than the remaining deadline. Include cancellation during the wait and repeated 429 responses until the attempt budget is exhausted. Check the actual outbound attempts and final job state, including what happens when the process restarts.

This article was prepared with AI assistance from the linked primary documentation. The catalogue import, timing example and proposed rehearsal are hypothetical. No API load test or live retry sequence was run.