Webhook Retries: Accept One Event Without Starting Two Jobs

Suppose your webhook handler records a paid order, then the connection drops before the sender receives your response. A second delivery arrives. The first request may already have done its work, so repeating the handler can start the same job twice.

Accept a verified event into durable storage under a unique event identity, commit it, and only then acknowledge it. Process that stored job separately. A repeat delivery should find the existing record instead of creating another job. You still need a separate guard around the business action, especially when a worker calls an external service.

This is a hypothetical failure scenario and an implementation walkthrough, not an account of a Krova customer incident. The examples are illustrative and have not been executed against a production integration.

The response disappeared; the database may have committed

The sender knows it didn't receive a successful response. It doesn't necessarily know whether your database committed. Retrying is how delivery recovers, but your application has to make that recovery safe. HTTP's idempotency definition describes repeated requests having the same intended effect; it does not make every POST handler safe automatically.

For a concrete provider example, Stripe documents duplicate event deliveries and recommends asynchronous processing. Its event IDs identify repeated deliveries of an event. That is a useful starting point, not permission to assume every webhook provider uses the same identifier, retry window, or acknowledgment rules.

Before changing code, write down three identities: who sent the event, which event it is, and which business action it requests. A delivery attempt, an event, and an order are different things. Collapsing them into one ID makes debugging much harder.

Give the event one durable place to land

For a modest PostgreSQL-backed service, an inbox table can also be the pending-job table. That removes the gap between saving a deduplication marker and publishing to a separate queue. The schema below is a proposed starting point for PostgreSQL 18; it omits a complete worker implementation.

CREATE TABLE webhook_jobs (
  provider text NOT NULL,
  account_scope text NOT NULL,
  event_id text NOT NULL,
  payload jsonb NOT NULL,
  state text NOT NULL DEFAULT 'pending'
    CHECK (state IN ('pending', 'running', 'done', 'failed')),
  accepted_at timestamptz NOT NULL DEFAULT now(),
  PRIMARY KEY (provider, account_scope, event_id)
);

-- Bind these parameters after signature verification.
INSERT INTO webhook_jobs
  (provider, account_scope, event_id, payload)
VALUES ($1, $2, $3, $4::jsonb)
ON CONFLICT (provider, account_scope, event_id) DO NOTHING
RETURNING event_id;

The composite key scopes an event to its provider and account. Derive that account scope from trusted endpoint configuration or verified provider context; don't let an untrusted payload choose another tenant's namespace. Use one stable event ID across attempts, rather than generating a new job ID for each HTTP request.

PostgreSQL's INSERT documentation defines ON CONFLICT DO NOTHING and RETURNING. Here, a returned row means this transaction inserted the event. No returned row means the key already exists. The unique constraint arbitrates concurrent deliveries; a separate “check whether it exists” query cannot safely replace it.

Keep this insert and any related local acceptance records in one transaction. Amazon's idempotent API guidance explains why the request token and mutations must form an atomic operation. If you later introduce a separate message broker, use a transactional outbox or an equivalent recovery design rather than assuming two independent writes succeed together.

Acknowledge acceptance, then let the worker work

receive request
  enforce a request-body size limit
  verify provider signature using the required raw bytes
  validate the event and resolve its trusted account scope
  begin database transaction
    insert event into webhook_jobs, ignoring its unique-key conflict
  commit
  return the provider's expected success response

worker
  claim pending work with concurrency control and a recoverable lease
  apply the business-action deduplication rule
  perform the action, reconciling uncertain external outcomes
  record success, or schedule a bounded retry / investigation

The success response comes after a confirmed commit, including when a duplicate points to an already accepted record. If storage is unavailable, don't acknowledge work you cannot recover. If the commit result itself is uncertain, reconnect and check the same key or let a later delivery resolve it; don't create a new identity.

Stripe's signature guidance requires the unmodified request body. Verify before trusting identifiers or storing work for execution. An inbox key is a duplicate guard, not authentication.

Returning success now means “the event is durably accepted.” It does not mean the order is fulfilled. Alert on pending-job age and failures so that a healthy HTTP endpoint cannot hide a stuck worker.

One event is not always one business action

Return to the hypothetical order. The inbox blocks a repeat of the same event ID, but a worker can still crash after an external fulfillment call succeeds and before it records completion. Running that job again may repeat the external action.

Choose a business key for the action you intend to perform once: for example, a provider account, order ID, and fulfillment operation. Enforce it atomically with local database changes. For a remote API, use its documented idempotency facility with the same operation key on retries, or reconcile the remote state before attempting another mutation. An inbox alone cannot promise exactly-once effects across systems.

Stripe also notes that distinct Event objects can describe duplicates and suggests checking the object ID together with event type. Apply that to the specific action: deduplicating every update to the same subscription forever would discard legitimate later changes. Document what counts as the same operation and how long its guard must remain.

Ordering needs its own rule too. A later delivery is not necessarily a later change. For stateful integrations, use the provider's documented versioning or retrieve current state instead of blindly overwriting a newer state with an older event.

Retention is part of that contract. If you delete an event record, an old replay may look new again. Choose retention using the provider's documented retry and manual-replay behavior plus your own business requirements. Limit stored payloads to what you need, protect sensitive fields, and give failed work a recovery path.

Try the failures before you rely on the happy path

Run these checks in a separate test environment with provider test events and controlled downstream services. They are proposed acceptance checks, not reported test results.

• Deliver the same verified event sequentially and concurrently. Confirm one inbox record and one pending job.
• Drop the connection after commit but before the response reaches the sender. Redelivery should find the accepted event.
• Make storage unavailable. Confirm the endpoint does not claim durable acceptance.
• Crash a worker after a simulated remote success but before marking completion. Confirm reconciliation or the remote operation key prevents a second effect.
• Send two legitimate updates for the same object. Confirm the business rule preserves both when they represent different actions.
• Replay an older event and test an invalid signature. Check the documented retention boundary and signature rejection behavior.

Log provider, account scope, event ID, action key, acceptance result, and worker state without copying secrets or unnecessary personal data into logs. Watch oldest pending age, retry attempts, failed jobs, and duplicate receipts separately. A higher duplicate count can reflect normal redelivery; a growing queue age tells you work is falling behind.

Where a Cube fits

A Krova Cube gives you a Linux microVM with root access for a webhook receiver and worker you operate. You remain responsible for the application, database durability, backups, worker recovery, and the deduplication contract. Separate instances can separate workloads; they don't automatically make an operation idempotent.

If burst traffic is also a problem, the existing API rate-limit guide covers a different control. Rate limits govern how much work enters; the inbox governs whether a repeated event creates another job. You may need both.

Creation context: AI-assisted, source-backed technical guidance. No production test results or personal testing by the bylined author are claimed. Sources checked on 4 November 2026.