Idempotency Keys in API Design: How to Make Retries Safe

Network failures are not rare edge cases. They are a normal part of operating distributed systems at scale. Requests time out. Clients retry. Load balancers fail over mid-request. The question is not whether your API will receive duplicate requests — it is whether it handles them correctly when it does.

For most GET requests this does not matter. For operations that charge a payment, create an order, or send a notification, processing the same request twice is a serious correctness problem. Idempotency keys are the standard solution.

What Idempotency Keys Are and How They Work

An idempotency key is a unique string the client generates and attaches to a request. The server uses this key to detect duplicate requests and replay the original response rather than re-executing the operation.

The client sends:

POST /payments
Idempotency-Key: 7f2d3b1a-e49c-4f8a-b2c0-3a5e1d9f0c84
Content-Type: application/json

{ "amount": 5000, "currency": "usd", "source": "card_abc123" }

The server checks whether it has seen this key before:

• First request: execute the operation, store the key and the response, return the response
• Duplicate request (same key): return the stored response, do not re-execute the operation

From the client's perspective, both the first request and any retry receive the same response. The server processes the operation exactly once regardless of how many times the client sends the request.

Why Idempotency Keys Matter for Critical Operations

Consider a payment endpoint without idempotency support. A client sends a charge request. The server processes the charge and sends a 201 Created response. The network drops the response before it reaches the client. The client, having received no response after its timeout, retries. The server processes the charge a second time.

The user was charged twice. This is not a hypothetical — it is the failure mode that has caused real production incidents for real companies.

Idempotency keys prevent this. If the client includes the same Idempotency-Key on the retry, the server recognizes it, returns the stored 201 from the first request, and does not charge the card again.

Implementing Idempotency Keys Correctly

Storage

Store idempotency keys in a persistent, shared store — not in application memory. Redis works well for keys with short TTLs (24 hours to 7 days). A database table works for keys that need longer retention.

The stored record needs:

• The idempotency key
• The client identifier (keys should be namespaced per client — two different clients can use the same key value independently)
• The stored response body and status code
• A creation timestamp
• A processing status flag

Handling In-Flight Duplicate Requests

What happens if two requests with the same key arrive simultaneously — before the first one has completed?

The safest approach: use a distributed lock on the idempotency key. If the lock is held, the second request waits or returns 409 Conflict with a message indicating the operation is in progress. Do not allow two workers to concurrently process the same idempotency key.

Without this protection, a concurrent duplicate can start executing before the first request has finished, defeating the purpose of idempotency.

What to Return for Replays

Return the exact same HTTP status code and response body as the original request. Do not return a different status code (like 200 instead of 201) to signal that this was a replay — clients should not need to distinguish between an original response and a replay.

Some implementations add a response header to indicate a replay:

Idempotent-Replayed: true

This is optional but useful for debugging and for clients that want to log whether they were the cause of a successful operation or just confirmed a previously successful one.

Key Format and Client Responsibility

Idempotency keys should be:

• Client-generated, not server-generated
• Globally unique from the client's perspective (UUIDs are conventional)
• Attached to a specific operation, not reused across different operations

Clients are responsible for generating a new key for each distinct intended operation and reusing the same key on retries of the same intended operation. The API documentation should make this responsibility explicit.

TTL and Expiration

Keys should not be stored indefinitely. A reasonable TTL is 24 hours to 7 days depending on your retry window. After the TTL, the key expires and a request with that key is treated as a new operation.

Document the TTL explicitly. Clients need to know whether a retry sent 8 days after the original request will be treated as a duplicate or a new operation.

Which Endpoints Need Idempotency Keys

Not all endpoints benefit from idempotency key support. Focus on operations where:

• Double-execution has financial, data integrity, or communication consequences
• Network failure during the operation is plausible and the client will retry

Typical candidates:

• Payment and charge endpoints
• Order creation
• Email or SMS notification sends
• Job submission (where a job should run exactly once)
• Resource creation where duplicates are harmful (account creation)

GET, PUT, and DELETE should be idempotent by HTTP semantics without needing explicit key support. PUT applied twice should produce the same state. DELETE applied to an already-deleted resource should return 404 or 204, not an error — this is inherent idempotency, not key-based idempotency.

What to Do When a Request Conflicts With a Stored Key

If a client sends a request with a key it has used before, but with a different request body, the server should return 422 or 409 indicating a conflict. Do not silently process the new request body while ignoring the key mismatch. The client has made a mistake — using the same key for two different intended operations — and that mistake should be surfaced as an error.

Testing Idempotency

Idempotency is a behavior contract that needs explicit test coverage:

• Submit a request, confirm the operation executed, submit the same request with the same key, confirm the operation did not execute again and the response is identical
• Submit two concurrent requests with the same key, confirm only one execution occurred
• Submit a request, wait for the TTL to expire, submit again with the same key, confirm the operation executes again

These are integration tests that require a real or realistic persistence layer — they cannot be unit-tested in isolation.

For backend systems where reliability, idempotency, and payment integrity are core requirements, Clixo designs and builds the API infrastructure that gets these semantics right.