What is idempotency

In software engineering and distributed network protocols, Idempotency is a mathematical and architectural property of an operation where executing it multiple times with the ex...

🟢 Junior Level

In software engineering and distributed network protocols, Idempotency is a mathematical and architectural property of an operation where executing it multiple times with the exact same parameters produces the exact same resulting state on the server as executing it a single time.

In mathematical notation: $f(f(x)) = f(x)$

Executing an idempotent operation once changes the system state to the desired target. Executing it a second, tenth, or hundredth time produces zero additional side effects on the server’s resource state.

Real-World Analogies

Idempotent Action: Pressing a dedicated “Turn Off Light” button. If the light is on, it turns off. If the light is already off, pressing the button 10 more times leaves the light off.
Non-Idempotent Action: Pressing a “Toggle Light” button or a “Debit $100 from Account” button. Every subsequent press alters the state further (the light toggles on and off; your bank balance decreases by another $100 each time).

Idempotency in HTTP (RFC 9110)

The HTTP specification explicitly classifies standard methods by idempotency:

GET /users/42 — Idempotent (and Safe). Reading data does not alter server state, regardless of repetition count.
PUT /users/42 — Idempotent. The payload {"name": "Alice"} completely overwrites the target resource. Replaying the exact same payload leaves the resource state identical.
DELETE /users/42 — Idempotent. The first request removes the user. Subsequent requests observe that the user is already absent. The server’s final state (User 42 does not exist) is identical after call 1 and call 100.
POST /users — Non-idempotent. Every execution creates an additional, distinct user entity in the database with a newly generated identifier.

Why Idempotency Is Critical in Distributed Systems

Computer networks are inherently unreliable. When a client encounters a Read timed out or Connection Reset error while invoking an API, the client faces three possibilities:

The request packet never reached the server.
The server crashed midway through execution.
The server executed the operation successfully, but the response packet was dropped by the network.

If an endpoint is idempotent, the client or proxy can safely retry the request automatically without the risk of creating duplicate database records or executing duplicate financial transactions.

🟡 Middle Level

Safe vs. Idempotent Methods (RFC 9110)

RFC 9110 establishes a clear distinction between Safety and Idempotency:

Property	Formal Definition	Permitted State Mutations	HTTP Methods
Safe (Read-Only)	The method is semantically intended for retrieval only; it must not modify server resource state.	❌ Zero mutations permitted	`GET`, `HEAD`, `OPTIONS`, `TRACE`
Idempotent	Multiple identical requests leave the server in the same resulting resource state as a single request.	✅ Permitted on initial call; subsequent calls cause zero additional change	`GET`, `HEAD`, `OPTIONS`, `TRACE`, `PUT`, `DELETE`
Non-Idempotent	Successive identical requests may accumulate state mutations and side effects.	✅ Every execution can modify server state	`POST`, `PATCH` (by default)

[!NOTE] All Safe methods are automatically Idempotent ($f(x) = x \implies f(f(x)) = f(x)$). However, not all idempotent methods are safe: PUT and DELETE mutate database records, but their mutations stabilize after the first execution.

The Fundamental Paradox: Server Resource State vs. HTTP Status Codes

A classic interview trap asks:

“The first call to DELETE /orders/101 returns 204 No Content. The second call returns 404 Not Found. Because the HTTP status codes differed, is the method non-idempotent?”

The Correct Answer: The method is strictly idempotent. RFC 9110 defines idempotency strictly by the side effects on the server’s resource state, not by the identity of HTTP response status codes or headers:

Prior to Request 1: Order 101 exists in the database.
After Request 1: Order 101 is deleted from the database.
After Request 2: Order 101 remains absent from the database. Because the server’s database state is identical after Request 1 and after Request 2 (Order 101 does not exist), the operation satisfies the definition of idempotency.

The Post/Redirect/Get (PRG) Pattern in Web Architectures

When web browsers submit an HTML form (e.g., checkout order creation), they dispatch a non-idempotent POST request. If the user refreshes the browser page (F5), the browser prompts a warning: “Confirm Form Resubmission”, risking duplicate orders.

The Post/Redirect/Get (PRG) pattern eliminates this vulnerability:

The browser submits form data via POST /orders.
The server creates the order in the database and responds with 303 See Other (or 302 Found) and a Location: /orders/123 header.
The browser automatically follows the redirect by issuing an idempotent GET /orders/123.
If the user subsequently refreshes the page (F5), the browser safely re-executes the idempotent GET request without re-submitting the payment form.

🔴 Senior Level

Making POST Idempotent: The `Idempotency-Key` Pattern

In high-concurrency e-commerce, banking, and payment gateways (such as Stripe, Adyen, and PayPal), non-idempotent POST endpoints must be protected against network retries.

The industry standard pattern (formalized in IETF Draft draft-ietf-httpapi-idempotency-key-header) requires clients to attach a unique client-generated UUID header:

POST /api/v1/payments HTTP/1.1
Host: api.example.com
Idempotency-Key: 9b1deb4d-3b7d-4bad-9bdd-2b0d7b3dcb6d
Content-Type: application/json

{
  "accountId": 1001,
  "amount": 5000
}

Idempotency Engine State Machine

Client                     API Gateway / Filter                      Redis Cache               Business Logic
  │                                  │                                    │                          │
  ├─ POST + Idempotency-Key ────────>│                                    │                          │
  │                                  ├─ SETNX lock:key (requestHash) ────>│                          │
  │                                  │<─ 1 (Lock Acquired) ───────────────┤                          │
  │                                  ├──────────────────────────────────────────────────────────────>│
  │                                  │                                    │             (Execute TX) │
  │                                  │<─ Response (201 Created + Body) ──────────────────────────────┤
  │                                  ├─ SET data:key responseBody (TTL) ─>│                          │
  │<─ 201 Created + Body ────────────┤                                    │                          │
  │                                  │                                    │                          │
  │── Retry POST (Same Key) ────────>│                                    │                          │
  │                                  ├─ SETNX lock:key ──────────────────>│                          │
  │                                  │<─ 0 (Lock Failed / Key Exists) ────┤                          │
  │                                  ├─ GET data:key ────────────────────>│                          │
  │                                  │<─ Cached Response Body ────────────┤                          │
  │<─ 201 Created + Cached Body ─────┤ (Skip Business Logic!)             │                          │

Edge Case Handling:

Concurrent Duplicate (PROCESSING Lock): If a second request arrives with the same key while the first is still executing, the server rejects it with 409 Conflict or 425 Too Early to prevent concurrent execution races.
Payload Hash Mismatch: If an incoming request carries an existing Idempotency-Key but the request payload has changed (e.g., amount: 9999 instead of 5000), the server must reject the request with 422 Unprocessable Content or 400 Bad Request. The cache stores the SHA-256 hash of the initial payload to detect tampering.
Cache Expiration (TTL): Distributed cache records (Redis) must enforce a bounded TTL (typically 24 hours), after which keys expire.

Production-Ready Spring Boot 3 + Redis Idempotency Filter

package com.example.idempotency;

import jakarta.servlet.FilterChain;
import jakarta.servlet.ServletException;
import jakarta.servlet.http.HttpServletRequest;
import jakarta.servlet.http.HttpServletResponse;
import org.springframework.data.redis.core.StringRedisTemplate;
import org.springframework.http.HttpStatus;
import org.springframework.http.MediaType;
import org.springframework.stereotype.Component;
import org.springframework.util.DigestUtils;
import org.springframework.web.filter.OncePerRequestFilter;
import org.springframework.web.util.ContentCachingRequestWrapper;
import org.springframework.web.util.ContentCachingResponseWrapper;

import java.io.IOException;
import java.nio.charset.StandardCharsets;
import java.time.Duration;

@Component
public class IdempotencyFilter extends OncePerRequestFilter {

    private static final String IDEMPOTENCY_HEADER = "Idempotency-Key";
    private static final Duration LOCK_TIMEOUT = Duration.ofMinutes(2);
    private static final Duration CACHE_TTL = Duration.ofHours(24);

    private final StringRedisTemplate redisTemplate;

    public IdempotencyFilter(StringRedisTemplate redisTemplate) {
        this.redisTemplate = redisTemplate;
    }

    @Override
    protected void doFilterInternal(HttpServletRequest request,
                                    HttpServletResponse response,
                                    FilterChain filterChain) throws ServletException, IOException {

        String idempotencyKey = request.getHeader(IDEMPOTENCY_HEADER);
        if (idempotencyKey == null || !"POST".equalsIgnoreCase(request.getMethod())) {
            filterChain.doFilter(request, response);
            return;
        }

        ContentCachingRequestWrapper wrappedRequest = new ContentCachingRequestWrapper(request);
        ContentCachingResponseWrapper wrappedResponse = new ContentCachingResponseWrapper(response);

        // Read request body to compute checksum
        byte[] requestBody = wrappedRequest.getInputStream().readAllBytes();
        String requestHash = DigestUtils.md5DigestAsHex(requestBody);

        String redisLockKey = "idemp:lock:" + idempotencyKey;
        String redisDataKey = "idemp:data:" + idempotencyKey;

        // Atomic distributed lock acquisition via Redis SETNX
        Boolean acquired = redisTemplate.opsForValue().setIfAbsent(redisLockKey, requestHash, LOCK_TIMEOUT);

        if (Boolean.FALSE.equals(acquired)) {
            // Lock was not acquired: check if completed response is already available
            String cachedResponse = redisTemplate.opsForValue().get(redisDataKey);
            if (cachedResponse != null) {
                response.setStatus(HttpStatus.OK.value());
                response.setContentType(MediaType.APPLICATION_JSON_VALUE);
                response.setHeader("X-Cache-Lookup", "HIT-IDEMPOTENT");
                response.getWriter().write(cachedResponse);
                return;
            }

            // Key is still locked in PROCESSING state by another concurrent thread/pod
            response.setStatus(HttpStatus.CONFLICT.value());
            response.setContentType(MediaType.APPLICATION_PROBLEM_JSON_VALUE);
            response.getWriter().write("{\"title\":\"Conflict\",\"detail\":\"Concurrent request in progress\"}");
            return;
        }

        try {
            filterChain.doFilter(wrappedRequest, wrappedResponse);

            byte[] responseBytes = wrappedResponse.getContentAsByteArray();
            String responseStr = new String(responseBytes, StandardCharsets.UTF_8);

            // Cache only successful completions
            if (wrappedResponse.getStatus() < 400) {
                redisTemplate.opsForValue().set(redisDataKey, responseStr, CACHE_TTL);
            }

            wrappedResponse.copyBodyToResponse();
        } finally {
            // Release the lock key; completed result remains in redisDataKey
            redisTemplate.delete(redisLockKey);
        }
    }
}

Database-Level Idempotency Guardrails

Even if network filters or distributed caches encounter temporary partitions, the persistence tier must enforce ultimate idempotency:

Database Unique Constraints:

ALTER TABLE payments ADD CONSTRAINT uq_payments_idempotency_key UNIQUE (idempotency_key);

Atomic UPSERT Mutations:

INSERT INTO payments (idempotency_key, account_id, amount, status)
VALUES (:key, :account, :amount, 'COMPLETED')
ON CONFLICT (idempotency_key) DO NOTHING;

Idempotent Consumer & Transactional Inbox Pattern: In event-driven messaging (Kafka, RabbitMQ), consumers track processed message UUIDs in an inbox_events table within the same ACID transaction as the business operation. If duplicate messages arrive, the database constraint rejects them silently.

4 Tricky Questions

1. If the first call to `DELETE` returns `204 No Content` and the second call returns `404 Not Found`, why is the method strictly idempotent?

Answer: Under RFC 9110, idempotency is evaluated exclusively by the resulting resource state on the server, not by the HTTP response status code or header equality.

Prior to Call 1: The resource exists in the database.
After Call 1: The resource is deleted (system state: resource is absent).
After Call 2: The resource remains deleted (system state: resource is absent). Because the server’s persistent state is completely identical after Call 1 and Call 2, the operation satisfies the mathematical definition of idempotency ($f(f(x)) = f(x)$).

2. How do Safe and Idempotent methods differ mathematically and practically?

Answer:

Mathematical Distinction:
- A Safe method produces zero state modification: $f(x) = x$.
- An Idempotent method may produce an initial state modification, but repeated calls stabilize: $f(f(x)) = f(x)$.
Practical Distinction:
- Every safe method is inherently idempotent (GET, HEAD, OPTIONS).
- Idempotent methods can modify data (PUT overwrites a record, DELETE removes a record), but executing them $N$ times leaves the database in the exact same state as executing them once.

3. What must the server do if a request arrives with an existing `Idempotency-Key` but with a different request payload?

Answer: The server must reject the request immediately with 422 Unprocessable Content or 400 Bad Request.

Reusing an existing Idempotency-Key with a different payload (e.g., different payment amount or currency) indicates a client integration bug or an intentional tampering attempt.
The server computes a cryptographic digest (SHA-256) of the initial request payload and stores it alongside the key. If a subsequent request arrives with a matching key but mismatched payload digest, the server aborts the request before invoking business logic.

4. What is a “Retry Storm” and how does idempotency interact with it?

Answer: A Retry Storm occurs when a backend service experiences transient degradation or partial latency spikes, causing thousands of clients to time out simultaneously and immediately retry their requests.

Idempotency’s Role: Idempotency guarantees data safety during retries (preventing duplicate orders or payments).
The Threat: Without client-side backoff, thousands of retried requests overwhelm the already struggling backend, triggering a cascading total outage.
Mitigation:
- Clients must implement Exponential Backoff with Full Random Jitter (delay = random(0, min(max_delay, base * 2^attempt))).
- Servers must deploy Rate Limiters and Circuit Breakers (e.g., Resilience4j) to shed load during incidents.

🎯 Interview Cheat Sheet

30-Second Elevator Pitch

Idempotency (RFC 9110) is the property guaranteeing that executing an operation multiple times with identical parameters leaves the server in the exact same resource state as executing it once ($f(f(x)) = f(x)$).

Idempotent Methods: GET, HEAD, OPTIONS, PUT (full replacement), DELETE (removal).

Non-Idempotent Methods: POST (creates new resources on every call), PATCH (operation-dependent).

State vs. Status: Idempotency is measured by server resource state, not HTTP status codes (DELETE returning 204 then 404 is 100% idempotent).

POST Idempotency: Implemented via the Idempotency-Key pattern with Redis distributed locks, payload hashing, and database UNIQUE constraints.

Red Flags (What NOT to Say)

❌ “An operation is only idempotent if it returns the exact same HTTP status code every time.” (Idempotency measures server state side effects, not response codes).
❌ “Safe and Idempotent are interchangeable synonyms.” (Safe methods cause no mutations; Idempotent methods may mutate data on the first invocation).
❌ “It is impossible to make a POST request idempotent.” (POST endpoints are made idempotent in production via Idempotency-Key headers and transactional deduplication).
❌ “PATCH is always idempotent just like PUT.” (PATCH operations such as array appends or numeric increments are non-idempotent).

Which HTTP methods are idempotent — Comprehensive method catalog
Why GET and DELETE are idempotent — Invariants of deletion and retrieval
Is POST idempotent — Detailed analysis of POST processing
What is the difference between PUT and PATCH — Mutation semantics
What are the main HTTP methods used in REST — HTTP verb semantics