🏗️ Section 17 · Question #6

How Does Circuit Breaker Work and What States Does It Have

A Circuit Breaker functions as a Finite State Machine (FSM) positioned between a client application and a remote downstream microservice. It monitors the outcome and latency of...

🏗️ Microservices 📄 Source Markdown

🟢 Junior Level

A Circuit Breaker functions as a Finite State Machine (FSM) positioned between a client application and a remote downstream microservice. It monitors the outcome and latency of all outbound requests, automatically cutting off traffic to a struggling or degraded service to protect system resources.

                    ┌────────────────────────┐
                    │         CLOSED         │◄──────────────┐
                    │  (Traffic passes,      │               │
                    │   metrics recorded)    │               │
                    └───────────┬────────────┘               │
       failureRate >= threshold │                            │ All permitted
       OR slowCalls >= threshold│                            │ probe calls succeed
                                ▼                            │
                    ┌────────────────────────┐               │
                    │          OPEN          │               │
                    │  (Zero network calls,  │               │
                    │   instant Fail-Fast)   │               │
                    └───────────┬────────────┘               │
                                │ waitDuration expires       │
                                ▼                            │
                    ┌────────────────────────┐               │
                    │       HALF-OPEN        │               │
                    │  (Trial probe calls)   ├───────────────┘
                    └───────────┬────────────┘
                                │ Any probe call fails
                                └──► (Return to OPEN)

The Three Core States

State Traffic Flow Behavior on Request
CLOSED 🟢 100% Permitted Requests execute against the remote service. Every call (success, failure, latency) is recorded in a sliding window buffer.
OPEN 🔴 0% Permitted Outbound network calls are completely blocked. Calls fail fast immediately ($0\text{ ms}$) throwing CallNotPermittedException or executing a fallback.
HALF-OPEN 🟡 Limited Probe Batch A fixed number of trial requests are sent to evaluate if the remote service has recovered. Other concurrent calls are rejected immediately.

🟡 Middle Level

1. State Transition Rules

Rule 1: CLOSED $\to$ OPEN

The transition from CLOSED to OPEN occurs only when both conditions are met:

  1. Total calls evaluated in the sliding window $\ge$ minimumNumberOfCalls.
  2. The percentage of failed calls $\ge$ failureRateThreshold (e.g., $50\%$) OR the percentage of slow calls $\ge$ slowCallRateThreshold (e.g., $60\%$).

Rule 2: OPEN $\to$ HALF-OPEN

When the circuit breaker trips to OPEN, an internal timer starts. Once the configured waitDurationInOpenState (e.g., $15\text{ seconds}$) expires, the breaker transitions to HALF-OPEN to test downstream service health.

Rule 3: HALF-OPEN $\to$ CLOSED (Recovery)

In HALF-OPEN, the breaker permits exactly permittedNumberOfCallsInHalfOpenState (e.g., $5\text{ to }10$) test requests:

  • If the error rate and slow call rate among these probe requests are below the configured thresholds, the circuit resets to CLOSED, the sliding window is cleared, and normal traffic resumes.

Rule 4: HALF-OPEN $\to$ OPEN (Relapse)

If the error rate among the trial probe calls reaches or exceeds the threshold, the circuit trips immediately back to OPEN, restarting the cooldown timer.

2. Sliding Window Mechanics: Count-Based vs Time-Based

Resilience4j calculates failure and latency rates using two distinct sliding window models:

1. COUNT_BASED (Fixed Request Buffer):
   Window Size = 10:  [ ✓,  ✓,  ✗,  ✓,  ✗,  ✗,  ✓,  ✓,  ✓,  ✗ ]  ──► 4 failures / 10 calls = 40%

2. TIME_BASED (Aggregated Time Buckets):
   |  Second 1  |  Second 2  |  Second 3  | ... |  Second 60 |
   |  12 calls  |  20 calls  |   0 calls  | ... |  15 calls  |  ──► Aggregated metrics over 60s
Dimension COUNT_BASED TIME_BASED
Evaluation Unit Fixed number of consecutive calls (e.g., last 100 requests) Fixed time duration (e.g., last 60 seconds)
Memory Buffer Circular ring buffer / Bitset Array of time-bucketed aggregates
Low-Traffic Impact Risk: Old errors from hours ago remain in memory until new calls arrive Safe: Old errors naturally expire and age out as time passes
Best Used For High-throughput, consistent request streams Irregular, bursty, or low-frequency traffic

3. Production Configuration (application.yml)

resilience4j:
  circuitbreaker:
    instances:
      paymentGateway:
        sliding-window-type: COUNT_BASED
        sliding-window-size: 50
        minimum-number-of-calls: 20
        failure-rate-threshold: 50
        slow-call-rate-threshold: 70
        slow-call-duration-threshold: 2000ms
        wait-duration-in-open-state: 15s
        permitted-number-of-calls-in-half-open-state: 10
        automatic-transition-from-open-to-half-open-enabled: true

🔴 Senior Level

1. Administrative Special States in Resilience4j

Beyond the three canonical states, production environments require operational control states:

Special State System Behavior Operational Use Case
DISABLED Breaker is deactivated; 100% of calls pass through, no metrics tracked. High-volume stress testing and performance benchmarking.
METRICS_ONLY Breaker never blocks calls, but records all metrics to Micrometer/Prometheus. Testing and tuning threshold parameters in production without risk of outages.
FORCED_OPEN Breaker is manually locked open; 100% of calls fail fast to fallback. Scheduled maintenance windows, zero-day mitigation, Chaos Engineering drills.

2. Lock-Free High-Throughput Internals

To handle millions of requests per second without becoming a CPU bottleneck, Resilience4j avoids global synchronized locks:

  • State Management: The current state is held in an AtomicReference<State>. State transitions occur via lock-free compareAndSet().
  • Ring Buffer Bitset: COUNT_BASED windows use a circular bitset backed by primitive arrays and atomic pointers. A success is recorded as 0 and a failure as 1.
  • Performance Characteristics: Recording an execution result is an $O(1)$ time and $O(1)$ space operation with zero GC allocations in the hot path.

3. Concurrency in the HALF-OPEN State (The Thundering Herd Guard)

A common design pitfall is mishandling sudden concurrency when transitioning to HALF-OPEN:

Scenario:

  • Configured: permittedNumberOfCallsInHalfOpenState = 10.
  • At $T = 15\text{s}$, the timer expires and 200 concurrent client requests hit the service simultaneously.
Incoming 200 Parallel Requests in HALF-OPEN:
  ├── First 10 threads acquire probe permits ──► Executed against downstream service
  └── Remaining 190 threads REJECTED instantly ──► Throws CallNotPermittedException / Fallback

The remaining 190 threads do not wait for the 10 probe calls to complete. They fail fast immediately. This prevents a “Thundering Herd” shockwave from overwhelming a downstream service that is just beginning to restart.

4. Exception Classification: Ignored vs Recorded Errors

Not all exceptions indicate that a downstream microservice is unhealthy. Business validation errors (e.g., HTTP 400 Bad Request, HTTP 404 Not Found) represent normal operational logic:

resilience4j:
  circuitbreaker:
    instances:
      orderService:
        recordExceptions:
          - java.io.IOException
          - java.util.concurrent.TimeoutException
          - org.springframework.web.client.HttpServerErrorException # HTTP 5xx errors
        ignoreExceptions:
          - com.example.exceptions.InvalidInputException           # HTTP 400
          - com.example.exceptions.EntityNotFoundException         # HTTP 404

[!CAUTION] If client-side 4xx errors are not added to ignoreExceptions, a malicious user or buggy frontend sending invalid requests will artificially inflate the failure rate and trip the circuit breaker, taking down the backend for all legitimate users!

4 Tricky Questions

1. What happens if in HALF_OPEN state 5 requests are permitted, 4 succeed, and the 5th hangs for 60 seconds on a network timeout?

Answer: The Circuit Breaker cannot transition back to CLOSED until all permitted calls (permittedNumberOfCallsInHalfOpenState) have terminated and reported their results. During the entire 60 seconds that the 5th call is hanging, all incoming client traffic will be immediately rejected with CallNotPermittedException (or routed to fallback). To prevent this operational gridlock, you must pair the Circuit Breaker with a strict HTTP client timeout (ReadTimeout) or Resilience4j’s @TimeLimiter (e.g., 2 seconds). This ensures the hanging request terminates quickly with a TimeoutException, causing the breaker to transition cleanly (either tripping to OPEN or completing the probe evaluation).

2. Why might a Circuit Breaker never trip to OPEN under low traffic even if 100% of requests are failing?

Answer: This is caused by the minimumNumberOfCalls guard. If minimumNumberOfCalls = 50, but the service receives only 40 requests in a day, the circuit breaker refuses to evaluate the failure rate, even if all 40 calls returned HTTP 500. It remains in CLOSED state indefinitely. On low-traffic endpoints, architects must lower minimumNumberOfCalls (e.g., to 5–10) or switch to a TIME_BASED sliding window with a small bucket duration.

3. How does a TIME_BASED sliding window differ fundamentally from COUNT_BASED during a sudden post-incident traffic drop?

Answer:

  • In a COUNT_BASED window (e.g., size 100), if an incident produces 70 errors and traffic suddenly plummets to near zero, those 70 errors remain locked in the ring buffer. Hours later, the breaker will still calculate a high failure rate and trip to OPEN on the next few requests.
  • In a TIME_BASED window (e.g., 60 seconds), metrics are partitioned into time buckets. Once the 60-second window expires, historical error buckets are pruned from memory. When traffic resumes after the outage, the breaker evaluates downstream health with a fresh slate.

4. What is the fundamental difference between slowCallDurationThreshold and network SocketTimeout / ReadTimeout?

Answer:

  • SocketTimeout / ReadTimeout: Implemented at the OS/TCP socket level. When the timer expires, the socket is forcibly closed, throwing a SocketTimeoutException. The caller aborts and does not receive any partial payload.
  • slowCallDurationThreshold: A passive metric measurement within Resilience4j. The connection is never terminated; the request is allowed to complete successfully and deliver its payload to the client. However, Resilience4j marks the call as “slow” in its sliding window. If the proportion of slow calls exceeds slowCallRateThreshold, future calls are prevented by tripping the breaker to OPEN.

🎯 Interview Cheat Sheet

Must Know

  • State Machine Model: Operates across three primary states: CLOSED (pass), OPEN (block), and HALF-OPEN (probe).
  • CLOSED $\to$ OPEN: Requires reaching minimumNumberOfCalls AND exceeding failureRateThreshold or slowCallRateThreshold.
  • OPEN $\to$ HALF-OPEN: Automatic transition after waitDurationInOpenState elapses.
  • HALF-OPEN Mechanics: Evaluates a strict, small quota of test calls (permittedNumberOfCallsInHalfOpenState). Remaining concurrent traffic is rejected immediately (no Thundering Herd).
  • Two Window Types: COUNT_BASED (last $N$ requests) and TIME_BASED (requests over last $T$ seconds).
  • Exception Filtering: Always place HTTP 4xx / business exceptions into ignoreExceptions to prevent clients from tripping backend breakers.

Key Resilience4j Parameters

  • sliding-window-type (COUNT_BASED vs TIME_BASED)
  • sliding-window-size
  • minimum-number-of-calls
  • failure-rate-threshold (default 50%)
  • slow-call-duration-threshold & slow-call-rate-threshold
  • wait-duration-in-open-state
  • permitted-number-of-calls-in-half-open-state

Common Follow-up Questions

  • “Does OPEN state wait for server responses?” $\to$ No, it fails fast in 0ms without opening network sockets.
  • “Does HALF-OPEN allow 50% of traffic through?” $\to$ No, it allows only a small fixed number of probe requests (e.g. 5–10); the rest are rejected instantly.
  • “Why is lock-free concurrency important here?” $\to$ Outbound calls run on every request; lock contention in the circuit breaker would degrade throughput.

Red Flags (DO NOT Say)

  • ❌ “When the breaker is OPEN, requests are queued up in memory until it closes.” (Incoming requests fail fast or execute fallbacks; queueing would exhaust memory).
  • ❌ “Circuit Breaker automatically closes connections that exceed slowCallDurationThreshold.” (It does not close connections; only SocketTimeout or TimeLimiter does that).
  • ❌ “Every Java exception should trip the circuit breaker.” (Business errors like UserNotFoundException must be explicitly ignored).