🧵 Section 9 · Question #19

How to prevent deadlock

Four primary techniques exist in production Java systems: 4. Open Calls Rule: Never invoke external callbacks, foreign listener interfaces, or slow I/O (network/database) while...


🟢 Junior Level

30-Second Summary

Deadlock Prevention is an architectural methodology in concurrent systems design aimed at eliminating the theoretical possibility of a deadlock at compile/design time by proactively invalidating at least one of the four Coffman conditions.

Four primary techniques exist in production Java systems:

  1. Global Lock Ordering: Enforce that all application threads acquire shared synchronizers in an identical, predetermined sequence (e.g., ascending order of entity id). This breaks the Circular Wait condition.
  2. Timed Acquisition (tryLock): Replace uninterruptible synchronized monitors with ReentrantLock.tryLock(timeout, unit). If a subsequent lock cannot be obtained within the deadline, the thread releases all currently held locks and backs off, breaking Hold and Wait and No Preemption.
  3. Lock-Free Concurrency: Replace coarse mutex locks with non-blocking atomic primitives (java.util.concurrent.atomic) and lock-free structures (ConcurrentHashMap). No mutex locks means no Mutual Exclusion.
  4. Open Calls Rule: Never invoke external callbacks, foreign listener interfaces, or slow I/O (network/database) while holding a lock.

Demonstrating Prevention via Lock Ordering in Java 21

public class SafeBankTransfer {

    public record Account(long id, int balance) {}

    // Safe transfer: acquisition sequence is deterministic based on account ID
    public void transfer(Account from, Account to, int amount) {
        // Prevent self-transfer deadlock
        if (from.id() == to.id()) {
            throw new IllegalArgumentException("Cannot transfer funds to the same account");
        }

        // Deterministic ordering: smaller ID is unconditionally locked first!
        Account firstLock = from.id() < to.id() ? from : to;
        Account secondLock = from.id() < to.id() ? to : from;

        synchronized (firstLock) {
            synchronized (secondLock) {
                // Critical section: circular dependency is mathematically impossible!
                debit(from, amount);
                credit(to, amount);
            }
        }
    }

    private void debit(Account acc, int amount) {}
    private void credit(Account acc, int amount) {}
}

🟡 Middle Level

Timed Lock Acquisition with Jitter Back-off (tryLock)

When dynamic resources lack a natural domain order, systems rely on timed lock acquisition:

[!WARNING] Livelock Danger: If two threads fail to acquire their secondary resource, simultaneously release their primary lock, and immediately retry in lockstep, they enter a high-frequency collision loop consuming 100% CPU (Livelock). A randomized delay (Jitter / Random Back-off) is mandatory to break synchronization.

import java.util.concurrent.ThreadLocalRandom;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.locks.ReentrantLock;

public class SafeTryLockTransfer {

    public boolean transferWithTimeout(ReentrantLock lockA, ReentrantLock lockB, 
                                       long maxTimeout, TimeUnit unit) throws InterruptedException {
        long deadline = System.nanoTime() + unit.toNanos(maxTimeout);

        while (System.nanoTime() < deadline) {
            // Attempt to acquire primary resource
            if (lockA.tryLock(50, TimeUnit.MILLISECONDS)) {
                try {
                    // Attempt to acquire secondary resource
                    if (lockB.tryLock(50, TimeUnit.MILLISECONDS)) {
                        try {
                            executeTransfer();
                            return true; // Successfully transferred
                        } finally {
                            lockB.unlock();
                        }
                    }
                } finally {
                    // CRITICAL: RELEASE lockA IMMEDIATELY if lockB acquisition fails!
                    lockA.unlock();
                }
            }

            // Randomized back-off (Jitter) eliminates synchronous livelock thrashing
            int backoffMs = ThreadLocalRandom.current().nextInt(10, 40);
            Thread.sleep(backoffMs);
        }

        return false; // Deadline expired, aborted safely
    }

    private void executeTransfer() {}
}

Architectural Pattern: Open Calls

Invoking foreign or unverified code while holding an active lock is the single greatest source of hidden deadlocks in enterprise frameworks:

// HAZARDOUS ANTI-PATTERN: Calling alien listeners while holding monitor!
public synchronized void updateState(State newState) {
    this.state = newState;
    for (StateListener listener : listeners) {
        // CATASTROPHE: listener may acquire an internal lock,
        // while another thread holding that lock attempts to call updateState()!
        listener.onStateChanged(newState); 
    }
}

Canonical Open Call Implementation (Zero Lock Holding)

import java.util.List;
import java.util.concurrent.CopyOnWriteArrayList;

public class SafeStateHolder {
    private final List<StateListener> listeners = new CopyOnWriteArrayList<>();
    private final Object stateLock = new Object();
    private State state;

    public void updateState(State newState) {
        // 1. Mutate internal state inside a minimal, tight critical section
        synchronized (stateLock) {
            this.state = newState;
        }

        // 2. Dispatch notifications COMPLETELY OUTSIDE the critical section!
        for (StateListener listener : listeners) {
            listener.onStateChanged(newState); // Open Call
        }
    }
}

🔴 Senior Level

Ordering Arbitrary In-Memory Objects: The Tie-Breaker Lock

When concurrent logic must order two arbitrary in-memory objects lacking natural business keys (e.g., graph vertices), developers sort them using System.identityHashCode().

Because 32-bit identity hash codes can collide, a global static Tie-Breaker Lock guarantees serialization during collisions:

public class ArbitraryLockOrdering {
    private static final Object TIE_BREAKER_LOCK = new Object();

    public void performAction(Object objA, Object objB) {
        int hashA = System.identityHashCode(objA);
        int hashB = System.identityHashCode(objB);

        if (hashA < hashB) {
            synchronized (objA) {
                synchronized (objB) {
                    process(objA, objB);
                }
            }
        } else if (hashA > hashB) {
            synchronized (objB) {
                synchronized (objA) {
                    process(objA, objB);
                }
            }
        } else {
            // HASH COLLISION BRANCH: Serialize rare collisions via static tie-breaker
            synchronized (TIE_BREAKER_LOCK) {
                synchronized (objA) {
                    synchronized (objB) {
                        process(objA, objB);
                    }
                }
            }
        }
    }

    private void process(Object a, Object b) {}
}

Database-Level Deadlocks in JPA / Hibernate

Deadlocks occur in relational databases when concurrent database transactions update the same rows in divergent order:

  • Transaction 1: UPDATE account SET balance = ... WHERE id = 1; followed by WHERE id = 2;
  • Transaction 2: UPDATE account SET balance = ... WHERE id = 2; followed by WHERE id = 1;

ORM/SQL Prevention Rules:

  1. Deterministic Batch Sorting: When executing batch updates or row-level locking (SELECT ... FOR UPDATE), the list of entities must always be sorted ascending by primary key (ORDER BY id ASC) prior to issuing queries.
  2. Pessimistic Lock with Timeout: Always provide explicit lock acquisition timeouts using jakarta.persistence.lock.timeout:
    @Lock(LockModeType.PESSIMISTIC_WRITE)
    @QueryHints({@QueryHint(name = "jakarta.persistence.lock.timeout", value = "3000")})
    @Query("SELECT a FROM Account a WHERE a.id IN :ids ORDER BY a.id ASC")
    List<Account> findAccountsForUpdate(@Param("ids") List<Long> ids);
    

Automated Verification: Static Analysis & ArchUnit

Preventing deadlocks cannot rely exclusively on manual pull request code reviews:

  1. SonarQube / SpotBugs: Enforce rules prohibiting unsorted nested lock acquisitions (Locks should be acquired in a specific order).
  2. ArchUnit Tests: Write architectural assertions forbidding calls to foreign Spring beans or REST/RPC clients from methods annotated with @Transactional or @Synchronized.
  3. ThreadSanitizer (TSan): Enable JVM dynamic data race and lock-order inversion detection during integration testing via -XX:+UseThreadSanitizer.

🎯 Interview Cheat Sheet

1. 30-Second Elevator Pitch

Deadlock prevention focuses on breaking at least one Coffman condition at design time. The industry standard is Lock Ordering (sorting locks deterministically by unique key), which eliminates Circular Wait. Secondary defenses include ReentrantLock.tryLock(timeout) with randomized jitter back-off (breaking Hold and Wait without inducing livelocks) and Open Calls (strictly ensuring external callbacks and I/O are invoked outside synchronized blocks).


2. 4 Tricky Follow-up Questions with Deep Answers

Question 1: What is the “Open Calls” rule in multithreaded architecture, and how does it prevent deadlocks?

Answer: An Open Call is an architectural discipline where invocations of alien code (event listeners, third-party libraries, network clients) are executed strictly outside synchronized blocks or lock scopes. Holding a lock while calling external code surrenders control of the lock hierarchy: foreign code may acquire another lock internally, creating an inadvertent reverse dependency with another thread and inducing deadlock.

Question 2: Why does naive retry looping with tryLock() often morph deadlocks into livelocks?

Answer: If Thread 1 holds Lock $A$ and tries Lock $B$, while Thread 2 holds Lock $B$ and tries Lock $A$, both tryLock() attempts fail. If both threads immediately release their respective locks and re-attempt acquisition synchronously without delay, they lock, fail, release, and repeat continuously in lockstep. The system burns 100% CPU without accomplishing useful work (Livelock). Adding randomized exponential back-off (Jitter) de-synchronizes retry cycles, allowing one thread to win both locks.

Question 3: How do you prevent deadlocks when executing JPA/Hibernate batch updates across multiple entities?

Answer: Database row-level deadlocks arise from inconsistent row acquisition order across parallel transactions. To prevent them:

  1. Pre-sort all target entity IDs: ids.sort(Long::compareTo) before executing batch statements.
  2. In SQL queries, append explicit ordering: SELECT ... FOR UPDATE ORDER BY id ASC.
  3. Configure query hints with timeouts (jakarta.persistence.lock.timeout or Postgres NOWAIT / SKIP LOCKED) so contested transactions fail fast rather than stalling indefinitely.

Question 4: Why is a Tie-Breaker Lock required when ordering locks using System.identityHashCode()?

Answer: Identity hash codes are 31-bit integers generated by the JVM. By the pigeonhole principle and birthday paradox, distinct objects will occasionally collide with the identical identity hash code. When a collision occurs, relational ordering (hashA < hashB) fails, making lock acquisition order non-deterministic. A global static TIE_BREAKER_LOCK serializes access for this rare collision branch, guaranteeing safe mutual exclusion.


3. Red Flags & Common Gotchas

❌ “Using a single global lock (synchronized(globalLock)) solves all deadlocks.”
Correction: While it eliminates deadlocks, it serializes entire distributed pipelines, collapsing system throughput down to single-threaded performance.

❌ “Holding a lock while making an HTTP or database RPC call.”
Correction: Violates the Open Calls principle; a delay in the remote network service blocks the local worker thread, cascading into thread pool exhaustion.

❌ “Assuming tryLock() without timeouts is completely safe.”
Correction: Immediate tryLock() loops without random back-off create severe livelocks.