Junior Level

OutOfMemoryError comes in different types — each points to its own problem:

Type	What it means
Java heap space	Heap overflowed
GC overhead limit exceeded	GC runs constantly
Metaspace	Class metadata overflowed
Direct buffer memory	Memory outside Heap (NIO)
Unable to create new native thread	OS thread limit

---

Middle Level

Java heap space

Cause: leak or small -Xmx
Solution:
  1. -XX:+HeapDumpOnOutOfMemoryError
  2. Analysis in MAT
  3. Find leak or increase -Xmx

GC overhead limit exceeded

98% time on GC, < 2% freed
→ Application "froze" in infinite Full GC loop

Solution:
  - Increase Heap
  - Find leak
  - Disable: -XX:-UseGCOverheadLimit (not recommended!)

Metaspace

Class metadata overflowed
Causes:
  - Dynamic generation (Spring, CGLIB)
  - ClassLoader leaks

Solution:
  -XX:MaxMetaspaceSize=512m
  jcmd <pid> VM.metaspace

Direct buffer memory

NIO memory outside Heap
Causes: Netty, ByteBuffer.allocateDirect()

Solution:
  -XX:MaxDirectMemorySize=2g
  Check Netty leaks

Unable to create new native thread

OS cannot create thread
Causes:
  - ulimit -u reached
  - RAM for stacks (-Xss) exhausted

Solution:
  Decrease -Xmx (more RAM for stacks)
  Check ulimit -u

---

Senior Level

Requested array size exceeds VM limit

Array > Integer.MAX_VALUE elements
→ Most often indicates a logic error. But legitimate cases exist: attempting to load a large file into memory entirely.

Compressed Class Space

Compressed Class Space — part of Metaspace for compressed class pointers (4 bytes instead of 8).
Limit: -XX:CompressedClassSpaceSize (1 GB default). Overflow → OOME, even if Metaspace has free space.

Map failed (Memory Mapped Files)

MappedByteBuffer.map() → error
  → vm.max_map_count reached
  → No virtual memory for mapping

Solution: sysctl -w vm.max_map_count=262144

Linux OOM Killer

NOT a Java error!
  → Container: limit < RSS
  → OS kills process via SIGKILL
  → No dumps, no logs!

Diagnostics:
  dmesg | grep -i oom
  → "Out of memory: Kill process"

Summary Table

Type	Where to look	Flag
Heap	Heap leaks	HeapDumpOnOOM
Metaspace	ClassLoader leaks	VM.metaspace
Direct buffer	Netty/NIO	NMT
Native thread	ulimit, -Xss	ps -Hu
GC overhead	Thrashing	GC Logs

Best Practices

1. HeapDumpOnOutOfMemoryError — mandatory
2. NMT=detail for Native Memory
3. MaxMetaspaceSize — limit for containers
4. MaxDirectMemorySize — NIO control
5. Monitor dmesg for OOM Killer
6. jcmd VM.metaspace for Metaspace analysis

Senior Summary

• OOME type = first clue in investigation
• Heap space = Heap leaks
• Metaspace = ClassLoader leaks
• Direct buffer = NIO leaks (outside Heap)
• Native thread = OS limits
• OOM Killer = not a Java error (containers)
• NMT = Native Memory diagnostics

---

Interview Cheat Sheet

Must know:

• Java heap space — Heap overflowed; solution: HeapDumpOnOOM + MAT analysis + find leak or increase -Xmx
• GC overhead limit exceeded — 98% time on GC, < 2% freed; infinite Full GC loop
• Metaspace — class metadata overflowed; causes: dynamic generation (Spring, CGLIB), ClassLoader leaks
• Direct buffer memory — NIO memory outside Heap; Netty, ByteBuffer.allocateDirect(); solution: MaxDirectMemorySize
• Unable to create new native thread — ulimit -u reached or RAM for stacks (-Xss) exhausted
• Requested array size exceeds VM limit — array > Integer.MAX_VALUE; most often a logic error
• Linux OOM Killer — NOT a Java error! Container: limit < RSS → OS kills via SIGKILL → no dumps

Common follow-up questions:

• How to distinguish JVM OOME from OS OOM Killer? — JVM OOME: stack trace in logs; OOM Killer: dmesg | grep -i oom, process disappears without logs
• What is Compressed Class Space OOME? — Part of Metaspace for compressed class pointers (1 GB limit); overflow → OOME, even if Metaspace is free
• Why does “Requested array size exceeds VM limit” happen? — Attempt to create array > Integer.MAX_VALUE elements; often a bug (loading large file entirely)
• How to diagnose OOM Killer in Kubernetes? — dmesg | grep -i oom; Pod terminated (OOMKilled); solution: increase limit or reduce RSS

Red flags (DO NOT say):

• “I’ll disable -XX:-UseGCOverheadLimit and the problem is solved” — this masks the problem; application will still crash with Heap space OOME
• “OOM Killer is a Java bug” — it’s an OS decision; Java doesn’t control container limits
• “Direct buffer memory = part of Heap” — DirectByteBuffer allocates memory OUTSIDE Heap (-Xmx doesn’t affect it)

Related topics:

• [[19. What happens on OutOfMemoryError]]
• [[11. What is Metaspace (or PermGen)]]
• [[6. What is a memory leak in Java]]
• [[18. What are -Xms and -Xmx parameters]]
• [[21. What is a memory leak and how to detect it]]