What do distinct(), sorted(), limit(), skip() operations do

All four methods are intermediate stateful operations in the Java Stream API that reshape, reorder, or alter the cardinality of elements passing through the pipeline:

🟢 Junior Level

30-Second Summary

All four methods are intermediate stateful operations in the Java Stream API that reshape, reorder, or alter the cardinality of elements passing through the pipeline:

distinct(): Filters out duplicate elements, retaining only unique items based on their equals() and hashCode() contract.
sorted(): Sorts stream elements into natural order (Comparable) or according to a provided Comparator.
limit(long maxSize): Truncates the stream so that it emits no more than maxSize elements downstream (a short-circuiting operation).
skip(long n): Discards the first n elements of the stream and forwards all remaining elements downstream.

Java 21 Code Examples

import java.util.Comparator;
import java.util.List;

public class StatefulOpsDemo {

    public record Product(String name, double price) {}

    public static void main(String[] args) {
        List<Integer> numbers = List.of(5, 2, 8, 2, 9, 1, 5, 8, 3);

        // 1. Combining distinct, sorted, skip, and limit
        List<Integer> result = numbers.stream()
            .distinct()            // [5, 2, 8, 9, 1, 3] (Duplicates removed)
            .sorted()              // [1, 2, 3, 5, 8, 9] (Ascending order)
            .skip(1)               // [2, 3, 5, 8, 9]    (Smallest item skipped)
            .limit(3)              // [2, 3, 5]          (Takes next 3 items)
            .toList();

        System.out.println("Processed numbers: " + result);

        // 2. Sorting objects using Comparator
        List<Product> products = List.of(
            new Product("Keyboard", 75.0),
            new Product("Mouse", 25.0),
            new Product("Monitor", 300.0)
        );

        List<Product> sortedByPrice = products.stream()
            .sorted(Comparator.comparingDouble(Product::price))
            .toList();

        System.out.println("Sorted products: " + sortedByPrice);
    }
}

🟡 Middle Level

Stateless vs. Stateful Intermediate Operations

In the Stream API architecture, intermediate operations are fundamentally divided into two categories:

Stateless Pipeline (filter, map):
Element 1 ──► [Filter] ──► [Map] ──► Downstream (Continuous vertical stream; O(1) memory)

Stateful Pipeline Barrier (sorted, distinct):
Element 1 ──┐
Element 2 ──┼──► [ Internal Memory Buffer ] ──► (Barrier: awaits all elements before emitting)
Element 3 ──┘

Stateless Operations (filter, map, flatMap, peek):
- Each element is evaluated independently on the fly.
- Processing requires zero intermediate memory buffers ($O(1)$ memory).
Stateful Operations (distinct, sorted, limit, skip):
- distinct(): Maintains an internal hash set (LinkedHashSet or ConcurrentHashMap) of already observed elements. Requires $O(N)$ additional heap memory.
- sorted(): Acts as a full pipeline barrier. It cannot emit a single element downstream until it has read and buffered every single element from the source into an internal array and executed TimSort / Dual-Pivot Quicksort.
- limit(n) / skip(n): Track element counts via internal state registers and coordinate short-circuiting termination.

Stream Characteristic Optimizations (`StreamOpFlag`)

If the backing collection already exhibits the required characteristic, the Stream runtime optimizes the operation as a no-op (zero-cost pass-through):

import java.util.List;
import java.util.TreeSet;

public class StreamFlagsOptimizationDemo {

    public static void main(String[] args) {
        TreeSet<String> sortedSet = new TreeSet<>(List.of("C", "A", "B"));

        // TreeSet spliterator reports Spliterator.DISTINCT and Spliterator.SORTED:
        List<String> list = sortedSet.stream()
            .distinct() // 🚀 NO-OP: Collection is already guaranteed distinct
            .sorted()   // 🚀 NO-OP: Collection is already pre-sorted
            .toList();

        System.out.println(list); // [A, B, C]
    }
}

🔴 Senior Level

3 Critical Production Performance Traps

1. The Top-N Anti-Pattern: `sorted().limit(K)`

Sorting millions of records just to retrieve the top 5 is a major performance hazard:

// ❌ CRITICAL ANTI-PATTERN: Sorts all 1,000,000 elements in O(N log N)
List<User> top5 = users.stream()
    .sorted(Comparator.comparing(User::getScore).reversed())
    .limit(5)
    .toList();

Senior Solution: When $K \ll N$, use a bounded Min-Heap (PriorityQueue) of size $K$. This reduces time complexity from $O(N \log N)$ to $O(N \log K)$ and heap memory consumption from $O(N)$ down to $O(K)$.

2. Infinite Stream Hangs

// 💥 DISASTER: OutOfMemoryError / Infinite Thread Hang!
Stream.iterate(0, i -> i + 1)
    .sorted()  // Pipeline barrier hangs forever trying to buffer infinity!
    .limit(10)
    .toList();

// ✅ PROPER ARCHITECTURE: limit() MUST precede the sorted() barrier:
Stream.iterate(0, i -> i + 1)
    .limit(10) // Bounds the infinite stream to 10 elements first!
    .sorted()
    .toList();

3. In-Memory Pagination Anti-Pattern: `skip(N).limit(M)`

Executing stream.skip(100_000).limit(20) forces the JVM to iterate and discard 100,000 objects in memory. Pagination must be delegated to the database engine (LIMIT/OFFSET or Keyset Pagination).

Behavior in `parallelStream()`

Stateful operations introduce heavy synchronization overhead in parallel mode:

distinct() in Parallel: Requires cross-thread deduplication via a thread-safe container or merging partial sets during tree combination.
sorted() in Parallel: Utilizes Arrays.parallelSort(), which only yields performance advantages on datasets exceeding $100{,}000$ elements.
limit() and skip() in Parallel: If the stream reports ORDERED, worker threads must buffer and coordinate to preserve encounter order. Adding .unordered() prior to limit() removes the ordering constraint, enabling threads to return the first $N$ elements completed by any worker and dramatically boosting throughput.

4 Tricky Questions

1. Why is `sorted()` referred to as a “full pipeline barrier”, and how does it affect Loop Fusion?

Answer: A sorting algorithm cannot determine which element should be emitted first until it has compared every element against all other elements in the dataset.

Consequently, sorted() breaks the continuous vertical flow (Loop Fusion) of the stream pipeline. It forces the runtime to stop processing downstream stages, buffer all upstream elements into an internal contiguous array, sort the array completely, and only then resume feeding sorted elements one by one into downstream Sink stages.

2. What occurs when executing `Stream.generate(Math::random).sorted().limit(5)`, and how do you resolve it?

Answer: The application hangs indefinitely with 100% CPU utilization on the executing core and eventually crashes with an OutOfMemoryError.

Because sorted() is a full pipeline barrier, it attempts to drain the entire upstream source into memory before yielding any elements to limit(5). Since Stream.generate() produces an infinite sequence, the internal buffer expands until heap memory is exhausted. To resolve this, place .limit(5) before .sorted(), transforming the stream into a bounded 5-element pipeline before it enters the sort barrier.

3. How does `distinct()` establish element uniqueness, and what happens if `equals()` and `hashCode()` are mismatched?

Answer: distinct() stores seen elements in an internal hash-based set (LinkedHashSet sequentially, or a concurrent map in parallel mode). It relies on standard Java hashing semantics:

It hashes the element via hashCode() to find the target hash bucket.
It verifies equivalence via equals().

If a class overrides equals() without overriding hashCode(), logically equal objects may produce different hash codes and end up in different buckets. Consequently, distinct() will fail to filter them, producing duplicate objects in the final output.

4. Why is calling `.unordered()` before `.limit()` recommended in a `parallelStream()`?

Answer: When a stream is marked as ORDERED (e.g., derived from a List), limit(n) is contractually bound to return the first $n$ elements according to their original encounter order.

In a parallel stream, worker threads processing downstream chunks must synchronize with upstream chunks, buffering their results and discarding them if upstream workers produce enough elements. Calling .unordered() informs the stream engine that encounter order does not matter. The workers can emit whichever $n$ elements complete first without cross-thread coordination, significantly reducing thread contention.

🎯 Interview Cheat Sheet

Operational Characteristics Matrix

Key Takeaways

Stateful Intermediate: All four operations track internal state across multiple elements.
Pipeline Barrier: sorted() halts vertical loop fusion until all elements are buffered and sorted.
Infinite Streams: Never call sorted() on an infinite stream without first applying limit().
Stream Flag Elision: Pre-sorted or distinct collections (TreeSet) skip sorted() and distinct() as no-ops.

Red Flags & Anti-Patterns to Avoid

❌ Calling sorted() on an infinite stream before limit(): Triggers an infinite loop and crashes with OutOfMemoryError.
❌ Using sorted().limit(K) for small $K$ on massive datasets: Wasteful $O(N \log N)$ computation; use a PriorityQueue ($O(N \log K)$).
❌ Using distinct() on classes that do not override hashCode(): Silently fails to filter logical duplicates.
❌ Paginating large database tables via stream.skip().limit(): Pulls entire result sets into heap; delegate to SQL LIMIT/OFFSET.