When your application needs to do many things at once, handling it properly is what separates a service that scales from one that buckles under pressure. I’ve spent a lot of time tuning systems to handle thousands of operations per second, and the right approach to concurrency is everything. It’s not just about using threads; it’s about structuring your code so those threads work together efficiently without corrupting data or wasting resources.

Let’s talk about how to do that.

The most basic way to run tasks in parallel is to create threads manually. But if you’ve ever tried that for more than a trivial example, you know it gets messy fast. Creating a thread is expensive, and managing hundreds of them by hand leads to chaos. This is where the ExecutorService comes in. Think of it as a manager for a team of worker threads.

You tell the manager, “Here’s a job to do,” and it assigns it to an available worker. If all workers are busy, it can queue the job. You don’t need to worry about the details.

// Create a manager with a fixed team of 4 workers
ExecutorService executor = Executors.newFixedThreadPool(4);

// Prepare to track the jobs we submit
List<Future<Integer>> jobResults = new ArrayList<>();

for (int jobId = 0; jobId < 10; jobId++) {
    final int currentJobId = jobId;
    // Submit a job. This returns a 'Future' - a receipt for the result.
    Future<Integer> receipt = executor.submit(() -> {
        // Simulate a time-consuming calculation
        Thread.sleep(500);
        return currentJobId * 100;
    });
    jobResults.add(receipt);
}

// Now, collect all the results
int grandTotal = 0;
for (Future<Integer> receipt : jobResults) {
    // .get() waits for the worker to finish and gets the result
    grandTotal += receipt.get();
}

System.out.println("Total from all jobs: " + grandTotal);
executor.shutdown(); // Important: tell the manager the work is done.

Why is this better? The thread pool reuses workers. For 10 jobs, it only creates 4 threads. Without a pool, you’d create 10 threads, which is wasteful. For CPU-heavy work, a fixed pool size close to your processor core count is often best. For tasks that wait a lot on I/O, you might use a cached pool that can grow. The key is you let the framework handle the complexity.

Sometimes, you have a group of tasks that need to reach a checkpoint together before anyone moves on. Imagine a team of data processors. Each one must finish loading its chunk of data before any of them can start the analysis phase. You could try to coordinate this with flags and checks, but there’s a simpler tool: the CyclicBarrier.

A CyclicBarrier is like a meeting point in the code. You set it for, say, 3 threads. When the first two threads reach it, they wait. When the third one arrives, the barrier “trips,” and all three are released to continue. You can even run a quick action at that moment.

int teamSize = 3;
// Create a barrier for 3 threads, with a celebration task
CyclicBarrier meetingPoint = new CyclicBarrier(teamSize,
    () -> System.out.println("--- All data loaded! Starting phase 2. ---"));

Runnable worker = () -> {
    try {
        System.out.println(Thread.currentThread().getName() + ": Loading data...");
        Thread.sleep((long) (Math.random() * 2000)); // Simulate variable load time
        System.out.println(Thread.currentThread().getName() + ": Data ready. Waiting...");
        
        meetingPoint.await(); // Wait here for the other 2 workers
        
        System.out.println(Thread.currentThread().getName() + ": Starting analysis...");
        // Phase 2 work here...
        
    } catch (Exception e) {
        Thread.currentThread().interrupt();
    }
};

// Start the team
for (int i = 0; i < teamSize; i++) {
    new Thread(worker, "Worker-" + i).start();
}

The output will show all workers finishing their load phase at different times, but the “Starting analysis” messages will cluster together. This pattern is perfect for parallel algorithms that work in distinct stages.

Now, what if you have just two threads that need to hand off data, like a producer and a consumer? You could use a shared queue, but for a simple, tight handshake, the Exchanger is a fascinating tool.

It’s a synchronization point for exactly two threads. One arrives with an object (say, an empty container) and waits. The other arrives with its object (a full container). They swap and go on their way. It’s like two runners in a relay race passing a baton.

Exchanger<String> batonExchange = new Exchanger<>();

Runnable producer = () -> {
    String dataPackage = "Empty";
    try {
        for (int i = 1; i <= 3; i++) {
            Thread.sleep(1000); // Time to "produce" data
            dataPackage = "Data-" + i;
            System.out.println("Producer ready to pass: " + dataPackage);
            // Exchange the full package for an empty one
            dataPackage = batonExchange.exchange(dataPackage);
            System.out.println("Producer received back: " + dataPackage);
        }
    } catch (InterruptedException e) {
        Thread.currentThread().interrupt();
    }
};

Runnable consumer = () -> {
    String receivedPackage = "Empty";
    try {
        for (int i = 1; i <= 3; i++) {
            // Exchange the empty package for a full one
            receivedPackage = batonExchange.exchange(receivedPackage);
            System.out.println("Consumer received: " + receivedPackage);
            Thread.sleep(500); // Time to "consume" data
            receivedPackage = "Empty";
            System.out.println("Consumer returning empty package.");
        }
    } catch (InterruptedException e) {
        Thread.currentThread().interrupt();
    }
};

new Thread(producer).start();
new Thread(consumer).start();

This creates a very efficient, lock-free handoff when you have a pair of threads working in tandem. The exchange() call blocks until its partner thread is also ready, ensuring perfect coordination.

Caching is a universal performance tactic, but in a concurrent world, it’s a source of headaches. The classic problem: two threads find a cache miss for the same key at the same time. Both start the expensive load operation, duplicating work. You need a way to make the load happen only once.

The naive solution uses locks, which creates a bottleneck. A better way uses a ConcurrentHashMap and its computeIfAbsent method. This method is atomic. If the key is absent, it runs your loading function exactly once, even if other threads are requesting the same key simultaneously.

import java.util.concurrent.ConcurrentHashMap;
import java.util.function.Function;

public class SimpleCache<K, V> {
    private final ConcurrentHashMap<K, V> map = new ConcurrentHashMap<>();
    private final Function<K, V> loadingFunction;

    public SimpleCache(Function<K, V> loadingFunction) {
        this.loadingFunction = loadingFunction;
    }

    public V get(K key) {
        // This is the magic line. It's thread-safe.
        return map.computeIfAbsent(key, loadingFunction);
    }
}

// Let's use it
SimpleCache<String, BigDecimal> priceCache = new SimpleCache<>(productId -> {
    System.out.println("!! Loading price for " + productId + " from database...");
    // Simulate a slow database call
    try { Thread.sleep(2000); } catch (InterruptedException e) {}
    return new BigDecimal("99.99");
});

// Simulate multiple threads requesting the same product
Runnable client = () -> {
    BigDecimal price = priceCache.get("PROD_123");
    System.out.println(Thread.currentThread().getName() + " got price: " + price);
};

// Start 5 threads at nearly the same time
for (int i = 0; i < 5; i++) {
    new Thread(client, "Thread-" + i).start();
}

If you run this, you’ll likely see the “!! Loading…” message print only once, even though five threads called get. The other four threads wait and get the computed value. This pattern gives you thread-safe lazy initialization for free.

When you fire off many tasks and want to process their results as soon as they’re ready, you face a problem. If you submit 10 tasks and store their Futures in a list, processing them in order means you might wait for a slow task at the front while faster tasks at the back are already done.

A CompletionService solves this. It’s a wrapper around an ExecutorService that gives you a queue of completed tasks. You can take results in the order they finish, not the order you submitted them.

ExecutorService executor = Executors.newFixedThreadPool(3);
// Create a completion service that uses our executor
CompletionService<String> resultsService = new ExecutorCompletionService<>(executor);

List<String> workItems = List.of("A", "B", "C", "D", "E");

// Submit all tasks
for (String item : workItems) {
    resultsService.submit(() -> {
        // Simulate work with variable time
        int sleepTime = (int) (Math.random() * 3000);
        Thread.sleep(sleepTime);
        return "Processed " + item + " (took " + sleepTime + "ms)";
    });
}

// Collect results as they become available
for (int i = 0; i < workItems.size(); i++) {
    // .take() retrieves and removes the Future of the next completed task.
    // It waits if none are ready yet.
    Future<String> completedFuture = resultsService.take();
    String result = completedFuture.get(); // This will return immediately
    System.out.println("Result in: " + result);
}

executor.shutdown();

The output will show results in random order based on sleep time. This pattern maximizes throughput in batch processing pipelines because you’re never idle waiting for a specific task; you’re always working on the fastest-available result.

Java’s synchronized keyword is the most common way to protect shared data. But sometimes you need more flexibility. What if you want threads to wait for a specific condition, like “queue not full” or “queue not empty”? You can use Object.wait() and Object.notify(), but it’s easy to get wrong.

The ReentrantLock class, used with Condition objects, gives you this control explicitly and safely. It’s the tool you use to build your own blocking queues, resource pools, or any state-dependent coordination.

Let’s build a very simple bounded buffer, a fixed-size container for items shared between producers and consumers.

import java.util.concurrent.locks.Condition;
import java.util.concurrent.locks.ReentrantLock;

public class SimpleBoundedBuffer<T> {
    private final T[] buffer;
    private final ReentrantLock lock = new ReentrantLock();
    private final Condition spaceAvailable = lock.newCondition(); // "not full"
    private final Condition itemAvailable = lock.newCondition();  // "not empty"
    private int count = 0, putIndex = 0, takeIndex = 0;

    @SuppressWarnings("unchecked")
    public SimpleBoundedBuffer(int capacity) {
        buffer = (T[]) new Object[capacity];
    }

    public void put(T item) throws InterruptedException {
        lock.lock();
        try {
            while (count == buffer.length) {
                // Buffer is full. Wait for space.
                spaceAvailable.await();
            }
            buffer[putIndex] = item;
            putIndex = (putIndex + 1) % buffer.length;
            count++;
            // Signal that an item is now available for takers
            itemAvailable.signal();
        } finally {
            lock.unlock(); // Always unlock in a finally block
        }
    }

    public T take() throws InterruptedException {
        lock.lock();
        try {
            while (count == 0) {
                // Buffer is empty. Wait for an item.
                itemAvailable.await();
            }
            T item = buffer[takeIndex];
            buffer[takeIndex] = null; // Help garbage collection
            takeIndex = (takeIndex + 1) % buffer.length;
            count--;
            // Signal that space is now available for putters
            spaceAvailable.signal();
            return item;
        } finally {
            lock.unlock();
        }
    }
}

The pattern is clear: lock, check a condition in a while loop (not an if—this is crucial to avoid spurious wakeups), and await on the appropriate Condition if it’s not met. After changing the state, signal the other Condition. This is the blueprint for many thread-safe classes.

For tasks that need to run on a schedule—like sending a heartbeat, refreshing a cache, or polling an API—the Timer class is an old choice. It has significant drawbacks, like using only a single thread. If one task takes too long, others get delayed.

The ScheduledExecutorService is the modern replacement. It uses a thread pool, so long-running tasks don’t block others, and it handles task exceptions much more gracefully.

ScheduledExecutorService scheduler = Executors.newScheduledThreadPool(2);

// Schedule a one-time task for 5 seconds from now
scheduler.schedule(() -> {
    System.out.println("This runs once after a delay.");
}, 5, TimeUnit.SECONDS);

// Schedule a task to run every 2 seconds, starting in 1 second.
// This is fixed-rate: it tries to maintain the rate regardless of execution time.
ScheduledFuture<?> heartbeatFuture = scheduler.scheduleAtFixedRate(() -> {
    System.out.println("Heartbeat at " + System.currentTimeMillis());
}, 1, 2, TimeUnit.SECONDS);

// Schedule a task with a fixed delay BETWEEN the end of one execution and the start of the next.
ScheduledFuture<?> pollFuture = scheduler.scheduleWithFixedDelay(() -> {
    System.out.println("Starting slow poll...");
    try { Thread.sleep(3000); } catch (InterruptedException e) {} // Slow task
    System.out.println("Poll finished.");
}, 0, 2, TimeUnit.SECONDS); // Waits 2 seconds AFTER the previous execution finishes

// To cancel a repeating task later
// scheduler.schedule(() -> heartbeatFuture.cancel(false), 30, TimeUnit.SECONDS);

Use scheduleAtFixedRate for tasks where maintaining a strict frequency is critical, like updating a display. Use scheduleWithFixedDelay when you need a guaranteed cool-down period between executions, like calling an external API where you want to avoid overloading it.

Sometimes you need to limit how many threads can access a resource at the same time. Imagine you have a pool of 5 database connections, or you’re allowed only 10 concurrent calls to a payment gateway. A Semaphore is like a bouncer at a club with a limited capacity.

It holds a number of permits. To access the resource, a thread must acquire() a permit. If none are available, it waits. When done, the thread release()es the permit back.

public class SimpleConnectionLimiter {
    private final Semaphore permits;

    public SimpleConnectionLimiter(int maxConcurrentConnections) {
        // Initialize the semaphore with the total number of permits.
        this.permits = new Semaphore(maxConcurrentConnections);
    }

    public void doWorkWithResource() throws InterruptedException {
        permits.acquire(); // Wait for a permit to become available
        try {
            // This section now has controlled concurrency
            System.out.println(Thread.currentThread().getName() + " acquired a permit. " +
                               "Permits left: " + permits.availablePermits());
            // Simulate using the scarce resource
            Thread.sleep(2000);
        } finally {
            permits.release(); // Always release the permit in a finally block
            System.out.println(Thread.currentThread().getName() + " released a permit.");
        }
    }
}

// Simulate 8 threads trying to use a resource limited to 3 concurrent accesses
SimpleConnectionLimiter limiter = new SimpleConnectionLimiter(3);

for (int i = 0; i < 8; i++) {
    new Thread(() -> {
        try {
            limiter.doWorkWithResource();
        } catch (InterruptedException e) {
            Thread.currentThread().interrupt();
        }
    }, "Thread-" + i).start();
}

You’ll see that only 3 threads are in the “acquired a permit” section at any time. The others wait. A semaphore with one permit acts like a simple lock. With more permits, it’s a powerful throttling mechanism.

For problems that can be broken down recursively—like sorting a huge array, searching a directory tree, or calculating a sum of millions of numbers—the ForkJoinPool is a specialized tool. It’s designed for “divide and conquer.”

The key is the ForkJoinTask, usually a RecursiveTask (which returns a result) or a RecursiveAction (which doesn’t). The task splits its work into subtasks and fork()s them. It later join()s to collect results. The pool uses “work-stealing”: idle threads can steal tasks from the backs of other threads’ queues, which keeps all CPUs busy.

import java.util.concurrent.ForkJoinPool;
import java.util.concurrent.RecursiveTask;
import java.util.Random;

public class FindMaxTask extends RecursiveTask<Integer> {
    private final int[] array;
    private final int start, end;
    private static final int THRESHOLD = 10_000; // Don't split if smaller than this

    FindMaxTask(int[] array, int start, int end) {
        this.array = array;
        this.start = start;
        this.end = end;
    }

    @Override
    protected Integer compute() {
        // If the chunk is small enough, solve it directly.
        if (end - start <= THRESHOLD) {
            int max = Integer.MIN_VALUE;
            for (int i = start; i < end; i++) {
                if (array[i] > max) {
                    max = array[i];
                }
            }
            return max;
        } else {
            // Split the task into two subtasks.
            int middle = start + (end - start) / 2;
            FindMaxTask leftTask = new FindMaxTask(array, start, middle);
            FindMaxTask rightTask = new FindMaxTask(array, middle, end);

            // Fork the left task to run asynchronously.
            leftTask.fork();
            // Compute the right task in this thread (could also fork it).
            int rightResult = rightTask.compute();
            // Wait for the left task's result.
            int leftResult = leftTask.join();

            // Combine results
            return Math.max(leftResult, rightResult);
        }
    }
}

// Usage
int[] hugeArray = new int[10_000_000];
Random rand = new Random();
for (int i = 0; i < hugeArray.length; i++) {
    hugeArray[i] = rand.nextInt();
}

ForkJoinPool pool = new ForkJoinPool();
FindMaxTask mainTask = new FindMaxTask(hugeArray, 0, hugeArray.length);
int overallMax = pool.invoke(mainTask);
System.out.println("Maximum value found: " + overallMax);

The fork()/join() pattern lets the pool dynamically balance the load. This framework is highly efficient for parallelizing recursive algorithms.

Finally, let’s look at the most advanced pattern: writing lock-free code using atomic variables. Traditional locking blocks a thread if another holds the lock. A lock-free algorithm uses low-level processor instructions (like Compare-And-Swap, or CAS) to update shared state without blocking.

In Java, classes like AtomicInteger, AtomicReference, and AtomicLong expose this CAS capability. The most common pattern is a loop: read the current value, calculate a new value, and attempt to swap them only if the original value hasn’t changed in the meantime. If it has changed, you loop and try again.

This is how you might build a simple, thread-safe, non-blocking stack.

import java.util.concurrent.atomic.AtomicReference;

public class LockFreeStack<T> {
    private static class Node<T> {
        final T value;
        Node<T> next;

        Node(T value) {
            this.value = value;
        }
    }

    // The 'top' of the stack is an atomic reference to a Node.
    private final AtomicReference<Node<T>> top = new AtomicReference<>();

    public void push(T item) {
        Node<T> newHead = new Node<>(item);
        Node<T> oldHead;
        do {
            oldHead = top.get();          // 1. Read the current top
            newHead.next = oldHead;       // 2. Point new node to old top
        } while (!top.compareAndSet(oldHead, newHead)); // 3. CAS: swap if top hasn't changed
    }

    public T pop() {
        Node<T> oldHead;
        Node<T> newHead;
        do {
            oldHead = top.get();          // 1. Read the current top
            if (oldHead == null) {
                return null;               // Stack is empty
            }
            newHead = oldHead.next;       // 2. Find the next node
        } while (!top.compareAndSet(oldHead, newHead)); // 3. CAS: remove oldHead if top hasn't changed
        return oldHead.value;
    }
}

The compareAndSet method is the key. It says, “If the current value of top is exactly the oldHead I read, then set it to newHead. Otherwise, tell me I failed.” In the failure case, another thread modified the stack between our read and our write, so we simply retry with the new oldHead. This creates a system where threads help each other make progress instead of blocking each other.

These patterns are the building blocks. In practice, you’ll often combine them. You might use a ForkJoinPool for computation, a CompletionService to gather results, and an AtomicReference to update a shared status flag. The goal is always the same: write code that is correct under any timing of threads, and that uses your hardware resources efficiently.

Start with the high-level tools like ExecutorService. Master the coordination tools like CyclicBarrier and Semaphore. Use explicit locks when you need fine-grained conditions. Reserve lock-free programming for the hottest parts of your code where contention is truly high. Understanding these patterns gives you a vocabulary to design systems that are not just fast, but reliably fast under real-world load.