Ruby Concurrency Mastery: From Thread Pools to Ractors for High-Performance Applications

Let’s talk about making Ruby applications handle many tasks at once. When I first started working with Ruby, I thought concurrency was just about using threads. I quickly learned it’s more nuanced than that. Ruby, especially the standard MRI version, has a unique way of handling simultaneous operations because of something called the Global Interpreter Lock, or GIL. This means that, in MRI, only one thread can execute Ruby code at a time, even if you have multiple CPU cores. It sounds limiting, but it simplifies things by preventing a whole class of complicated bugs. However, it also means we have to be smart about how we manage work to keep our applications fast and responsive.

The goal isn’t just to run things in parallel; it’s to manage resources wisely, whether we’re waiting on a database, processing a file, or handling a web request. Over time, I’ve learned that different problems call for different tools. Let me walk you through some of the most effective ways I’ve found to manage this.

---

Imagine you have a hundred tasks to complete. Starting a new Ruby thread for each one seems logical, but creating threads isn’t free. Each one consumes memory, and switching between them adds overhead. A thread pool solves this by creating a team of workers upfront. This team waits for jobs to appear in a shared to-do list, a queue.

Here’s a basic version of a thread pool I might build. It controls how many workers exist and how many jobs can be waiting.

class SimplePool
  def initialize(size: 5)
    @size = size
    @queue = Queue.new
    @workers = []

    size.times do
      @workers << Thread.new do
        loop do
          job = @queue.pop
          break if job == :stop
          job.call
        end
      end
    end
  end

  def perform(&block)
    @queue << block
  end

  def shutdown
    @size.times { @queue << :stop }
    @workers.each(&:join)
  end
end

# Using it
pool = SimplePool.new(size: 3)

10.times do |i|
  pool.perform do
    sleep(1) # Simulating work
    puts "Job #{i} done by #{Thread.current.object_id}"
  end
end

sleep(4) # Let jobs finish
pool.shutdown

In this setup, only three threads ever exist. They work through the ten jobs. If all three are busy, new jobs simply wait in the queue. This prevents your system from being overwhelmed. For a production system, you’d add error handling, ways to gracefully stop, and perhaps a way to limit the queue size so it doesn’t grow forever. The key idea is resource management: you control the maximum impact on your system.

---

Sometimes, you need to manage thousands of lightweight tasks, like handling network connections for a chat server. Using a thread for each connection would be heavy. This is where Fibers come in. Think of a Fiber as a task that can pause itself and let another task run, all within a single thread. It’s cooperative, not preemptive. The task says, “I’m waiting for data now, someone else can go.”

This allows you to manage massive concurrency with very little overhead. Here’s a conceptual look at how a Fiber-based scheduler might work.

require 'socket'

# A very basic fiber scheduler for reads
scheduler = Thread.new do
  fibers_waiting_on_io = {}

  loop do
    # Check which sockets are ready
    readable_sockets, = IO.select(fibers_waiting_on_io.keys, [], [], 0.1)

    if readable_sockets
      readable_sockets.each do |socket|
        waiting_fiber = fibers_waiting_on_io.delete(socket)
        waiting_fiber.resume(socket) if waiting_fiber
      end
    end

    sleep 0.01
  end
end

# A method to 'await' a socket read
def await_read(socket)
  # This fiber yields control back to the scheduler thread
  Fiber.yield(socket)
  # When resumed, the socket should be ready to read
  socket.read_nonblock(1024, exception: false)
end

# In your main application fiber
Fiber.new do
  server = TCPServer.new(1234)
  puts "Server started on port 1234"

  loop do
    client = server.accept
    puts "Accepted connection"

    # Handle each client in its own fiber, but all in one thread
    Fiber.new do
      begin
        loop do
          # This will pause this fiber, not the whole thread
          data = await_read(client)
          break if data.nil? # Connection closed

          if data != :wait_readable
            client.puts "Echo: #{data}"
          end
        end
      ensure
        client.close
        puts "Connection closed"
      end
    end.resume
  end
end.resume

This is a simplified view. Ruby 3 introduced a more formal Fiber.scheduler interface. The power here is handling ten thousand connections in a single thread, because you’re only actively working on the ones that have data ready. It’s perfect for I/O-heavy applications.

---

For truly parallel execution—making full use of multiple CPU cores for heavy calculations—Ruby 3 introduced Ractors. A Ractor is like a separate mini-Ruby process with its own interpreter, but it lives within your main process. The critical rule is that Ractors cannot share mutable objects. They must pass copies of data or send messages. This “share-nothing” design prevents concurrency bugs by making them impossible.

Let’s say you need to process a large array of numbers with a complex calculation.

# Without Ractors (sequential)
data = (1..10_000).to_a
result = data.map { |n| Math.sqrt(n) * Math.log(n) }

# With Ractors (parallel)
def parallel_process(data, worker_count: 4)
  # Split the work
  chunks = data.each_slice((data.size / worker_count.to_f).ceil).to_a

  # Create Ractors, each working on a chunk
  ractors = chunks.map do |chunk|
    Ractor.new(chunk) do |sub_array|
      sub_array.map { |n| Math.sqrt(n) * Math.log(n) }
    end
  end

  # Collect and combine results
  ractors.flat_map(&:take)
end

result = parallel_process(data, worker_count: 4)

In the Ractor example, each chunk is processed in a truly parallel fashion on a separate CPU core, if available. The Ractor.new creates the isolated worker, and take waits for its result. Remember, the block you give to Ractor.new and the data you send it are isolated. You can’t directly modify a variable from the outside. This forces clean, safe design.

---

When threads do need to share some state, like a counter or a cache, you must coordinate access. The simplest way is a Mutex, a lock that ensures only one thread enters a piece of code at a time.

class SharedCounter
  def initialize
    @value = 0
    @mutex = Mutex.new
  end

  def increment
    @mutex.synchronize do
      @value += 1
    end
  end

  def value
    @mutex.synchronize do
      @value
    end
  end
end

counter = SharedCounter.new

threads = 10.times.map do
  Thread.new do
    1000.times { counter.increment }
  end
end

threads.each(&:join)
puts "Counter is: #{counter.value}" # Correctly 10000

Without the mutex.synchronize, two threads could read the old value (say, 5) at the same time, both add one, and both write back 6, losing one increment. This is a race condition. The mutex prevents it.

For more complex structures, like a hash shared across many threads, you can use finer-grained locking or thread-safe classes from the concurrent-ruby gem, which is an invaluable library for real-world applications.

require 'concurrent'

# A thread-safe map
cache = Concurrent::Map.new

# This is safe from race conditions
cache.compute_if_absent(:user_123) do
  fetch_user_from_database(123) # Expensive operation
end

The compute_if_absent method ensures the expensive fetch_user_from_database is called only once, even if a hundred threads ask for :user_123 at the same moment. It handles the locking internally.

---

The Reactor pattern is the engine behind high-performance network servers. At its core is an event loop that watches many I/O streams (sockets, files). When a stream becomes ready for reading or writing, the loop fires a callback. This means a single thread can manage thousands of network connections. Libraries like nio4r and servers like Falcon use this pattern.

While building a full reactor is complex, you can see the idea in this loop:

require 'socket'

connections = []
server = TCPServer.new('localhost', 3000)

loop do
  # Check for new connections AND ready sockets, wait up to 0.1 sec
  readable, = IO.select([server] + connections, [], [], 0.1)

  if readable
    readable.each do |socket|
      if socket == server
        # New client
        conn = server.accept
        connections << conn
        puts "New client connected."
      else
        # Existing client has data
        begin
          data = socket.read_nonblock(1024)
          if data
            socket.puts "You said: #{data.chomp}"
          end
        rescue IO::WaitReadable
          # Not enough data yet, skip for now
        rescue EOFError
          # Client disconnected
          connections.delete(socket)
          socket.close
          puts "Client disconnected."
        end
      end
    end
  end
end

This single loop accepts new clients and reads data from existing ones. It never blocks for long because read_nonblock returns immediately. It’s efficient because the thread is only busy when there’s actual data to process.

---

In a work-stealing system, each worker thread has its own queue of tasks. When a worker finishes its own tasks, it doesn’t sit idle. Instead, it looks at other workers’ queues and “steals” a task from the end. This is a fantastic way to balance load automatically.

You can find this pattern in parallel task frameworks. Here’s a conceptual sketch.

class Worker
  def initialize(id, all_queues)
    @id = id
    @my_queue = Queue.new
    @all_queues = all_queues
    @thread = Thread.new { run }
  end

  def run
    loop do
      task = get_task
      break if task == :stop
      task.call
    end
  end

  def get_task
    # 1. Try my own queue first
    return @my_queue.pop unless @my_queue.empty?

    # 2. Try to steal from others
    @all_queues.each do |q|
      next if q == @my_queue || q.empty?
      begin
        return q.pop(true) # non-blocking pop
      rescue ThreadError
        # Someone else stole it, try next
      end
    end

    # 3. Nothing to do, wait on my own queue
    @my_queue.pop
  end

  def add_task(task)
    @my_queue << task
  end

  def stop
    add_task(:stop)
    @thread.join
  end
end

# Create workers sharing access to each other's queues
queues = []
workers = 4.times.map do |i|
  Worker.new(i, queues).tap { |w| queues << w.instance_variable_get(:@my_queue) }
end

# Distribute tasks unevenly
workers[0].add_task(-> { puts "Heavy task A" })
workers[0].add_task(-> { puts "Heavy task B" })
# Workers 1, 2, 3 have no tasks initially

# The idle workers will steal from worker 0's queue
sleep(1)
workers.each(&:stop)

Even though we initially gave all tasks to worker 0, the idle workers will steal them, ensuring all CPU cores are utilized. This pattern is excellent for uneven, unpredictable workloads.

---

So, how do you choose? It depends on your problem. Ask yourself: Is my work mostly waiting on I/O (like database calls or HTTP requests)? Then a thread pool or Fiber-based concurrency is great. The thread pool is simpler and well-understood. Fibers offer higher scalability for specialized I/O-heavy apps.

Is my work pure, hard calculation? If you’re using MRI and the calculation is in Ruby code, threads won’t run in parallel due to the GIL. Here, you need Ractors to get true parallelism across CPU cores. Or, consider using JRuby or TruffleRuby, which don’t have a GIL, where threads can run Ruby code in parallel.

Are you building a high-performance network server like a web socket endpoint? Look closely at the Reactor pattern and libraries built on it.

Do you have a bag of many independent tasks? A work-stealing pool is often the most efficient.

The most common mistake I see is reaching for threads for everything. Start simple. Use a thread pool from concurrent-ruby for background jobs. Use a mutex to protect simple shared state. Profile your application. If you hit a limit, then explore the more advanced patterns. Concurrency is a tool, and the best tool is the one that makes your application correct, understandable, and fast enough for your needs. Ruby provides the options; it’s up to us to apply them wisely.