Background job processing is one of those concepts that seems simple until you actually need to implement it. When a user clicks a button, they shouldn’t have to wait for minutes while your application sends 10,000 emails or generates a complex report. That’s where moving work out of the immediate request comes in. I want to walk you through several ways to handle this, from the straightforward to the more complex, showing you the code that makes it work.

Let’s start with a basic truth: your web server is designed to handle requests and return responses quickly. It’s not built for long, tedious tasks. A background job system lets you say, “I need to do this important thing, but do it later, on your own time.” This keeps your application snappy for the user.

Think of it like a kitchen in a restaurant. The waiter (your web server) takes the order and immediately gives the customer their drinks. Then, they hand the food ticket (the job) to the chefs in the kitchen (your background workers). The customer isn’t staring at an empty table while their steak is cooked.

One of the most common tools for this in the Ruby world is Sidekiq. It’s powerful and efficient. Instead of spinning up a whole new process for each job, it uses threads within a single process, managed by a Redis database to keep track of what needs to be done.

Here’s what a typical job looks like. Imagine you need to send a welcome email to a new user. Doing that during the sign-up request could add a couple of seconds of delay. Instead, we hand it off.

# app/jobs/welcome_email_job.rb
class WelcomeEmailJob
  include Sidekiq::Job
  # This job goes to the 'mailers' line and will try 3 times if it fails.
  sidekiq_options queue: 'mailers', retry: 3

  def perform(user_id)
    # Find the user from the ID we were given.
    user = User.find(user_id)

    # This is the actual time-consuming work.
    UserMailer.welcome(user).deliver_now

    # Maybe we also want to log that we sent it.
    Rails.logger.info "Welcome email sent to #{user.email}"
  end
end

To use this job, you don’t call it directly. You tell Sidekiq to line it up.

# In your controller, after a user signs up:
def create
  @user = User.new(user_params)
  if @user.save
    # This is the magic line. It returns instantly.
    WelcomeEmailJob.perform_async(@user.id)
    redirect_to root_path, notice: 'Account created!'
  else
    render :new
  end
end

You can also schedule jobs for the future. Need to send a reminder in 24 hours? It’s just as simple.

# Schedule a reminder for one day from now.
ReminderEmailJob.perform_in(1.day, @user.id)

Now, Sidekiq is a specific tool. Rails, understanding that there are many such tools, created a common interface called Active Job. It’s like a universal remote. You write your job once using Active Job, and you can configure it to use Sidekiq, Delayed Job, or another system behind the scenes. This is very helpful if you think you might change your queue system later.

An Active Job looks quite similar but with some extra features baked in.

# app/jobs/image_processing_job.rb
class ImageProcessingJob < ApplicationJob
  # Use the default queue.
  queue_as :default

  # Automatically retry if a network error happens, waiting longer between each try.
  retry_on NetworkError, attempts: 5, wait: :exponentially_longer

  def perform(image_id)
    image = Image.find(image_id)

    # Create different sized versions of the image.
    image.file.variant(resize_to_limit: [800, 800]).processed
    image.file.variant(resize_to_limit: [200, 200]).processed

    # Mark the image as ready.
    image.update(processed_at: Time.current)
  end

  # You can run code if the job fails completely.
  def discard(error)
    # Maybe notify an admin about a persistent problem.
    AdminMailer.job_failed(self, error).deliver_later
  end
end

You enqueue it with perform_later.

ImageProcessingJob.perform_later(@image.id)

What if you don’t want to set up Redis or another service? For smaller applications, you might use a tool like Delayed Job. Its big appeal is simplicity—it stores the jobs right in your application’s main database. No new infrastructure is needed. A job is just a record in a delayed_jobs table.

Here’s how you might use it for cleaning up old files.

# app/jobs/cleanup_job.rb
class CleanupJob
  def perform(directory_path)
    # Find all temporary files older than 1 day.
    old_files = Dir.glob(File.join(directory_path, "tmp_*")).select do |file|
      File.mtime(file) < 1.day.ago
    end

    # Delete each one.
    old_files.each { |file| File.delete(file) }
    Rails.logger.info "Cleaned up #{old_files.count} old files."
  end
end

To run it, you’d delay the method call.

# Run this job in the background.
CleanupJob.new.delay.perform('/uploads/tmp')

You can even delay methods on your existing models, which can be convenient.

# Delay sending a notification.
@comment.delay.notify_subscribers

Sometimes your tasks aren’t just isolated bits of Ruby code. You might need to communicate with other services written in different languages. This is where dedicated message queues like RabbitMQ shine. They act as a central post office, routing messages (our jobs) between different, independent applications.

In a Rails app, you might have a publisher that sends tasks.

# app/services/task_publisher.rb
class TaskPublisher
  def self.publish_export_task(user_id, format)
    # Connect to RabbitMQ.
    connection = Bunny.new
    connection.start

    channel = connection.create_channel
    # Declare an 'exchange' where we send messages.
    exchange = channel.direct('tasks')

    message = { user_id: user_id, format: format, requested_at: Time.current.to_i }

    # Publish the message with a specific routing key.
    exchange.publish(message.to_json, routing_key: 'data_export')
    Rails.logger.info "Published export task for user #{user_id}"

    connection.close
  end
end

Another separate program, perhaps written in Python or Go, could be listening on the data_export queue, processing the export, and placing the result file somewhere. Your Rails app doesn’t do the work; it just declares what work needs to be done.

As systems grow, you often find that a single job is too big. It’s better to break it down into a series of smaller, linked jobs. This is a job pipeline. It makes each step easier to understand, test, and retry if something goes wrong.

Consider processing a new video upload. The steps are clear: validate the file, transcode it to different resolutions, generate a thumbnail, and then notify the user.

# We can chain these jobs together.
class VideoProcessingPipeline
  def self.schedule(video_id)
    # `perform_now` runs in the foreground, but we can use it to chain.
    # In reality, you'd likely use job IDs to link them in the background.
    ValidateVideoJob.perform_now(video_id)
    TranscodeVideoJob.perform_now(video_id)
    GenerateThumbnailJob.perform_now(video_id)
    NotifyUserJob.perform_now(video_id)
  end
end

But what if the validation fails? You don’t want the other jobs to run. Modern job systems allow you to define dependencies. While the syntax varies, the idea is that each job, upon success, triggers the next one in the chain.

A crucial concept for making background jobs reliable is idempotency. An idempotent job is one you can run multiple times, and it will have the same end result as running it once. This is vital because jobs can be retried automatically. If sending an email isn’t idempotent, a network glitch could cause your system to send the same email five times to a user.

You design for idempotency. For example, when updating a calculated total, you should calculate from scratch each time, not just add a value.

class UpdateUserScoreJob
  include Sidekiq::Job

  def perform(user_id)
    user = User.find(user_id)
    # Re-calculate the total score from all activities.
    new_score = user.activities.sum(:points)
    # Setting the score is idempotent. Running this twice doesn't double it.
    user.update(total_score: new_score)
  end
end

For actions that absolutely must not be duplicated, like charging a credit card, you need a different strategy. You might store a unique transaction ID with the job and check your database to see if that transaction has already been processed before you do anything.

class ProcessPaymentJob
  include Sidekiq::Job

  def perform(order_id, unique_transaction_token)
    # Check our ledger first.
    return if Payment.exists?(transaction_token: unique_transaction_token)

    order = Order.find(order_id)
    # ... complex payment gateway logic ...

    # Record the transaction as done.
    Payment.create!(order: order, transaction_token: unique_transaction_token, amount: order.total)
  end
end

Finally, you can’t just set and forget a background job system. You need to watch it. How many jobs are waiting? Are any failing repeatedly? Is the queue getting too long?

Simple monitoring can be added right into your jobs.

class MonitoredJob < ApplicationJob
  def perform
    start_time = Time.current

    # ... do the actual work ...

    duration = Time.current - start_time
    # Send this metric to a monitoring service.
    StatsD.measure('job.monitored_job.duration', duration)

    if duration > 10.seconds
      Rails.logger.warn "MonitoredJob is slow! Took #{duration}s"
    end
  end
end

Many job systems also come with web dashboards. Sidekiq has one that shows you all your queues, how busy they are, and what jobs have failed. Checking this dashboard should become part of your daily routine, like checking your email.

You should also set up alerts. If the “mailers” queue has more than 10,000 jobs stuck in it, something is probably wrong, and you need to know immediately.

# A simple rake task to check queue health.
namespace :jobs do
  desc 'Check for stalled queues'
  task check_health: :environment do
    Sidekiq::Queue.all.each do |queue|
      if queue.size > 10000
        # Trigger a PagerDuty alert or send an email.
        AlertService.critical("Queue #{queue.name} is huge: #{queue.size} jobs")
      end
    end
  end
end

Choosing the right pattern starts with asking simple questions. How fast does the job need to run? What happens if it gets lost? Does it talk to other systems? For many Rails applications, starting with Active Job and a simple backend like Sidekiq or Delayed Job covers most needs. As complexity grows, you can introduce pipelines, message queues, and sophisticated monitoring.

The goal is always the same: to make things happen for the user without making them wait. By moving work to the background, you build an application that feels fast and responsive, even when it’s doing heavy lifting behind the scenes. Start simple, monitor everything, and break big problems into small, manageable jobs.