When I first began implementing background jobs I found myself moving my code into an appropriate background class whether it was in app or lib somewhere. I found it frustrating that I needed to shift code around based only on the decision to run it in the background. It seemed to be the conventional wisdom to do this, or at least that's what I thought.

It's not uncommon to find a reason to move some of your application into the background.

We build up systems that do a lot of work and at a certain point find that the work to be done takes too long. This often means that we reach for a new background job class, move our code to it, stick it in a place like app/jobs and then we're done with it.

Our code now lives in the background jobs and we move along to the next feature. So we separate app/models and app/jobs.

But this ends up feeling like I have two applications. I have my user facing application where most of my work happens and the one where things must happen in the background. But this is mostly a false dichotomy. That's not really how it works nor how any of us think about the system.

While it is important that background processing happen when necessary, I'd rather make those decisions as I determine them. I'd rather those descisions not require I rearrange my code like that.

Make decisions about background processing without rearranging your code

Here's an example of being able to make those decisions as you determine them. Developers using Rails may be familiar with this easy to change aspect of ActionMailer:

Notifier.welcome(some_user).deliver_now
# or with a minor change to the code...
Notifier.welcome(some_user).deliver_later

This feature is built-in and ready to use. Few things are easier to change than altering now now into later.

We can do the same in our code too.

To be able to handle this same approach, we'd need to create objects which can be initialized with data from your background data store.

A specialized class to write our object data to the background data store can do the job well, and it's pretty easy to do.

Here's a quick example.

Let's say you need to run some process.

class SomeProcess
  def initialize(some_id)
    @object = User.find(some_id)
  end

  def call
    # perform some long-running action (use your imagination)
  end
end
process = SomeProcess.new('ee6f1d66-b4e5-11e6-80f5-76304dec7eb7')
process.call

But you've implemented a long running process that runs as soon as some user requests it. If you follow the example from ActionMailer, maybe could just change process.call to process.call_later.

I want to have the ability to make decisions as easy as this when I need to run the Process code. When you keep all related code together, it’s easier to understand and make changes.

To make this work, we'll need that class to have a call_later method. We might have other methods we want to be able to run later too. Implementing method_missing can make this work...

class SomeProcess
  def method_missing(method_name, *args, &block)
    if method_name.to_s =~ /_later\z/
      # run it in the background
    else
      super # do the normal thing ruby does
    end
  end
end

The above implementation of method_missing will catch any methods ending in _later, but I'd rather not do that.

Using method_missing hides the implementation of our hook into the world of background jobs. It's hard to know that it is there and will probably difficult to find later when you want to understand how it works.

Instead, I'm going to write some code so that I can run my code in the backgound by using later(:call) instead of call. It might not be as elegant as appending _later to a method name, but the implementation easier is to get going and will put the code in a place where you can more easily find it.

Saving for later

So here's where we'll start:

class SomeProcess
  def later(trigger_method)
    # ...now what?
  end
end

We've made a later method that will accept a single argument of some method that we want to run.

That's easy enough but now we need to actually save this to the background. Usually this involves referring to some background job class and saving it.

We need to create a class that will save our object information to the background data store, and then later initialize the object and run our trigger_method.

Writing the data to your background store will be handled by whatever library you use for managing background jobs. This example will use Que but the differences with yours won't matter much.

Our backgroud class needs to initialize the SomeProcess object and tell it to run the trigger_method.

There's a trick to doing this. Our backgroud class needs to know what attributes are required to initialize the object, and which one to use as the method we're calling.

First, let's make a minor change to our initializer to store the argument we're given:

class SomeProcess
  def initialize(some_id)
    @some_id = some_id # <-- keeping track of the argument
    @object = User.find(some_id)
  end
  attr_reader :some_id
end

This change allows us to reference the argument we're given so that we can use it when we enqueue our background job. Rails allows us to pass an object into an ActiveJob instance (which we're not using here) and make it's best guess about how to serialize and deserialize the data to initialize our objects. Given our simple example here, we don't really need that feature (but we could implement the same if we like).

We really only need the class loaded by the background process to be a thin mediator between the data in the job store and the class which defines our business process. So I just make the class as small and isolated as possible.

class SomeProcess
  class Later < Que::Job
    def run(*args)
      options = args.pop # get the hash passed to enqueue
      ::SomeProcess.new(args).send(options['trigger_method'])
    end
  end

  def later(which_method)
    Later.enqueue(some_id, 'trigger_method' => which_method)
  end
end

There's no need to make a new thing inside of app/jobs (or anywhere else) since we never directly access this Later class.

If my main purpose in the code is to initialize Process and use the call method. The only decision I need to make is to either run it immediately, or run it later.

Once I began organizing my code with small and focused background classes, I was able to push aside the concern of when it would run until I needed to make that decision. Reading my code left my head clearer when I kept it all together.

With this code, I can make that decision as I determine the need for it.

Here's the final code for the class:

class SomeProcess
  class Later < Que::Job
    def run(*args)
      options = args.pop # get the hash passed to enqueue
      ::SomeProcess.new(args).send(options['trigger_method'])
    end
  end

  def initialize(some_id)
    @some_id = some_id
    @object = User.find(some_id)
  end
  attr_reader :some_id

  def later(which_method)
    Later.enqueue(some_id, 'trigger_method' => which_method)
  end

  def call
    # perform some long-running action
  end
end

There are some other features we could add to this, but this small class and later method get us where we want to be.

Now our decision to run this immediately or at a later time is as simple as changing this:

process.call

to this:

process.later(:call)

Are the objects in your system able to change like this? How do you handle decisions to move actions into the background?

Check the next article in the series: Turning a specific solution into a general tool