From implicit magic to explicit code

In this series we've built a tool to help us move processing into the background and made it flexible enough to easily use in more that one situation and made sure it was easy to use without forgetting something important.

If you haven't read through the series yet, start with "How I fixed my biggest mistake with implementing background jobs"

In the last article, we made sure than when initializing an object to use our ProcessLater module, we would store the appropriate data in initializer_arguments when it is initialized. This allows us to send that data to the background so it can do the job of initialization when it needs.

Our final code included parts which provided a hijack of the new method on the class using ProcessLater:

module ProcessLater
  def self.included(klass)
    # ...omitted code...
    klass.extend(Initializer)
  end

  module Initializer
    def new(*args)
      instance = allocate
      instance.instance_variable_set(:@initializer_arguments, args)
      instance.send(:initialize, *args.flatten)
      instance
    end
  end
end

This allowed us to keep our object initialization simple:

class SomeProcess
  include ProcessLater

  def initialize(some_id)
    @some_id = some_id
  end
end

SomeProcess.new(1).later(:call)

Hijacking the new method might feel strange. It's certainly unusual and it's done implicitly. When you use this ProcessLater module, you may not be aware that this magically happens for you.

Building clear and explicit methods

We can make our code more explicit as well as provide an easy interface for other developers to use this library.

I wrote about solving problems like this and knowing how to build your own tools in the Ruby DSL Handbook.

The following is a chapter from the Ruby DSL Handbook called Creating a Custom Initializer.

This has some ideas about how we can build a method which would provide a clear and explicit initializer. Take a deep dive in to understanding solving this problem with some metaprogramming techniques. Afterward, I'll wrap it up and show how it ties into the ProcessLater module that we've been building for background jobs.

---

Creating a custom initializer

Common and repetitive tasks are ripe for moving into a DSL and often Ruby developers find themselves wanting to take care of initialization and setting of accessor methods.

The following example is a modified version of a custom initializer from the Surrounded project.

The goal of the custom initializer is to allow developers to simplify or shorten code like this:

class Employment
  attr_reader :employee, :boss
  private :employee, :boss
  def initialize(employee, boss)
    @employee = employee
    @boss = boss
  end
end

The above sample creates attr_reader methods for employee and boss. Then it makes those methods private, and next defines an initializer method which takes the same named arguments and assigns them to instance variables of the same name.

This code is verbose and the repetition of the same words makes it harder to understand what's going on at a glance. If we understand the idea that we want to define an initializer which also defines private accessor methods, we can boil that down to a simple DSL.

This is far easier to type and easier to remember all the required parts:

class Employment
  initialize :employee, :boss
end

There is one restriction. We can't provide arguments like a typical method. If we tried this, it would fail:

initialize employee, boss
  # or
  initialize(employee, boss)

Ruby will process that code and expect to find employee and boss methods which, of course, don't exist. We need to provide names for what will be used to define arguments and methods. So we need to stick with symbols or strings.

Let's look at how to make that work.

Our first step is to define the class-level initialize method.

class Employment
  def self.initialize()

  end
end

Because we're creating a pattern that we can follow in multiple places, we'll want to move this to a module.

module CustomInitializer
  def initialize()
  end
end

class Employment
  extend CustomInitializer
end

Now we're setup to use the custom initializer and we can use it in multiple classes.

Because we intend to use this pattern in multiple places, we want the class-level initialize method to accept any number of arguments. To do that we can easily use the splat operator: *. Placing the splat operator at the beginning of a named parameter will treat it as handling zero or more arguments. The parameter *setup_args will allow however many arguments we provide.

The next step is to take those same arguments and set them as attr_readers and make them private.

module CustomInitializer
  def initialize(*setup_args)
    attr_reader(*setup_args)
    private(*setup_args)

  end
end

With that change, we have the minor details out of the way and can move on to the heavy lifting.

As we saw in Chapter 2: Structure With Modules we want to define any generated methods on a module to preserve some flexability for later alterations. We only initialize Ruby objects once; since we're defining the initialize method in a special module, it doesn't make sense for us to check to see if the module already exists. All we need to do is create it and include it:

module CustomInitializer
  def initialize(*setup_args)
    attr_reader(*setup_args)
    private(*setup_args)

    initializer_module = Module.new
    line = __LINE__; method_module.class_eval %{

    }, __FILE__, line
    const_set('Initializer', initializer_module)
    include initializer_module
  end
end

After we set the private attribute readers, we created a module with Module.new. We prepared the lines to evaluate the code we want to generate, and then we gave the module a name with const_set. Finally we included the module.

The last step is to define our initialize instance method, but this is tricky. At first glance it might seem that all we want to do is create a simple method definition in the evaluated string:

line = __LINE__; method_module.class_eval %{
    def initialize(*args)

    end
  }, __FILE__, line

This won't work the way we want it to. Remember that we are specifying particular names to be used for the arguments to this generated method in our class-level initialize using employee and boss as provided by our *setup_args.

The change in scope for these values can get confusing. So let's step back and look at what we want to generate.

In our end result, this is what we want:

def initialize(employee, boss)
    @employee = employee
    @boss = boss
  end

Our CustomInitializer is merely generating a string to be evaluated as Ruby. So we need only to look at our desired code as a generated string. With the surrounding code stripped away, here's what we can do:

%{
    def initialize(#{setup_args.join(',')})
      #{setup_args.map do |arg|
        ['@',arg,' = ',arg].join
      end.join("\n")}
    end
  }

The setup_args.join(',') will create the string "employee, boss" so the first line will appear as we expect:

def initialize(employee,boss)

Next, we use map to loop through the provided arguments and for each one we complile a string which consists of "@", the name of the argument, " = ", and the name of the argument.

So this:

['@',arg,' = ',arg].join

Becomes this:

@employee = employee

Because we are creating individual strings in our map block, we join the result with a newline character to put each one on it's own line.

%{

    #{setup_args.map do |arg|

    end.join("\n")}

  }

Here's our final custom initializer all the pieces assembled:

module CustomInitializer
  def initialize(*setup_args)
    attr_reader(*setup_args)
    private(*setup_args)

    mod = Module.new
    line = __LINE__; method_module.class_eval %{
      def initialize(#{setup_args.join(',')})
        #{setup_args.map do |arg|
          ['@',arg,' = ',arg].join
        end.join("\n")}
      end
    }, __FILE__, line
    const_set('Initializer', mod)
    include mod
  end
end

---

Custom initializer with our custom tool

With the techniques from this Ruby DSL Handbook chapter, we can have our ProcessLater module provide an initialize method which can handle the dependencies we need for the background work, as well as be a warning sign to developers that something different is going on.

Here's an alternative to our original solution which hijacked the new method.

module ProcessLater
  # ...omitted code...

  def self.included(klass)
    # ...omitted code...

    # add the initializer
    klass.extend(CustomInitializer)
  end

  class Later < Que::Job
    # ... omitted code...
    def run(*args)
      options = args.pop

      # Arguments changed from just "args" to "*args"
      self.class_to_run.new(*args).send(options['trigger_method'])
    end
  end

  module CustomInitializer
    def initialize(*setup_args)
      attr_reader(*setup_args)

      mod = Module.new
      line = __LINE__; mod.class_eval %{
        def initialize(#{setup_args.join(',')})
          #{setup_args.map do |arg|
            ['@',arg,' = ',arg].join
          end.join("\n")}

          @initializer_arguments = [#{setup_args.join(',')}]
        end
      }, __FILE__, line
      const_set('Initializer', mod)
      include mod
    end
  end
end

This highlights changes since the new hijack approach and with these changes we'll be able to use our new initialize method:

class SomeProcess
  include ProcessLater

  initialize :some_id

  def call
    puts "#{self.class} ran #{__method__} with #{some_id}"
  end
end

Now we can see explicit use of initialize.

This gives us an alternative approach that may do a better job of communicating with other developers about what this object needs to initialize.

Building better solutions with your team in mind

Ruby gives us a lot of power to make decisions not only about how the code works, but how we want to understand it when we use it.

The code we've created supports our desire to stay focused on an individual task. We can decide to run code now or run it later without the need to build an intermediary background class in a way that keeps our code cohesive with closely related features tied together.

We've altered the code be flexible enough to run in multiple places. Once we've made it work with one class, we were able to use it on any other class without the burden of having to rethink the solution of using a background job.

Finally, we made the code so that it would ensure that developers would not forget an important dependency. Our solution first had an interception of new with no extra work for the developer and we later balanced that decision and rethought it to provide some explicit indicators to future developers about how this code works.

As we work together to build software, we communicate in unspoken way through the structure of our code. We can push each other to make good or bad decisions and we can even make the next developer to come along feel powerful when using our code.

This is the reason I wrote the Ruby DSL Handbook.

There are many ways to approach the challenges we face in our code. Knowing how to build and use our own tools and how to help others repeat our good work with ease can make a team work more efficiently. With Ruby, we can build a language around what we think and say and do that helps to guide our software development.

As you go forward solving problems with your team, consider the many ideas that each team member can bring to the code. When you decide to build your tool in a certain way, what does that do to the alternatives?

I'd love to know how you build your tools and what sorts of decisions you have made. Drop me a note especially if you've picked up the Ruby DSL Handbook and read through it, watched the videos, and experimented or applied any ideas or techniques.

If you haven't yet, grab a copy of the Ruby DSL Handbook and bend Ruby to your will... or maybe just build tools that make you and your team happy to have a solid understanding of your code.