Using method_missing to customize SearchLogic

June 12th 2010 —   Comments and   Reactions

The Ruby interpreter calls method_missing on a Ruby object whenever it receives a message (method call) that it cannot handle. One of the best examples of using method_missing that I’ve come across is in the SearchLogic plugin, which allows you to dynamically create named scopes. Today I’m going to take some time to explain how method_missing works, show how it’s used by SearchLogic, and finally show how you can use method_missing yourself to customize SearchLogic’s behavior.

Simple sorting with SearchLogic

Suppose I have an ActiveRecord model called “book” with a “title” attribute:

$ rails books
$ cd books
$ script/generate model book title:string
$ rake db:migrate
$ script/console
>> ["one", "two", "three"].each { |title| Book.create :title => title }

The best way in Rails to display the books sorted by title would be to use a named scope like this in my model:

class Book < ActiveRecord::Base
  named_scope :sorted_by_title, { :order => 'title' }
end

If I use a trick my colleague Niranjan Sarade showed me, we can see the SQL produced by ActiveRecord for the named scope in the console, like this: 

$ script/console
>> ActiveRecord::Base.logger = Logger.new(STDOUT)
>> Book.sorted_by_title.collect { |book| book.title }
  Book Load (1.6ms)   SELECT * FROM "books" ORDER BY title
=> ["one", "three", "two"]

This is a good example of why I’m a Rails developer: with just a single line of code in my model I can sort the values in a database column! But it gets even easier if I install SearchLogic:

$ script/plugin install git://github.com/binarylogic/searchlogic.git

Now I get a whole series of named scopes created for me automatically! For example, I can now just call the “ascend_by_title” and “descend_by_title” named scopes as if I had written them myself:

$ script/console
>> Book.ascend_by_title.collect { |book| book.title }
  Book Load (1.3ms)   SELECT * FROM "books" ORDER BY books.title ASC
=> ["one", "three", "two"]
>> Book.descend_by_title.collect { |book| book.title }
  Book Load (2.0ms)   SELECT * FROM "books" ORDER BY books.title DESC
=> ["two", "three", "one"]

Brilliant! Using SearchLogic I can filter/sort on any attribute of any model in my application without writing a single line of code. I can even sort and filter on attributes of associated models, e.g. if I had “Book has_many :authors,” I could sort books by their author’s names, or sort the authors for each book, etc., all without writing any SQL or even Ruby code.

Sorting with NULL values last

Recently at my day job I came across a business requirement to sort a list of values, always displaying the NULL or empty values at the end of the list. In our example, this would mean that there might be some books with missing titles:

$ script/console
>> 2.times { Book.create :title => nil }

Here’s the behavior I get from the ascend_by_title and descend_by_title named scopes with NULL values:

>> Book.ascend_by_title.collect { |book| book.title }
  Book Load (2.5ms)   SELECT * FROM "books" ORDER BY books.title ASC
=> [nil, nil, "one", "three", "two"]
>> Book.descend_by_title.collect { |book| book.title }
  Book Load (2.7ms)   SELECT * FROM "books" ORDER BY books.title DESC
=> ["two", "three", "one", nil, nil]

In other words, the NULL values are considered to be less than the other values by the database server, and are sorted accordingly. To get the behavior I want, I need to use a slightly more complex sorting pattern in a named scope, like this:

class Book < ActiveRecord::Base
  named_scope :sorted_by_title_nulls_last,
              { :order => 'title IS NULL, title' }
  named_scope :sorted_by_title_nulls_last_desc,
              { :order => 'title IS NULL, title DESC' } 
end

Trying it out in the console:

$ script/console
>> Book.sorted_by_title_nulls_last.collect { |book| book.title }
  Book Load (2.6ms)   SELECT * FROM "books" ORDER BY title IS NULL, title
=> ["one", "three", "two", nil, nil]
>> Book.sorted_by_title_nulls_last_desc.collect { |book| book.title }
  Book Load (2.5ms)   SELECT * FROM "books" ORDER BY title IS NULL, title DESC
=> ["two", "three", "one", nil, nil]

These named scopes first sort on “title IS NULL” and then on the actual title value, causing the NULL values to appear at the end. This code is fairly clean and would work fine – the problem I had at my day job, however, was that I needed this sorting behavior for about six different columns in various database tables. To make this work, I would need to repeat these two named scopes in each model for each attribute that I wanted to sort on. If only SearchLogic had supported this sorting behavior, I wouldn’t need to copy and paste all of the named scopes.

Using method_missing

Specifically, here’s the method that I wished SearchLogic had implemented for me:

>> Book.ascend_by_title_nulls_last
undefined method `ascend_by_title_nulls_last' for #<Class:0x2234978>

As you can see it doesn’t. But I mentioned above that SearchLogic works by using method_missing; let’s see if I can use method_missing myself and implement the NULLs last behavior… in other words, let’s see if I can use metaprogramming to implement the “nulls last” named scopes on all of my model classes all at once!

I’ll start by using the simplest possible implementation of method_missing:

class Book < ActiveRecord::Base
  class << self
    def method_missing(name, *args, &block)
      puts "This method is missing: #{name}"
    end
  end
end

Here “class << self” indicates that method_missing will be a class method on my Book class; Ruby calls method_missing on the class that is missing the method. The code here simply writes out a message when an unknown method is called:

>> Book.ascend_by_title_nulls_last
This method is missing: ascend_by_title_nulls_last
=> nil

Now I’m ready to think about how to implement the nulls last sorting. But not so fast: it turns out that I have just broken my model class! Aside from SearchLogic, ActiveRecord itself also uses method_missing extensively. The simplest examples of this are the “find_by_...” methods. For example, calling find_by_title should return the book record with the given title:

>> Book.find_by_title 'one'
This method is missing: find_by_title

But now instead of Book “one” I just get the debug message from method_missing. The correct solution here is to pass along the method_missing call to the super class, like this:

class Book < ActiveRecord::Base
  class << self
    def method_missing(name, *args, &block)
      if name == :ascend_by_title_nulls_last
        puts "This method is missing: #{name}"
      else
        super
      end
    end
  end
end

Let’s try find_by_title again in the console:

>> Book.find_by_title 'one'
  Book Load (0.5ms)
  SELECT * FROM "books" WHERE ("books"."title" = 'one') LIMIT 1
=> #<Book id: 1, title: "one", created_at: "2010-06-11 18:39:26", etc...
>> Book.ascend_by_title_nulls_last
This method is missing: ascend_by_title_nulls_last
=> nil

Sigh of relief – it works again! Looking at the if statement above, you can see that I check if the missing method is called “ascend_by_title_nulls_last,” in which case I write the debug message; if any other missing method is called I pass the call along to the super class. In this case, the super class is actually the SearchLogic module; it uses method_missing with super in exactly the same way that I do here. If the missing method is not recognized by SearchLogic, super is called again and finally ActiveRecord receives the method_missing call, which eventually evaluates find_by_title.

How does SearchLogic work?

SearchLogic uses method_missing as follows, the first time a missing method is called on an ActiveRecord model:

  • Check if the unknown method matches a certain pattern, e.g. “ascend_by_XYZ.”
  • Call “named_scope” if so, to create the corresponding named scope.
  • Run the new named scope, to return the desired query results.

If the same missing method is called again, it will no longer be missing since the corresponding named scope will now exist. ActiveRecord caches a list of scopes that are created by calls to named_scope for each model class.

Ok – let’s try this idea on my Book model:

class Book < ActiveRecord::Base
  class << self
    def method_missing(name, *args, &block)
      if name == :ascend_by_title_nulls_last
        named_scope :ascend_by_title_nulls_last,
                    { :order => 'title IS NULL, title' }
        ascend_by_title_nulls_last
      else
        super
      end
    end
  end
end

In the console again:

>> Book.ascend_by_title_nulls_last.collect { |book| book.title }
  Book Load (1.9ms)   SELECT * FROM "books" ORDER BY title IS NULL, title
=> ["one", "three", "two", nil, nil]

It works! The code above implements SearchLogic’s algorithm: if someone tries to use a named scope called “ascend_by_title_nulls_last” then actually create the scope at that moment with the proper sorting behavior.

Adding custom sorting to SearchLogic

Now I’m ready to generalize this for any model and attribute. First, I’ll look for any missing method name that matches a certain regex pattern (“ascend_by_XYZ_nulls_last”):

class Book < ActiveRecord::Base
  class << self
    def method_missing(name, *args, &block)
      if name.to_s =~ /^ascend_by_(\w+)_nulls_last$/
        named_scope :ascend_by_title_nulls_last,
                    { :order => 'title IS NULL, title' }
        ascend_by_title_nulls_last
      else
        super
      end
    end
  end
end

The line highlighted above first converts the method name from a symbol to a string, and then matches it against the “nulls_last” syntax I’m looking for. Next, I’m still hard coding “title” in the named_scope call; let’s replace that with the proper value, and also use the name passed into method_missing instead of the hard coded symbol for the scope name:

class Book < ActiveRecord::Base
  class << self
    def method_missing(name, *args, &block)
      if name.to_s =~ /^ascend_by_(\w+)_nulls_last$/
        named_scope name, { :order => "#{$1} IS NULL, #{$1}" }
        ascend_by_title_nulls_last
      else
        super
      end
    end
  end
end

“$1” returns the string that matched the first expression contained in parentheses in the regex pattern above, “(\w+)” in this case. This will be the name of the attribute between ascend_by… and …nulls_last, taken from the missing method’s name. Now the proper named scope is created using this attribute name in the SQL fragment. So for example, if I call “Book.ascend_by_author_name_nulls_last” a named scope called “ascend_by_author_name_nulls_last” will be created, using :order => “author_name IS NULL, author_name.”

One last hard coded value to remove: the call to “ascend_by_title_nulls_last” still refers to title directly. To fix this, I just need to use “send(name)” – this calls the method whose name is in the “name” string, which is the named scope we just created. Here’s how that looks:

class Book < ActiveRecord::Base
  class << self
    def method_missing(name, *args, &block)
      if name.to_s =~ /^ascend_by_(\w+)_nulls_last$/
        named_scope name, { :order => "#{$1} IS NULL, #{$1}" }
        send(name)
      else
        super
      end
    end
  end
end

Now I can add in the case for the descending sort as well:

class Book < ActiveRecord::Base
  class << self
    def method_missing(name, *args, &block)
      if name.to_s =~ /^ascend_by_(\w+)_nulls_last$/
        named_scope name, { :order => "#{$1} IS NULL, #{$1}" }
        send(name)
      elsif name.to_s =~ /^descend_by_(\w+)_nulls_last$/
        named_scope name, { :order => "#{$1} IS NULL, #{$1} DESC" }
        send(name)
      else
        super
      end
    end
  end
end

The last thing I’ll do today is generalize this for any model in my application by moving the method_missing code into a module that I’ll call “SearchLogicExtensions,” and then extending ActiveRecord::Base with that:

module SearchLogicExtensions
  def method_missing(name, *args, &block)
    if name.to_s =~ /^ascend_by_(\w+)_nulls_last$/
      named_scope name, { :order => "#{$1} IS NULL, #{$1}" }
      send(name)
    elsif name.to_s =~ /^descend_by_(\w+)_nulls_last$/
      named_scope name, { :order => "#{$1} IS NULL, #{$1} DESC" }
      send(name)
    else
      super
    end
  end
end
ActiveRecord::Base.extend(SearchLogicExtensions)

Note that I need to use ActiveRecord::Base.extend and not ActiveRecord::Base.include here, since my method_missing code calls “super” if the missing method does not match the pattern. “Extend” means that the methods of ActiveRecord::Base, including method_missing, will be overridden by the methods of the SearchLogicExtensions module, but will still be present and available via a call to “super.” Another important detail here is that I removed the “class << self” syntax. Since this is a module and not a class like Book was, I just define method_missing directly. My method_missing will be added as a class method to Book and all of my other models by the last line, when we extend ActiveRecord::Base. In my application I put this code into a file called “config/initializers/search_logic_extensions.rb,” which caused it to be loaded during the Rails initialization process. I could have also packaged the code up as a separate plugin.

That’s it for today; next time I’ll continue this discussion of metaprogramming with SearchLogic by showing how to sort with NULL values in an associated database table, using a LEFT OUTER JOIN query.