Pat Shaughnessy

Ribadesella, Spain

Rails generator tutorial part 2: writing a custom manifest action

September 02, 2009 · 1 comment

In part one of this tutorial I wrote a simple rails generator that creates a controller code file in app/controllers. This time I want to finish up my simple “VersionGenerator” example by enabling it to add a line to the routes.rb file. In other words, I want to be able to use a manifest action called “route” like this:

def manifest
    record do |m|
      m.template('controller.rb', 'app/controllers/version_controller.rb')
      m.route :name => 'version',
              :controller => 'version',
              :action => 'display_version'
    end
  end

The problem for today is how to implement the “route” action, which is not supported by Rails by default. The Rails generator base class code does provide a few other actions you can use in your manifest, like “file” or “directory,” which create a file or directory. Rails even provides a method called “route_resources” to add a “map.resources” line to routes.rb, but not a simple named route line like what we need for this example.

This specific action might be useful for you, but my real goal is to explore how Rails generators work in general and discuss some of the issues you might run into if you need a custom manifest action like this for any purpose. Let’s get started by just adding the new action method we need right inside our generator class…

def route(route_options)
  puts "This will create a new route!"
end

…and see what happens when we run script/generate with manifest method above:

$ ./script/generate version ../build-info/version.txt 
   identical  app/controllers/version_controller.rb
This will create a new route!

Very promising! All we need to do is add the necessary code here and we’ll be all set. But first let’s take a closer look at what’s happening by adding a call to “caller” inside this new method and see how it’s being called by the Rails generator system:

def route(route_options)
  puts "This will create a new route!"
  puts caller
end

Here’s what is displayed when we run this (paths shortened for readability):

$ ./script/generate version ../build-info/version.txt
   identical  app/controllers/version_controller.rb
This will create a new route!
/usr/local/lib/ruby/1.8/delegate.rb:270:in `__send__'
/usr/local/lib/ruby/1.8/delegate.rb:270:in `method_missing'
/path/to/rails-2.3.2/lib/rails_generator/manifest.rb:47:in `send'
/path/to/rails-2.3.2/lib/rails_generator/manifest.rb:47:in `send_actions'
/path/to/rails-2.3.2/lib/rails_generator/manifest.rb:46:in `each'
/path/to/rails-2.3.2/lib/rails_generator/manifest.rb:46:in `send_actions'
/path/to/rails-2.3.2/lib/rails_generator/manifest.rb:31:in `replay'
/path/to/rails-2.3.2/lib/rails_generator/commands.rb:42:in `invoke!'
/path/to/rails-2.3.2/lib/rails_generator/scripts/../scripts.rb:31:in `run'
/path/to/rails-2.3.2/lib/commands/generate.rb:6
/usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `gem_original_require'
/usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `require'
./script/generate:3

There’s a lot going on here; in fact, when I started to learn more about Rails generators and how they work I was shocked at how complicated their object model and design are. I won’t try to explain all of it here, but let’s take a closer look at the one line I bolded above: line 31 of lib/rails_generator/scripts.rb

Rails::Generator::Base.instance(options[:generator], args, options)
                      .command(options[:command]).invoke!

This is called when you run script/generate, by the “Rails::Generator::Scripts::Generate” class. Translating this line of Ruby into English, we get: “Create an instance of the specified generator, passing in the arguments and options provided on the command line; then create an instance of the specified command that uses that generator; then invoke the command.”

What this means is that actually Rails generator classes do not do the real work of generating code – Rails generator “commands” do! The lib/rails_generator/commands.rb file defines a series of command objects inside the Rails::Generator::Commands module. The two most important command classes are Create and Destroy. The “Create” command is called when you execute script/generate, like what happened in the stack trace above. This class contains the implementation of the “template” method we used before, plus various other methods like “file” and “directory” that create files and directories, etc.

The “Destroy” command is called when you execute script/destroy from the command line (I removed “puts caller” first before running this):

$ ./script/destroy version ../build-info/version.txt 
This will create a new route!
          rm  app/controllers/version_controller.rb

This class has all of the same methods that the Create class does, except that they perform the opposite action: instead of creating a file or directory, they delete them. The Destroy class also executes the actions in the opposite order, by looping backwards through the manifest that we recorded in our generator. Here the destroy command has deleted our version_controller.rb file we generated earlier. Note that it still displays “This will create a new route!”

If you’re interesting in diving into the real details about how Rails generators work, then read the code in commands.rb very carefully. In particular, look at how the “self.included” and “self.instance” methods at the top work; these methods are used in the line I showed above to create the specified command, supplying the specified generator as an argument to the command constructor. The “invoke!” method on commands actually plays back the recorded manifest. Also all of the actions that are available to your generator's manifest method are defined in this file. One other interesting detail I don’t have space to explain carefully here is that the command objects contain the corresponding generator class as a delegate object; in other words they contain an instance of the generator as a member variable:

# module Rails::Generator::Commands...
class Base < DelegateClass(Rails::Generator::Base)

This explains the top two lines in the stack trace above:

/usr/local/lib/ruby/1.8/delegate.rb:270:in `__send__'
/usr/local/lib/ruby/1.8/delegate.rb:270:in `method_missing'

“DelegateClass” is defined by delegate.rb, which is a Ruby library that implements the delegate pattern. This is why we were able to add the new “route” method right in our generator; when Ruby found the “route” method missing in the Create command, it delegated the call to our generator object.

Obviously we have a problem here: our new “route” method needs to be able to remove a route from routes.rb if the Destroy command is called, as well as create a new one for the Create command. One way to implement our new “route” method would be check the “options[:command]” value that we saw above on line 31 of scripts.rb, like this:

def route(route_options)
  if options[:command] == :create
    puts "This will add a new route to routes.rb."
  elsif options[:command] == :destroy
    puts "This will remove the new route from routes.rb."
  end
end

Now if we run script/generate again, our new method will create a route:

$ ./script/generate version ../build-info/version.txt
      create  app/controllers/version_controller.rb
This will add a new route to routes.rb.

And if we run the destroy command, we’ll remove the route:

$ ./script/destroy version ../build-info/version.txt 
This will remove the new route from routes.rb.
          rm  app/controllers/version_controller.rb

It turns out a cleaner way to implement the “route” method is to directly add the method to both the Create and Destroy command classes. This allows me to call a utility method called “gsub_file” in the Rails::Generator::Commands::Base class which I wouldn’t have direct access to from my generator class. It also avoids the somewhat ugly if statement on the options[:command] value, and finally it might make it easier for me someday to refactor the new route methods into a separate module that I could use with various different generators that might need to add and remove routes.

Anyway, here’s the finished code for the entire generator:

class VersionGenerator < Rails::Generator::NamedBase
  attr_reader :version_path
  def initialize(runtime_args, runtime_options = {})
    super
    @version_path = File.join(RAILS_ROOT, name)
  end
  def manifest
    record do |m|
      m.template('controller.rb', 'app/controllers/version_controller.rb')
      m.route :name => 'version',
              :controller => 'version',
              :action => 'display_version'
    end
  end
end

module Rails
  module Generator
    module Commands

      class Base
        def route_code(route_options)
          "map.#{route_options[:name]} '#{route_options[:name]}', :controller => '#{route_options[:controller]}', :action => '#{route_options[:action]}'"
        end
      end

# Here's a readable version of the long string used above in route_code;
# but it should be kept on one line to avoid inserting extra whitespace
# into routes.rb when the generator is run:
# "map.#{route_options[:name]} '#{route_options[:name]}',
#     :controller => '#{route_options[:controller]}',
#     :action => '#{route_options[:action]}'"

      class Create
        def route(route_options)
          sentinel = 'ActionController::Routing::Routes.draw do |map|'
          logger.route route_code(route_options)
          gsub_file 'config/routes.rb', /(#{Regexp.escape(sentinel)})/mi do |m|
              "#{m}\n  #{route_code(route_options)}\n"
          end
        end
      end

      class Destroy
        def route(route_options)
          logger.remove_route route_code(route_options)
          to_remove = "\n  #{route_code(route_options)}"
          gsub_file 'config/routes.rb', /(#{to_remove})/mi, ''
        end
      end

    end
  end
end

More or less copied from the existing route_resources action found in commands.rb, this works as follows: If we call script/generate then the route action method in the Create command class is called. This uses gsub_file to look for the “sentinel” or target area of the file and replaces it with the sentinel + the new route code. I also use the “logger” method to display log messages using the Rails::Generator::SimpleLogger class. Here’s what it looks like on the command line:

$ ./script/generate version ../build-info/version.txt
      create  app/controllers/version_controller.rb
       route  map.version 'version',
                 :controller => 'version',
                 :action => 'display_version'

If script/destroy is called, then the second route implementation in the Destroy class is called. This uses gsub_file to remove the route code. Finally, the “route_code” method returns the route code that we want to generate or remove from routes.rb. This time on the command line we get:

$ ./script/destroy version ../build-info/version.txt 
remove_route  map.version 'version',
                 :controller => 'version',
                 :action => 'display_version'
          rm  app/controllers/version_controller.rb

1 comment Tags:

Tutorial: How to write a Rails generator

August 23, 2009 · 1 comment

I’ve been working on a generator called ViewMapper recently that allows you to create view scaffolding from an existing model. I found writing a Rails generator to be somewhat confusing and hard to do: Where do I need to put my generator class? What does it need to be called? How does it work? This article will show step by step how to write your own Rails generator from scratch – hopefully it will save you some time if you ever need to write your own.

First let's think of some sample Rails code that I might want to generate as an example. This is admittedly contrived, but it’s short and simple enough to show here while still interesting enough to illustrate how a generator would work. Let’s suppose my Capistrano deployment scripts wrote the last commit information from Git into a file called “version.txt” in a folder RAILS_ROOT/../build-info:

commit 217d9bd1d1d508a0f7f1e7afa2b489b130196e33
Author: Pat Shaughnessy <pat@patshaughnessy.net>
Date:   Wed Aug 5 07:45:01 2009 -0400

Now with a controller like this I could display the info, helping me keep track of what version is running on a given development development, QA or production server:

class VersionController < ApplicationController
  VERSION_FILE = '../build-info/version.txt'
  def display_version
    path = File.join(RAILS_ROOT, VERSION_FILE)
    render path
  end
end

If I add a route to this action like this in config/routes.rb:

map.version 'version', :controller => 'version', :action => 'display_version'

…then I’ll get the Git last commit info by just opening http://servername/version in my browser. Obviously it would be simpler to just put the version.txt file under the public folder… but let’s continue with this contrived example for now. Now let’s say I have 5 or 10 different Rails apps, and that I’d like to add the same controller and route to each one. Let’s write a simple Rails generator that would make this easy to do, called the “version” generator. Here’s how to do it…

Step 1: A skeleton Rails generator

All Rails generators are derived from a class called “Rails::Generator::Base.” You can find the source code for this file in vendor/rails/railties/lib/rails_generator/base.rb (link is to the Rails 2.3.3 version). Definately read the source code; there is helpful documentation in the comments and understanding the base class code is indispensible if you plan to use it for your generator.

Here’s an empty, skeleton Rails generator that we can use as a starting point:

class VersionGenerator < Rails::Generator::Base
  def manifest
    record do |m|
      # Do something
    end
  end
end

The name of the class is important: since we want a “version” generator we need to create a class called “VersionGenerator.” The other important detail here is where this class is located: we need to put it in lib/generators/version/version_generator.rb. The reason for all of this is that the code called by script/generate takes the generator name and looks for the corresponding generator class in this location. We can check that our class is being found by just running script/generate with no parameters, like this:

$ ./script/generate 
Usage: ./script/generate generator [options] [args]
…etc…
Installed Generators
  Lib: version
  Rubygems: cucumber, feature, install_rubigen_scripts, integration_spec, rspec, rspec_controller, rspec_model, rspec_scaffold, session
  Builtin: controller, helper, integration_test, mailer, metal, migration, model, observer, performance_test, plugin, resource, scaffold, session_migration

Note that our new “version” generator is listed as “Lib: version,” in bold above. The other generators found on my system were either part of Rails or else part of a gem or plugin. If you don’t see your generator in this list, or if it has the wrong name then double check your folder name: it should be lib/generators/XYZ or lib/generators/version in this example. After you finish developing your generator you'll want to move it into a plugin or gem so it can be reused in different apps on your machine; then "version" would appear in the "Rubygems" list for example.

If we try running our empty generator, we’ll get no output or changes; not a surprise:

$ ./script/generate version

If you see an error like this:

$ ./script/generate version
/usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:
in `gem_original_require': no such file to load --
/path/to/myapp/lib/generators/version/version_generator.rb (MissingSourceFile)

Then you have the wrong file name… it needs to be XYZ_generator.rb, or version_generator.rb in this example. Or if you see:

Missing VersionGenerator class in
  /path/to/myapp/lib/generators/version/version_generator.rb

… then you have the wrong class name inside this file. It needs to be XYZGenerator, or VersionGenerator in this example.

Step 2: Creating code using an ERB template

The way Rails generators work is by creating code using ERB, in just the same way that view code generates HTML for a Rails web site. To see how this works and to take the next step towards writing our version generator, let’s create a new template file for our version controller. Create a new text file called “controller.rb” and put it into this folder: lib/generators/version/templates/controller.rb; in other words, into a new subfolder under version called “templates.” Next, paste in the controller code from above:

class VersionController < ApplicationController
  VERSION_FILE = '../build-info/version.txt'
  def display_version
    path = File.join(RAILS_ROOT, VERSION_FILE)
    render path
  end
end

Now if we add this line to our manifest method back in the VersionGenerator class:

def manifest
  record do |m|
    m.template('controller.rb', 'app/controllers/version_controller.rb')
  end
end

… a new code file called version_controller.rb will be generated inside of the app/controllers folder in our app when we run the generator:

$ ./script/generate version
    create  app/controllers/version_controller.rb

If you take a look at this new version_controller.rb file, you’ll just see the code we pasted into the controller.rb template file. Here’s what happened when we ran script/generate version:

  • The Rails code called by script/generate looked for the “version” generator, by looking for a class called “VersionGenerator” in the lib/generators/version folder, among other places.
  • It called the “manifest” method in VersionGenerator. Using the “record” utility method, a series of actions are recorded that the Rails generator base classes will execute later. In our case, there’s only one action: “template.”
  • The template action indicates that the Rails generator code should run an ERB transformation to generate new code, using the “controller.rb” file. The generated code will then be copied to the app/controllers/version_controller.rb file.

To make our generator interesting, let’s provide a parameter to it that indicates the path of the file containing the version information, instead of hard coding “version.txt.” To do that we can take advantage of another Rails built in class called “Rails::Generator::NamedBase”. If we derive our VersionGenerator class from NamedBase instead of Base, then we’ll get the ability to take a name parameter from the script/generate command line for free. Let start by changing our base class in version_generator.rb:

class VersionGenerator < Rails::Generator::NamedBase

Now if you run the generator, you’ll get some helpful usage information:

$ ./script/generate version
Usage: ./script/generate version VersionName [options]
Etc…

Here the NamedBase class has written the usage info, explaining that an additional parameter is now expected. If we re-run the generator and add the version file name as a parameter, here’s what we’ll get:

$ ./script/generate version ../build-info/version.txt
   identical  app/controllers/version_controller.rb

This shows a helpful feature of Rails generators: they check if you’re about to overwrite some existing code with the new, generated code. In this case, since we already had a controller called “version_controller.rb” the Rails generator code displayed the “identical” message. Our new generated controller is identical to the previous one since we haven’t used the version name parameter yet. To take advantage of the version file name parameter and our new NamedBase base class, we need to add some new code to VersionGenerator, in bold:

class VersionGenerator < Rails::Generator::NamedBase
  attr_reader :version_path
  def initialize(runtime_args, runtime_options = {})
    super
    @version_path = File.join(RAILS_ROOT, name)
  end
  def manifest
    record do |m|
      m.template('controller.rb', 'app/controllers/version_controller.rb')
    end
  end
end

The way this works is:

  • The NamedBase base class parses the script/generate command line, expecting an additional parameter which indicates the name of some object.
  • This name is provided in "name," an instance variable/attribute of the NamedBase class.
  • The VersionGenerator class assumes "name" indicates the relative path of a version file, determines the full path of the file and then saves the full path in the @version_path attribute.

I mentioned above that Rails generators work by running an ERB transformation; to try using ERB in this example, we just need to refer to the new “version_path” attribute inside the controller.rb template file, like this:

class VersionController < ApplicationController
  VERSION_PATH = '<%= version_path %>'
  def display_version
    render VERSION_PATH
   end
end

The “version_path” variable in this template refers to the version_path attribute of the VersionGenerator class above. I’ve also removed the File.join line from here since we now handle that in the generator itself. If we run the generate command and provide the name of the version file, we’ll get this result:

$ ./script/generate version ../build-info/version.txt
overwrite app/controllers/version_controller.rb? (enter "h" for help) [Ynaqdh] Y
       force  app/controllers/version_controller.rb

As I mentioned above, Rails checks whether you’re about to overwrite some existing code file. Since in this case our controller code file is now different that what we had before (VERSION_PATH and not VERSION_FILE; no File.join, etc.) we get an overwrite warning. I just entered “Y” to overwrite the old file.

If we look at our new controller, app/controllers/version_controller.rb, we now have:

class VersionController < ApplicationController
  VERSION_PATH = '/path/to/myapp/../build-info/version.txt'
  def display_version
    render VERSION_PATH
   end
end

We’ve successfully generated a new VersionController class, based on a parameter passed to our “version” generator. While this is a good start, to be able to use this action in our app we still need to add a route to config/routes.rb. It turns out this is a lot harder to do with a generator, since we will need to insert a new line of code in an existing file, and not just generate a new routes.rb file. We will also need to worry about how to remove the route line when script/destroy is run. It’s possible to do all of this with a Rails generator, but will require a more thorough understanding of how Rails generators actually work. I’ll explain all of that in my next post…

1 comment Tags: