Pat Shaughnessy

Ribadesella, Spain

Code record/playback using Rails 3 generators

July 27, 2010 · 0 comments

For a while I’ve been thinking that writing a Rails generator is a fairly difficult thing to do. First you need to learn about Thor and the Rails generator system: what sort of Ruby class you need to write, how to handle arguments, how to run commands like “copy_file”, etc. Then you need to write ERB files to produce the code that you’d like to generate, which is always a chore.

So last night I wrote a gem called generate_from_diff that let’s you create Rails 3 generators automatically using a code record/playback model. Here’s how it works:

  • You make a code change in some Rails project, and commit it to your Git repo.
  • You run a command from my gem to extract this code change from the Git repo into a new Rails 3 generator.
  • You later run this new generator in any other Rails 3 app, and your recorded code changes are “played back,” or applied to your new project, using the Unix patch utility written by Larry Wall.

You’ve created a Rails 3 generator without ever writing a single line of generator code!

Disclaimer: I just got this working last night, so it’s still very rough; but if the idea seems worthwhile I’ll clean it up and try to make it more robust and useable. Another disclaimer: none of this will work on Windows since it relies on the Unix patch utility.

Example: recording code into a new generator

Let’s look at an example to try to make this a bit clearer. Support I create a new Rails 3 application:

$ rails new first_app
      create  
      create  README
      create  Rakefile
      create  config.ru
      create  .gitignore
      create  Gemfile
etc...

And let’s run bundle install to be sure I have all the required gems. This is usually not necessary for a new, empty Rails app, but I want to have my Gemfile.lock file created... more on that in a moment.

$ cd first_app
$ bundle install
Fetching source index for http://rubygems.org/
Using rake (0.8.7) 
Using abstract (1.0.0) 
Using activesupport (3.0.0.beta4) 
Using builder (2.1.2) 
etc...

And let’s create a new Git repo here and check the empty application into it:

$ git init
Initialized empty Git repository in /Users/pat/.../first_app/.git/
$ git add .
$ git commit -m"New sample app"

This first Git revision will serve as the baseline for recording my new generator, which I’ll do in a minute. The reason I ran bundle install was to insure that the Gemfile.lock file would be included in the baseline... and so not included in the recorded code change.

Now let’s write some code that I can record into a new generator. Suppose at my company I want to create a controller that returns the build number, diagnostics and some other information about each of my Rails apps. I might do this by creating a new controller as follows:

$ rails generate controller build_info
      create  app/controllers/build_info_controller.rb
      invoke  erb
      create    app/views/build_info
      invoke  test_unit
      create    test/functional/build_info_controller_test.rb
      invoke  helper
etc...

And in this new controller I’ll add a single index action:

class BuildInfoController < ApplicationController
  def index
    render :text => 'Some interesting build info about this app...'
  end
end

Finally, I’ll add a route to send “build_info” requests to this action:

FirstApp::Application.routes.draw do |map|

  match 'build_info' => 'build_info#index'

...etc...

This is somewhat silly, but it’s simple enough to use as an example here. Now if I run my app I’ll get this fascinating page:

Next let’s “record” this sample code by using generate_from_diff to create a new Rails generator for it. First, we need to install generate_from_diff:

$ gem install generate_from_diff
Successfully installed generate_from_diff-0.0.1
1 gem installed
Installing ri documentation for generate_from_diff-0.0.1...
Installing RDoc documentation for generate_from_diff-0.0.1...

Next, let’s commit my new controller and routes.rb code changes:

$ git add .

$ git status
# On branch master
# Changes to be committed:
#   (use "git reset HEAD <file>..." to unstage)
#
#        new file:   app/controllers/build_info_controller.rb
#        new file:   app/helpers/build_info_helper.rb
#        modified:   config/routes.rb
#        new file:   test/functional/build_info_controller_test.rb
#        new file:   test/unit/helpers/build_info_helper_test.rb
#

$ git commit -m"Build info"
Created commit 037ca3b: Build info
 5 files changed, 22 insertions(+), 0 deletions(-)
 create mode 100644 app/controllers/build_info_controller.rb
 create mode 100644 app/helpers/build_info_helper.rb
 create mode 100644 test/functional/build_info_controller_test.rb
 create mode 100644 test/unit/helpers/build_info_helper_test.rb

One last detail: we need to edit the Gemfile to load generate_from_diff into this application:

source 'http://rubygems.org'


gem 'generate_from_diff'

gem 'rails', '3.0.0.beta4'
etc...

Finally we create our new generator by just running this command:

$ rails generate generator_from_diff build_info HEAD~1 HEAD
    create  lib/generators/build_info
    create  lib/generators/build_info/build_info_generator.rb
    create  lib/generators/build_info/USAGE
       run  git diff --no-prefix HEAD~1 HEAD from "."

Ok - what happened here was that I ran a generator called “generator_from_diff” that is located inside the generate_from_diff gem. I provided it with the name of the new generator I want to create: “build_info” in this example. This is similar to how the Rails 3 “generator generator” works: it generates a generator. But next I provide two Git revisions, in this example “HEAD~1” and “HEAD,” the first and second revisions in my Git repo. The first value is the baseline revision: what to compare to. In this example, this is my new, empty Rails application. The second revision is what code to record and save into the new generator, in this example this revision contains all of my controller and routes.rb changes.

Example: playing back code using a generator

Now let’s see if we can use this new Rails generator to copy the build_info controller and route into a different Rails app. First, let’s create a second, new Rails application:

$ cd ..
$ rails new second_app
    create  
    create  README
    create  Rakefile
    create  config.ru
...etc...
$ cd second_app

And next, let’s copy the new generator we just created in the first app, over to this new app:

$ mkdir lib/generators
$ cp -r ../first_app/lib/generators/build_info lib/generators

And now we can just run our new generator to playback the code changes that I recorded above:

$ rails generate build_info
      gsub  lib/generators/build_info/build_info.patch
       run  patch -p0 < /Users/pat/.../second_app/lib/generators/build_info/build_info.patch from "."
  patching file app/controllers/build_info_controller.rb
  patching file app/helpers/build_info_helper.rb
  patching file config/routes.rb
  patching file test/functional/build_info_controller_test.rb
  patching file test/unit/helpers/build_info_helper_test.rb

That’s it! Now I can run the second app and see the same build status page that we had before:

How does this actually work?

Here’s what is going on under the hood. First, when you record your code changes into the new generator like this:

$ rails generate generator_from_diff build_info HEAD~1 HEAD

... the “generator_from_diff” code actually runs the “git diff” command like this:

$ git diff HEAD~1 HEAD
diff --git a/app/controllers/build_info_controller.rb
           b/app/controllers/build_info_controller.rb
new file mode 100644
index 0000000..c44d83e
--- /dev/null
+++ b/app/controllers/build_info_controller.rb
@@ -0,0 +1,5 @@
+class BuildInfoController < ApplicationController
+  def index

...etc...

This produces a list of all the text changes that were made from one revision (HEAD~1) to another (HEAD). These are then saved into a file called “build_info.patch,” saved inside the new generator.

Later, the text differences, the “patch,” are applied to whatever new or existing files are found relative to the current directory when you run the generator. This copies the new controller file as well as the new route inside of routes.rb into the other application. The patch file is applied using this command:

patch -p0 < lib/generators/build_info/build_info.patch

I use patch instead of git apply to avoid the need to match revision id’s; these will be different from one repo to another.

Ok sounds interesting - so where are you going with this next?

I think it’s cool to be able to “record” Rails generators without writing any code. If this seems like a useful idea, then I’ll spend some more time to clean it up and make it more robust. For example, I’m thinking of adding some code to warn you before the patch is run if there are unexpected files present, or if some expected files are missing.

Next, I’m considering enhancing the gem to perform search/replace using arguments or options that you specify when recording the generator. For example, suppose you recorded a series of code changes that had to do with a model called “Person.” But imagine that you want to be able to playback those code changes in a target application that might have a different model name, “User” instead of “Person” for example. Then the gem could search/replace on the patch file, both when it’s recorded and again when it’s played back, to cause the generated code to use User instead of Person.

→ 0 comments Tags:

How to install Paperclip in a Rails 3 app

May 23, 2010 · 14 comments

Update June 2010: I just heard from Jon Yurek in the comments below that he has, in fact, finished up the Rails 3 changes for Paperclip. This means that you can now just install Paperclip as usual in a Rails 3 app as a plugin:

$ rails plugin install git://github.com/thoughtbot/paperclip.git

... or as a gem by adding it to your Gemfile if you’ve already installed it with “gem install paperclip:”

source 'http://rubygems.org'

gem 'rails', '3.0.0.beta3'

# Bundle edge Rails instead:
# gem 'rails', :git => 'git://github.com/rails/rails.git'


gem 'paperclip'

I’ll leave my original article here as a reference – it was a fun learning experience trying out Paperclip with Rails 3, and the same ideas around Bundler, generators, etc., might still be helpful while using other gems or plugins with Rails 3.


 

To get Paperclip to work in a Rails 3 application, use this in your Gemfile:

gem 'paperclip', :git => 'git://github.com/thoughtbot/paperclip.git',
                 :branch => 'rails3'

… and this in application.rb:

module YourPaperclipApp
  class Application < Rails::Application


    Paperclip::Railtie.insert

etc...

  end
end

Right now it looks like Thoughtbot is finishing Rails 3 related changes in a “rails3” branch in their Paperclip github repository. The best thing to do if you have a Paperclip app you want to migrate to Rails 3 is simply to wait a bit longer for them to finish that work, test it and merge it back into the master branch.

The rest of this article is really not about Paperclip at all, but about Rails 3. Here’s what I learned about Rails 3 while troubleshooting Paperclip:

  1. The command line has changed
  2. Plugin generators have moved
  3. Rails 2.x generators don’t work at all
  4. You use Bundler and a “Gemfile” to declare gems
  5. You can install a gem from a specific git repository branch
  6. Rails 3 frameworks are now based on Rails::Railtie
  7. Bundler does not call rails/init.rb in each gem
Let’s go ahead and troubleshoot Paperclip together in a new Rails 3 app. First I’ll begin by verifying the versions of Ruby and Rails I’m using now:

$ ruby -v
ruby 1.8.7 (2010-01-10 patchlevel 249) [i686-darwin9.8.0]
$ rails -v
Rails 3.0.0.beta3

And next I’ll create a sample app to use with Paperclip:

$ rails paperclip-sample-app
      create  
      create  README
      create  .gitignore
      create  Rakefile
      create  config.ru
      create  Gemfile
      create  app
      create  app/controllers/application_controller.rb
etc…

Now we’re ready to install Paperclip into my new app. But what should I do exactly? Should I use Paperclip as a plugin or a gem? I wasn’t sure what to do, so I simply tried both.

Fact 1: the command line has changed

First let’s install it as a plugin, since that’s the most straightforward. In Rails 3 the plugin install command has changed a bit vs. Rails 2.x:

$ cd paperclip-sample-app
$ rails plugin install git://github.com/thoughtbot/paperclip.git
Initialized empty Git repository in .../vendor/plugins/paperclip/.git/
remote: Counting objects: 77, done.
remote: Compressing objects: 100% (68/68), done.
remote: Total 77 (delta 12), reused 20 (delta 0)
Unpacking objects: 100% (77/77), done.
From git://github.com/thoughtbot/paperclip
 * branch            HEAD       -> FETCH_HEAD

Next let’s use scaffolding to create a “User” model with a couple of attributes:

$ rails generate scaffold user name:string email:string
      invoke  active_record
      create    db/migrate/20100521034815_create_users.rb
      create    app/models/user.rb
      invoke    test_unit
      create      test/unit/user_test.rb
      create      test/fixtures/users.yml
       route  resources :users
      invoke  scaffold_controller
      create    app/controllers/users_controller.rb
      invoke    erb
      create      app/views/users
      create      app/views/users/index.html.erb
etc…
$ rake db:migrate

Fact 2: plugin generators have moved

The next step is to create a second migration for the additional database columns required by Paperclip. To make this easy, Paperclip provides a “paperclip” generator; let’s try that and specify that we want an “avatar” file attachment saved on the user model:

$ rails generate paperclip user avatar
Could not find generator paperclip.

Uhh… not what I expected. It looks like something has changed about Rails 3 generators that has broken the Paperclip generator. For now, let’s take a look at the Paperclip code to see if we can find the generator:

$ find vendor/plugins/paperclip -name *generator*
vendor/plugins/paperclip/generators
vendor/plugins/paperclip/generators/paperclip/paperclip_generator.rb

There it is… After some research, I found out that for Rails 3, plugin/gem generators need to be located inside a folder called “BASE_DIR/lib/generators” – we can see here that the Paperclip generator needs to be moved in order to comply with this new standard.

Fact 3: Rails 2.x generators don’t work at all

So let’s try just moving it and see what happens:

mv vendor/plugins/paperclip/generators vendor/plugins/paperclip/lib/.
$ rails generate paperclip user avatar
[WARNING] Could not load generator "generators/paperclip/paperclip_generator"
because it's a Rails 2.x generator, which is not supported anymore.
Error: uninitialized constant Rails::Generator.

Things are looking worse and worse! It turns out that the generators architecture for Rails 3 has been completely rewritten, and that generators written for Rails 2.x will simply not work at all in Rails 3. What to do now? Of course, I could simply hand code the migration for adding the avatar columns to the users table, and continue to work on my sample application. Instead, I decided to give up on the plugin entirely and to try using Paperclip as a gem.

Fact 4: You use Bundler and a “Gemfile” to declare gems

Let’s take a look at how gems are installed for a Rails 3 app. Rails 3 uses a new file called the “Gemfile,” which specifies which gems should be included in your application. This file is read and used by Bundler, which manages gems and their dependencies. We can specify that our application uses the Paperclip gem by adding a single line to the Gemfile like this:

source 'http://rubygems.org'
gem 'rails', '3.0.0.beta3'

# Bundle edge Rails instead:
# gem 'rails', :git => 'git://github.com/rails/rails.git'


gem 'paperclip'

etc…

This simply tells Bundler to install Paperclip from your default gem source: probably rubygems.org. Now I’ll delete the plugin I installed earlier and install the gem, using the “bundle install” command to install all of the gems in my Gemfile:

$ rm -rf vendor/plugins/paperclip/
$ bundle install
Fetching source index from http://rubygems.org/
Using rake (0.8.7) from system gems 
Using abstract (1.0.0) from bundler gems 

etc…

Installing paperclip (2.3.1.1) from rubygems repository at http://rubygems.org/ 

etc…

Bundler indicated that it found the official version of Paperclip on rubygems.org, downloaded and installed it. It also told us we have version 2.3.1.1. Let’s use the bundle show command and take a look at where Paperclip was installed to:

$ bundle show paperclip
  /Users/pat/.rvm/gems/ruby-1.8.7-p249/gems/paperclip-2.3.1.1

Bundler simply installed the gem in the standard location where all my other gems are located for my RVM version of Ruby 1.8.7, just as if I had run a gem install command manually. Now let’s try that generate command again and see if it works any better:

$ rails generate paperclip user avatar
DEPRECATION WARNING: RAILS_ROOT is deprecated! Use Rails.root instead. (called from expand_path at /Users/pat/.rvm/gems/ruby-1.8.7-p249/gems/paperclip-2.3.1.1/lib/paperclip.rb:39)
/Users/pat/.rvm/gems/ruby-1.8.7-p249/gems/paperclip-2.3.1.1/lib/paperclip.rb:39:in `expand_path': can't convert # into String (TypeError)
	from /Users/pat/.rvm/gems/ruby-1.8.7-p249/gems/paperclip-2.3.1.1/lib/paperclip.rb:39
	from /Users/pat/.rvm/gems/ruby-1.8.7-p249/gems/bundler-0.9.25/lib/bundler/runtime.rb:46:in `require'
	from /Users/pat/.rvm/gems/ruby-1.8.7-p249/gems/bundler-0.9.25/lib/bundler/runtime.rb:46:in `require'
	from /Users/pat/.rvm/gems/ruby-1.8.7-p249/gems/bundler-0.9.25/lib/bundler/runtime.rb:41:in `each'

Still broken! It looks like I’m just not running code that was intended to be used with Rails 3.

Fact 5: You can install a gem from a specific git repository branch

After more investigation, I noticed that there had been a lot of recent changes to the Paperclip github repository. At the time I wrote this, Thoughtbot was actively developing on a branch called “rails3.” I decided the best thing to do would be to try the code from the rails3 branch, hoping it might work better for me. Bundler makes this easy, since you can just specify git as a source for downloading a gem using a “git” option, as well as optionally a specific branch using a “branch” option, like this:

source 'http://rubygems.org'

gem 'rails', '3.0.0.beta3'

# Bundle edge Rails instead:
# gem 'rails', :git => 'git://github.com/rails/rails.git'


gem 'paperclip', :git => 'git://github.com/thoughtbot/paperclip.git',
                 :branch => 'rails3'

etc …

After saving this change to Gemfile, let’s re-run the bundle install command:

$ bundle install
Updating git://github.com/thoughtbot/paperclip.git
Fetching source index from http://rubygems.org/
Updating git://github.com/thoughtbot/paperclip.git
Using rake (0.8.7) from system gems 
Using abstract (1.0.0) from bundler gems 

…etc…

Using paperclip (2.3.2.beta1)
  from git://github.com/thoughtbot/paperclip.git (at rails3)

Hmm… interesting. Bundler is showing that it’s downloaded Paperclip from the github repository, and that it got the code at the head of the rails3 branch. Another interesting detail here is that I apparently now have version “2.3.2 beta1” of Paperclip. This is a good sign, since I have a more recent version than 2.3.1.1 (the rubygems.org version) and also it seems that Thoughtbot is actively working on it since it’s labeled “beta1.”

If we run bundle show again, we can see that Bundler has saved a special copy of Paperclip downloaded from github, along with the git commit id and branch of the version I have:

$ bundle show paperclip
      /Users/pat/.rvm/gems/ruby-1.8.7-p249/bundler/gems/paperclip-61f74de14812cabc026967a2b2c3ca8cbd2eed69-rails3

Now let’s try that generator once more:

$ rails generate paperclip user avatar
            create  db/migrate/20100521003113_add_attachment_avatar_to_user.rb

Yes! It’s working now!

Fact 6: Rails 3 frameworks are now based on Rails::Railtie

Let’s continue to put together my sample application by running the migration:

$ rake db:migrate

… and by editing my User model to call “has_attached_file:”

class User < ActiveRecord::Base
  has_attached_file :avatar
end

Before I start editing my views and adding the code to upload and display the avatar attachment, let’s start the server and see if Paperclip is working. Opening the users index page I get…

… more trouble! I’m definitely having a bad day… what now? Well it seems that Paperclip is just not being loaded at all, or is being initialized improperly for some reason. At this point I started to poke around the Paperclip source code a bit, and found that the code that includes the Paperclip module into ActiveRecord::Base was moved and is no longer being called. Since Paperclip is not included in my User/ActiveRecord class I get the error has_attached_file not defined, since that’s defined by Paperclip.

I found the include code in a file called “lib/paperclip/railtie.rb:”

require 'paperclip'

module Paperclip
  if defined? Rails::Railtie
    require 'rails'
    class Railtie < Rails::Railtie
      config.after_initialize do
        Paperclip::Railtie.insert
      end
    end
  end

  class Railtie
    def self.insert
      ActiveRecord::Base.send(:include, Paperclip)
      File.send(:include, Paperclip::Upfile)
    end
  end
end

I’m not quite sure what Thoughtbot’s plans are for Paperclip, but if you take some time to read through Yehuda Katz’s write up Rails and Merb Merge: Rails Core (Part 4 of 6), you’ll learn about how Rails frameworks like ActiveRecord and ActiveController have been recast as instances of this “Rails::Railtie” class. Possibly Paperclip will become one of these. Rails 3 has a new API for declaring how Railties are loaded and initialized, but it looks like this version of Paperclip and this version of Rails aren’t quite working correctly now.

Fact 7: Bundler does not call rails/init.rb in each gem

For now, the problem I’m having in my sample application is that the Paperclip::Railtie.insert method is not being called – the two lines I highlighted above need to be executed in order to enable “has_attached_file” to be present as a class method for ActiveRecord models. To make things more interesting, Thoughtbot did include a call to insert inside rails/init.rb, like this:

require 'paperclip/railtie'
Paperclip::Railtie.insert

… but for Rails 3, it turns out that Bundler no longer calls rails/init.rb.

Moving this line instead to config/application.rb will solve the problem:

module PaperclipSampleApp
  class Application < Rails::Application

    Paperclip::Railtie.insert

    etc…
    
  end
end

Alternatively, you could just create a file called “config/initializers/paperclip.rb” and put the call to insert there.

Now reloading the users index page we finally get Paperclip to work:

Instead of proceeding with my sample app now, I’m going to wait a few weeks while Thoughtbot finishes off the Rails 3 changes for Paperclip.

I don’t think troubleshooting these problems was a waste of time at all; in fact, it was a good excuse to get my hands dirty with Rails 3 and Bundler. Once Thoughtbot has finished their changes in the rails3 branch and merged them into the master I’ll update my tutorial from last year, and also update my Paperclip fork to support database BLOB storage for Rails 3.

→ 14 comments Tags:

Rails generator tutorial part 2: writing a custom manifest action

September 02, 2009 · 2 comments

In part one of this tutorial I wrote a simple rails generator that creates a controller code file in app/controllers. This time I want to finish up my simple “VersionGenerator” example by enabling it to add a line to the routes.rb file. In other words, I want to be able to use a manifest action called “route” like this:

def manifest
    record do |m|
      m.template('controller.rb', 'app/controllers/version_controller.rb')
      m.route :name => 'version',
              :controller => 'version',
              :action => 'display_version'
    end
  end

The problem for today is how to implement the “route” action, which is not supported by Rails by default. The Rails generator base class code does provide a few other actions you can use in your manifest, like “file” or “directory,” which create a file or directory. Rails even provides a method called “route_resources” to add a “map.resources” line to routes.rb, but not a simple named route line like what we need for this example.

This specific action might be useful for you, but my real goal is to explore how Rails generators work in general and discuss some of the issues you might run into if you need a custom manifest action like this for any purpose. Let’s get started by just adding the new action method we need right inside our generator class…

def route(route_options)
  puts "This will create a new route!"
end

…and see what happens when we run script/generate with manifest method above:

$ ./script/generate version ../build-info/version.txt 
   identical  app/controllers/version_controller.rb
This will create a new route!

Very promising! All we need to do is add the necessary code here and we’ll be all set. But first let’s take a closer look at what’s happening by adding a call to “caller” inside this new method and see how it’s being called by the Rails generator system:

def route(route_options)
  puts "This will create a new route!"
  puts caller
end

Here’s what is displayed when we run this (paths shortened for readability):

$ ./script/generate version ../build-info/version.txt
   identical  app/controllers/version_controller.rb
This will create a new route!
/usr/local/lib/ruby/1.8/delegate.rb:270:in `__send__'
/usr/local/lib/ruby/1.8/delegate.rb:270:in `method_missing'
/path/to/rails-2.3.2/lib/rails_generator/manifest.rb:47:in `send'
/path/to/rails-2.3.2/lib/rails_generator/manifest.rb:47:in `send_actions'
/path/to/rails-2.3.2/lib/rails_generator/manifest.rb:46:in `each'
/path/to/rails-2.3.2/lib/rails_generator/manifest.rb:46:in `send_actions'
/path/to/rails-2.3.2/lib/rails_generator/manifest.rb:31:in `replay'
/path/to/rails-2.3.2/lib/rails_generator/commands.rb:42:in `invoke!'
/path/to/rails-2.3.2/lib/rails_generator/scripts/../scripts.rb:31:in `run'
/path/to/rails-2.3.2/lib/commands/generate.rb:6
/usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `gem_original_require'
/usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:in `require'
./script/generate:3

There’s a lot going on here; in fact, when I started to learn more about Rails generators and how they work I was shocked at how complicated their object model and design are. I won’t try to explain all of it here, but let’s take a closer look at the one line I bolded above: line 31 of lib/rails_generator/scripts.rb

Rails::Generator::Base.instance(options[:generator], args, options)
                      .command(options[:command]).invoke!

This is called when you run script/generate, by the “Rails::Generator::Scripts::Generate” class. Translating this line of Ruby into English, we get: “Create an instance of the specified generator, passing in the arguments and options provided on the command line; then create an instance of the specified command that uses that generator; then invoke the command.”

What this means is that actually Rails generator classes do not do the real work of generating code – Rails generator “commands” do! The lib/rails_generator/commands.rb file defines a series of command objects inside the Rails::Generator::Commands module. The two most important command classes are Create and Destroy. The “Create” command is called when you execute script/generate, like what happened in the stack trace above. This class contains the implementation of the “template” method we used before, plus various other methods like “file” and “directory” that create files and directories, etc.

The “Destroy” command is called when you execute script/destroy from the command line (I removed “puts caller” first before running this):

$ ./script/destroy version ../build-info/version.txt 
This will create a new route!
          rm  app/controllers/version_controller.rb

This class has all of the same methods that the Create class does, except that they perform the opposite action: instead of creating a file or directory, they delete them. The Destroy class also executes the actions in the opposite order, by looping backwards through the manifest that we recorded in our generator. Here the destroy command has deleted our version_controller.rb file we generated earlier. Note that it still displays “This will create a new route!”

If you’re interesting in diving into the real details about how Rails generators work, then read the code in commands.rb very carefully. In particular, look at how the “self.included” and “self.instance” methods at the top work; these methods are used in the line I showed above to create the specified command, supplying the specified generator as an argument to the command constructor. The “invoke!” method on commands actually plays back the recorded manifest. Also all of the actions that are available to your generator's manifest method are defined in this file. One other interesting detail I don’t have space to explain carefully here is that the command objects contain the corresponding generator class as a delegate object; in other words they contain an instance of the generator as a member variable:

# module Rails::Generator::Commands...
class Base < DelegateClass(Rails::Generator::Base)

This explains the top two lines in the stack trace above:

/usr/local/lib/ruby/1.8/delegate.rb:270:in `__send__'
/usr/local/lib/ruby/1.8/delegate.rb:270:in `method_missing'

“DelegateClass” is defined by delegate.rb, which is a Ruby library that implements the delegate pattern. This is why we were able to add the new “route” method right in our generator; when Ruby found the “route” method missing in the Create command, it delegated the call to our generator object.

Obviously we have a problem here: our new “route” method needs to be able to remove a route from routes.rb if the Destroy command is called, as well as create a new one for the Create command. One way to implement our new “route” method would be check the “options[:command]” value that we saw above on line 31 of scripts.rb, like this:

def route(route_options)
  if options[:command] == :create
    puts "This will add a new route to routes.rb."
  elsif options[:command] == :destroy
    puts "This will remove the new route from routes.rb."
  end
end

Now if we run script/generate again, our new method will create a route:

$ ./script/generate version ../build-info/version.txt
      create  app/controllers/version_controller.rb
This will add a new route to routes.rb.

And if we run the destroy command, we’ll remove the route:

$ ./script/destroy version ../build-info/version.txt 
This will remove the new route from routes.rb.
          rm  app/controllers/version_controller.rb

It turns out a cleaner way to implement the “route” method is to directly add the method to both the Create and Destroy command classes. This allows me to call a utility method called “gsub_file” in the Rails::Generator::Commands::Base class which I wouldn’t have direct access to from my generator class. It also avoids the somewhat ugly if statement on the options[:command] value, and finally it might make it easier for me someday to refactor the new route methods into a separate module that I could use with various different generators that might need to add and remove routes.

Anyway, here’s the finished code for the entire generator:

class VersionGenerator < Rails::Generator::NamedBase
  attr_reader :version_path
  def initialize(runtime_args, runtime_options = {})
    super
    @version_path = File.join(RAILS_ROOT, name)
  end
  def manifest
    record do |m|
      m.template('controller.rb', 'app/controllers/version_controller.rb')
      m.route :name => 'version',
              :controller => 'version',
              :action => 'display_version'
    end
  end
end

module Rails
  module Generator
    module Commands

      class Base
        def route_code(route_options)
          "map.#{route_options[:name]} '#{route_options[:name]}', :controller => '#{route_options[:controller]}', :action => '#{route_options[:action]}'"
        end
      end

# Here's a readable version of the long string used above in route_code;
# but it should be kept on one line to avoid inserting extra whitespace
# into routes.rb when the generator is run:
# "map.#{route_options[:name]} '#{route_options[:name]}',
#     :controller => '#{route_options[:controller]}',
#     :action => '#{route_options[:action]}'"

      class Create
        def route(route_options)
          sentinel = 'ActionController::Routing::Routes.draw do |map|'
          logger.route route_code(route_options)
          gsub_file 'config/routes.rb', /(#{Regexp.escape(sentinel)})/mi do |m|
              "#{m}\n  #{route_code(route_options)}\n"
          end
        end
      end

      class Destroy
        def route(route_options)
          logger.remove_route route_code(route_options)
          to_remove = "\n  #{route_code(route_options)}"
          gsub_file 'config/routes.rb', /(#{to_remove})/mi, ''
        end
      end

    end
  end
end

More or less copied from the existing route_resources action found in commands.rb, this works as follows: If we call script/generate then the route action method in the Create command class is called. This uses gsub_file to look for the “sentinel” or target area of the file and replaces it with the sentinel + the new route code. I also use the “logger” method to display log messages using the Rails::Generator::SimpleLogger class. Here’s what it looks like on the command line:

$ ./script/generate version ../build-info/version.txt
      create  app/controllers/version_controller.rb
       route  map.version 'version',
                 :controller => 'version',
                 :action => 'display_version'

If script/destroy is called, then the second route implementation in the Destroy class is called. This uses gsub_file to remove the route code. Finally, the “route_code” method returns the route code that we want to generate or remove from routes.rb. This time on the command line we get:

$ ./script/destroy version ../build-info/version.txt 
remove_route  map.version 'version',
                 :controller => 'version',
                 :action => 'display_version'
          rm  app/controllers/version_controller.rb

→ 2 comments Tags:

Tutorial: How to write a Rails generator

August 23, 2009 · 1 comment

I’ve been working on a generator called ViewMapper recently that allows you to create view scaffolding from an existing model. I found writing a Rails generator to be somewhat confusing and hard to do: Where do I need to put my generator class? What does it need to be called? How does it work? This article will show step by step how to write your own Rails generator from scratch – hopefully it will save you some time if you ever need to write your own.

First let's think of some sample Rails code that I might want to generate as an example. This is admittedly contrived, but it’s short and simple enough to show here while still interesting enough to illustrate how a generator would work. Let’s suppose my Capistrano deployment scripts wrote the last commit information from Git into a file called “version.txt” in a folder RAILS_ROOT/../build-info:

commit 217d9bd1d1d508a0f7f1e7afa2b489b130196e33
Author: Pat Shaughnessy <pat@patshaughnessy.net>
Date:   Wed Aug 5 07:45:01 2009 -0400

Now with a controller like this I could display the info, helping me keep track of what version is running on a given development development, QA or production server:

class VersionController < ApplicationController
  VERSION_FILE = '../build-info/version.txt'
  def display_version
    path = File.join(RAILS_ROOT, VERSION_FILE)
    render path
  end
end

If I add a route to this action like this in config/routes.rb:

map.version 'version', :controller => 'version', :action => 'display_version'

…then I’ll get the Git last commit info by just opening http://servername/version in my browser. Obviously it would be simpler to just put the version.txt file under the public folder… but let’s continue with this contrived example for now. Now let’s say I have 5 or 10 different Rails apps, and that I’d like to add the same controller and route to each one. Let’s write a simple Rails generator that would make this easy to do, called the “version” generator. Here’s how to do it…

Step 1: A skeleton Rails generator

All Rails generators are derived from a class called “Rails::Generator::Base.” You can find the source code for this file in vendor/rails/railties/lib/rails_generator/base.rb (link is to the Rails 2.3.3 version). Definately read the source code; there is helpful documentation in the comments and understanding the base class code is indispensible if you plan to use it for your generator.

Here’s an empty, skeleton Rails generator that we can use as a starting point:

class VersionGenerator < Rails::Generator::Base
  def manifest
    record do |m|
      # Do something
    end
  end
end

The name of the class is important: since we want a “version” generator we need to create a class called “VersionGenerator.” The other important detail here is where this class is located: we need to put it in lib/generators/version/version_generator.rb. The reason for all of this is that the code called by script/generate takes the generator name and looks for the corresponding generator class in this location. We can check that our class is being found by just running script/generate with no parameters, like this:

$ ./script/generate 
Usage: ./script/generate generator [options] [args]
…etc…
Installed Generators
  Lib: version
  Rubygems: cucumber, feature, install_rubigen_scripts, integration_spec, rspec, rspec_controller, rspec_model, rspec_scaffold, session
  Builtin: controller, helper, integration_test, mailer, metal, migration, model, observer, performance_test, plugin, resource, scaffold, session_migration

Note that our new “version” generator is listed as “Lib: version,” in bold above. The other generators found on my system were either part of Rails or else part of a gem or plugin. If you don’t see your generator in this list, or if it has the wrong name then double check your folder name: it should be lib/generators/XYZ or lib/generators/version in this example. After you finish developing your generator you'll want to move it into a plugin or gem so it can be reused in different apps on your machine; then "version" would appear in the "Rubygems" list for example.

If we try running our empty generator, we’ll get no output or changes; not a surprise:

$ ./script/generate version

If you see an error like this:

$ ./script/generate version
/usr/local/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:31:
in `gem_original_require': no such file to load --
/path/to/myapp/lib/generators/version/version_generator.rb (MissingSourceFile)

Then you have the wrong file name… it needs to be XYZ_generator.rb, or version_generator.rb in this example. Or if you see:

Missing VersionGenerator class in
  /path/to/myapp/lib/generators/version/version_generator.rb

… then you have the wrong class name inside this file. It needs to be XYZGenerator, or VersionGenerator in this example.

Step 2: Creating code using an ERB template

The way Rails generators work is by creating code using ERB, in just the same way that view code generates HTML for a Rails web site. To see how this works and to take the next step towards writing our version generator, let’s create a new template file for our version controller. Create a new text file called “controller.rb” and put it into this folder: lib/generators/version/templates/controller.rb; in other words, into a new subfolder under version called “templates.” Next, paste in the controller code from above:

class VersionController < ApplicationController
  VERSION_FILE = '../build-info/version.txt'
  def display_version
    path = File.join(RAILS_ROOT, VERSION_FILE)
    render path
  end
end

Now if we add this line to our manifest method back in the VersionGenerator class:

def manifest
  record do |m|
    m.template('controller.rb', 'app/controllers/version_controller.rb')
  end
end

… a new code file called version_controller.rb will be generated inside of the app/controllers folder in our app when we run the generator:

$ ./script/generate version
    create  app/controllers/version_controller.rb

If you take a look at this new version_controller.rb file, you’ll just see the code we pasted into the controller.rb template file. Here’s what happened when we ran script/generate version:

  • The Rails code called by script/generate looked for the “version” generator, by looking for a class called “VersionGenerator” in the lib/generators/version folder, among other places.
  • It called the “manifest” method in VersionGenerator. Using the “record” utility method, a series of actions are recorded that the Rails generator base classes will execute later. In our case, there’s only one action: “template.”
  • The template action indicates that the Rails generator code should run an ERB transformation to generate new code, using the “controller.rb” file. The generated code will then be copied to the app/controllers/version_controller.rb file.

To make our generator interesting, let’s provide a parameter to it that indicates the path of the file containing the version information, instead of hard coding “version.txt.” To do that we can take advantage of another Rails built in class called “Rails::Generator::NamedBase”. If we derive our VersionGenerator class from NamedBase instead of Base, then we’ll get the ability to take a name parameter from the script/generate command line for free. Let start by changing our base class in version_generator.rb:

class VersionGenerator < Rails::Generator::NamedBase

Now if you run the generator, you’ll get some helpful usage information:

$ ./script/generate version
Usage: ./script/generate version VersionName [options]
Etc…

Here the NamedBase class has written the usage info, explaining that an additional parameter is now expected. If we re-run the generator and add the version file name as a parameter, here’s what we’ll get:

$ ./script/generate version ../build-info/version.txt
   identical  app/controllers/version_controller.rb

This shows a helpful feature of Rails generators: they check if you’re about to overwrite some existing code with the new, generated code. In this case, since we already had a controller called “version_controller.rb” the Rails generator code displayed the “identical” message. Our new generated controller is identical to the previous one since we haven’t used the version name parameter yet. To take advantage of the version file name parameter and our new NamedBase base class, we need to add some new code to VersionGenerator, in bold:

class VersionGenerator < Rails::Generator::NamedBase
  attr_reader :version_path
  def initialize(runtime_args, runtime_options = {})
    super
    @version_path = File.join(RAILS_ROOT, name)
  end
  def manifest
    record do |m|
      m.template('controller.rb', 'app/controllers/version_controller.rb')
    end
  end
end

The way this works is:

  • The NamedBase base class parses the script/generate command line, expecting an additional parameter which indicates the name of some object.
  • This name is provided in "name," an instance variable/attribute of the NamedBase class.
  • The VersionGenerator class assumes "name" indicates the relative path of a version file, determines the full path of the file and then saves the full path in the @version_path attribute.

I mentioned above that Rails generators work by running an ERB transformation; to try using ERB in this example, we just need to refer to the new “version_path” attribute inside the controller.rb template file, like this:

class VersionController < ApplicationController
  VERSION_PATH = '<%= version_path %>'
  def display_version
    render VERSION_PATH
   end
end

The “version_path” variable in this template refers to the version_path attribute of the VersionGenerator class above. I’ve also removed the File.join line from here since we now handle that in the generator itself. If we run the generate command and provide the name of the version file, we’ll get this result:

$ ./script/generate version ../build-info/version.txt
overwrite app/controllers/version_controller.rb? (enter "h" for help) [Ynaqdh] Y
       force  app/controllers/version_controller.rb

As I mentioned above, Rails checks whether you’re about to overwrite some existing code file. Since in this case our controller code file is now different that what we had before (VERSION_PATH and not VERSION_FILE; no File.join, etc.) we get an overwrite warning. I just entered “Y” to overwrite the old file.

If we look at our new controller, app/controllers/version_controller.rb, we now have:

class VersionController < ApplicationController
  VERSION_PATH = '/path/to/myapp/../build-info/version.txt'
  def display_version
    render VERSION_PATH
   end
end

We’ve successfully generated a new VersionController class, based on a parameter passed to our “version” generator. While this is a good start, to be able to use this action in our app we still need to add a route to config/routes.rb. It turns out this is a lot harder to do with a generator, since we will need to insert a new line of code in an existing file, and not just generate a new routes.rb file. We will also need to worry about how to remove the route line when script/destroy is run. It’s possible to do all of this with a Rails generator, but will require a more thorough understanding of how Rails generators actually work. I’ll explain all of that in my next post…

→ 1 comment Tags:

← Generating view scaffolding code for existing models Rails generator tutorial part 2: writing a custom manifest action →