Ruby, Smalltalk and Class Variables

December 17th 2012 —   Comments and   Reactions
Many of the ideas behind Ruby’s object model
were developed for Smalltalk in the 1970s.

A couple weeks ago an article by Ernie Miller got me interested in how class variables work in Ruby. After doing a bit of research, I found that class variables have been a perennial source of confusion. In fact, John Nunemaker wrote an article called Class and Instance Variables In Ruby way back in 2006 that still applies today. The fundamental problem with class variables in Ruby is that they are shared among a class and all of its subclasses – as John explained six years ago, this can lead to confusion and unexpected behavior.

But for me the interesting question here is: “Why?” Why does Ruby share a single value across all of the subclasses? Why have a distinction between “class variables” and “class instance variables?” Where do these ideas come from? It turns out the answer is simple: class variables in Ruby work the same way class variables work in a much older language called Smalltalk. Smalltalk was invented in the early 1970s by the renown computer scientist Alan Kay and a group of his colleagues working at the Xerox PARC laboratory. With Smalltalk, Alan Kay didn’t just invent a programming language; he also conceived of the entire concept of object oriented programming (OOP) and implemented it for the first time. While not in very widespread use now, Smalltalk has influenced many other object oriented programming languages that are used widely today – most importantly Objective C and Ruby.

Today I’m going to look at how class variables work in Smalltalk, and compare and contrast that against how they work in Ruby. As you’ll see, I found that class variables aren’t the only idea Ruby took from Smalltalk. Much of Ruby’s object model design was taken from Smalltalk as well.

Class variables in Ruby

First, let’s quickly review what a class variable is, and how they work in Ruby. Using John Nunemaker’s example from 2006, here’s a simple Ruby class, Polygon, that contains a single class variable, @@sides:

class Polygon
  @@sides = 10
  def self.sides
    @@sides
  end
end

puts Polygon.sides
=> 10

This is simple enough: @@sides is a variable that any class or instance method of Polygon can access. Here the sides class method returns it. At a conceptual level, internally Ruby associates the @@sides variable with the same memory structure used to represent the Polygon class:

The confusion comes in when you define a subclass; again here is another one of John Nunemaker’s examples:

class Triangle < Polygon
  @@sides = 3
end

puts Triangle.sides
#=> 3
puts Polygon.sides
#=> 3

Notice both class variables, Triangle.sides and Polygon.sides, were changed to 3. In fact, internally Ruby creates a single variable that both classes share:

I may write in more detail about the details of Ruby’s internal implementation of class variables in an upcoming blog post, but for today I’ll just use these very simple diagrams. Instead, now let’s switch gears and learn more about Smalltalk….

What is Smalltalk?

As I said above, Alan Kay invented Smalltalk along with object oriented programming while working at Xerox PARC in the early 1970s. This is the same laboratory that also invented the personal computer, the graphical user interface, and the Ethernet among many other things. Object oriented programming actually seems to be one of their less important inventions!

In Smalltalk, Kay introduced terminology and ideas that we all take for granted today. Every value in Smalltalk, including language constructs such as code blocks, is an object. A Smalltalk program consists of these objects and the way they interact; to call a particular Smalltalk function, you “send a message” to the object that implements that function. In Smalltalk, functions are known as “methods.” An object implements a series of methods. All of this should sound very familiar, of course.

In the 1970s, Alan Kay envisioned a laptop/tablet he called the
“Dynabook” would run Smalltalk. He and his team actually built a
computer called the “Interim Dynabook” and used it to teach
programming to middle school children.

From the very beginning, Kay’s conception of OOP included the idea of an object’s “class.” An object’s class described a series of behaviors (methods) each instance of that class exhibited. Smalltalk also implemented the concept of polymorphism, which allows the developer to define “subclasses” that share the behaviors of their “superclass.” All of these terms we use often today were coined by Kay and his colleagues 40 years ago.

Smalltalk, however, is more than just a programming language; it’s an entire graphical development environment. I think of Smalltalk as a precursor to Visual Studio or XCode, invented before Microsoft or Apple even existed, in a world where computers were found only in academic or government settings! One other impressive goal Alan Kay and the Smalltalk team had from the beginning was to use their visual environment as a teaching tool for school children. It’s a truly amazing story.

To learn more about the history and origin of Smalltalk, I would highly recommend reading The Early History Of Smalltalk (html or original pdf or easier to read pdf, but missing some diagrams), a retrospective account Kay wrote later in the 1990s. It’s a fascinating narrative of how Kay and his colleagues borrowed ideas from even earlier, but with the combination of hard work, creativity and pure talent managed to take a large step forward and revolutionize the computer science world of their day, and ours.

Alan Kay created the first working version of Smalltalk in 1972 – in his own words, here is how it happened:

I had expected that the new Smalltalk would be an iconic language and would take at least two years to invent, but fate intervened. One day, in a typical PARC hallway bullsession, Ted Kaehler, Dan Ingalls, and I were standing around talking about programming languages. The subject of power came up and the two of them wondered how large a language one would have to make to get great power. With as much panache as I could muster, I asserted that you could define the "most powerful language in the world" in "a page of code." They said, "Put up or shut up." Ted went back to CMU but Dan was still around egging me on. For the next two weeks I got to PARC every morning at four o'clock and worked on the problem until eight, when Dan, joined by Henry Fuchs, John Shoch, and Steve Purcell showed up to kibbitz the morning's work. I had originally made the boast because McCarthy's self-describing LISP interpreter was written in itself. It was about "a page", and as far as power goes, LISP was the whole nine-yards for functional languages. I was quite sure I could do the same for object-oriented languages….

Here Kay referred to John McCarthy, who invented LISP about 10 years earlier. It took Kay only eight early mornings of work to finish the first version of Smalltalk:

The first few versions had flaws that were soundly criticized by the group. But by morning 8 or so, a version appeared that seemed to work….

I wish I could be as creative, dedicated and productive as Alan Kay and his Xerox PARC colleagues were 40 years ago!

Class variables in Smalltalk

To find out how class variables actually work in Smalltalk, I installed GNU Smalltalk, a command line based version of the language which is easy to download and run on a Linux box. Initially I found Smalltalk to be very strange and unfamiliar; it’s syntax seems a bit odd and weird at first glance. For example, you need to remember to end each command with a period, and also to define a method you only need to specify a list of arguments… without a method name! I suppose the first argument is the method name or vice-versa. But after a couple of days I became accustomed to the idiosyncratic syntax, and the language began to make more sense to me.

Here is the same Polygon class again – now I have Smalltalk on the left, and Ruby on the right:

Object subclass: Polygon [
  Sides := 10.
]

Polygon class extend [
  sides [ ^Sides ]
]

Polygon sides printNl.
=> 10

class Polygon
  @@sides = 10
  def self.sides
    @@sides
  end
end


puts Polygon.sides
#=> 10

Here’s a quick explanation of what the Smalltalk code does:

  • Object subclass: Polygon – this means send the subclass message to the Object class and pass in the name Polygon. It creates a new class, which is a subclass of the Object class. This is analogous to class Polygon < Object in Ruby. Of course, in Ruby specifying Object as the superclass is unnecessary.

  • Sides := 10. – this declares a class variable Sides, and assigns it a value. Ruby instead uses the @@sides syntax.

  • Polygon class extend – this “extends” the Polygon class; i.e., it opens up the Polygon class and allows me to add a class method. In Ruby I use class Polygon; def self.sides.

  • The printNl method prints a value to the console; it works the same way as puts in Ruby, except printNl is a method of the Sides object. Imagine calling @@sides.puts in Ruby!

Aside from the superficial syntax differences, if you take a step back and think about this, it’s striking how similar Smalltalk and Ruby really are! Not only do both languages share the same class variable concept, but I wrote the Polygon class, declared a class variable and printed it out exactly the same way in both languages. In fact, you can think of Ruby as a newer version of Smalltalk with a simpler, easier to use syntax!

As I said at the top, Smalltalk shares class variables among subclasses the same way Ruby does. Here’s how I would declare the Triangle subclass in Smalltalk and Ruby:

Polygon subclass: Triangle [
]
Triangle class extend [
  set_sides: num [ Sides := num ]
]
Polygon sides printNl.
=> 10 


class Triangle < Polygon
  def self.sides=(num)
    @@sides = num
  end
end

puts Triangle.sides
#=> 10

Here I declare the Triangle subclass and a method to set the class variable’s value. Now let’s try changing the value of the class variable from the subclass:

Triangle set_sides: 3.
Triangle sides printNl.
=> 3
Triangle.sides = 3
puts Triangle.sides
=> 3

No surprise; by calling the set_slides class method (sides= in Ruby) I can update the value. But notice since both Polygon and Triangle share the same class variable, it’s changed for Polygon also:

Polygon sides printNl.
=> 3

puts Polygon.sides
=> 3

Again, we’ve seen Ruby and Smalltalk behave in exactly the same way.

One way the two languages differ is that Smalltalk does allow you to create a separate class variable for each subclass, if you want. By repeating the class variable definition and the accessor class method in both classes they become separate variables, at least in GNU Smalltalk which I was using:

Object subclass: Polygon [
  Sides := 10.
]

Polygon class extend [
  sides [ ^Sides ]
]

Polygon subclass: Triangle [
  Sides := 3.
]

Triangle class extend [
  sides [ ^Sides ]
]

Polygon sides printNl.
>= 10

Triangle sides printNl.
>= 3

This isn’t true in Ruby; as we saw above @@sides will always refer to the same value.

Class instance variables

In Ruby if you want to keep a separate value for each class, then you need to use a class instance variable instead of a class variable. What does this mean? Let’s take a look at another one of John Nunemaker’s examples:

class Polygon
  def self.sides
    @sides
  end
  @sides = 8
end

puts Polygon.sides
#=> 8

Now since I used the @sides notation instead of @@sides, Ruby created an instance variable instead of a class variable:

Conceptually there’s no difference, until I create the Triangle subclass again:

class Triangle < Polygon
  @sides = 3
end

Now each class has its own copy of the value @sides:

Now let’s try the same thing in Smalltalk. In Smalltalk to declare an instance variable you call the instanceVariableNames method on a class:

Object subclass: Polygon [
]

Polygon instanceVariableNames: 'Sides '!

Polygon extend [
  sides [ ^Sides ]
]

class Polygon
  def sides
    @sides
  end
end

Here I’ve created a new class Polygon, a subclass of Object. Then I send the instanceVariableNames message to this new class, telling Smalltalk to create a new instance variable called Sides. Finally, I reopen the Polygon class and add the sides method to it. Again I show the corresponding Ruby code on the right.

However, here Sides and @sides are instance variables of Polygon objects, and not of the Polygon class. To create a class instance variable in Smalltalk, you instead have to send the class message to Polygon first before calling instanceVariableNames or extend, like this:

Object subclass: Polygon [
]

Polygon class instanceVariableNames: 'Sides '!

Polygon class extend [
  sides [ ^Sides ]
]

class Polygon
  def self.sides
    @sides
  end
end

Again, notice that the Smalltalk and Ruby code snippets are really just two different ways of expressing the same commands. In Smalltalk you say Polygon class extend [ sides… while in Ruby you say class Polygon; def self.sides. To me Ruby seems to be a more succinct version of Smalltalk.

Metaclasses in Smalltalk and Ruby

This diagram, taken from Alan Kay’s fascinating article The Early
History Of Smalltalk, resembles the class hierarchy Ruby would
use 20 years later!

Let’s take another look at the line of code I used above to create an instance variable in Smalltalk:

Polygon instanceVariableNames: 'Sides '!

Translating from Smalltalk into English, this means:

  • Take the Polygon class,

  • send it a message called instanceVariableNames,

  • and pass the string Sides as a parameter.

Again, this is how you define instance variables in Smalltalk. That is, now when I create instances of the Polygon class, they will each have a Sides instance variable. Saying the same thing in a different way, to give all polygon instances an instance variable, I call a method on the Polygon class.

As I explained above, to create a class instance variable in Smalltalk you have to use the class keyword, like this:

Polygon class instanceVariableNames: 'Sides '!

This code literally means: call the instanceVariableNames method on the Polygon class’s class. Following the same pattern, now all instances of the Polygon class will contain a class instance variable. But what is the “class of the Polygon class” in Smalltalk? What does this mean? Spending just a moment at the GNU Smalltalk REPL we can find out:

$ gst
GNU Smalltalk ready

st> Polygon printNl.
=> Polygon

st> Polygon class printNl.
=> Polygon class

I first display the Polygon class object, and I get “Polygon”. Displaying the class of the Polygon class, I get “Polygon class.” But what type of object is this? Let’s call class on it:

st> Polygon class class printNl.
=> Metaclass

Ah… so the class of a class is a Metaclass. Above, when I called instanceVariableNames to create a class instance variable, I was actually using the Polygon metaclass, an instance of the Metaclass class.

Here’s a diagram showing how these classes are all related in Smalltalk:

By now, it should be no surprise if I tell you internally Ruby uses the same model. Here’s how classes work inside of Ruby:

In Ruby whenever you create a class, Ruby internally creates a corresponding new class called the “metaclass.” Unlike Smalltalk, Ruby doesn’t use this for class instance variables, but only to keep track of class methods. Also, Ruby doesn’t have a Metaclass class, but instead all metaclasses are simply instances of the Class class.

In Ruby the metaclass is a hidden, mysterious concept. Ruby silently creates it without telling you and doesn’t expose the metaclass in the language directly. In Smalltalk, however, the metaclasses are not hidden at all and instead play a large role in the language. Creating a class instance variable, as I did above, is just one example of using a metaclass in Smalltalk. Another good example is the way you add class methods by calling extend.

When you ask for a class’s class in Ruby, you simply get Class. Ruby doesn’t tell you about the metaclass:

$ irb
> class Polygon; end
> Polygon.class
Class

To see a Ruby metaclass, you have to use a trick instead:

$ irb
> class Polygon
>   def self.metaclass
>     class << self
>       self
>     end
>   end
> end
=> nil
> Polygon.metaclass
=> #<Class:Polygon>

“#<Class:Polygon>” is the metaclass of Polygon. This syntax means the metaclass is “an instance of Class for the Polygon class,” or the metaclass for Polygon.

* Quoted text and images from: The Early History Of Smalltalk, by Alan Kay, © 1993 Association for Computing Machinery