BlogFish

We are in the process of testing the new version of MeinProf.de and crossed a very interesting bug.

We are using fragment caching to speed up the serving of the university homepages. So far this is working great and helps a lot with coping with extra load.

The university homepages are served by unis/show.html.erb:

  ...
  <%= render :partial => 'shared/user_info' %>
  ...
  <% cache "uni_page_homepage_#{@uni.id}", :expires_in => 1.hour do %>
    <%= render :partial => 'unis/header' %>
    <%= render :partial => 'tabs_header' %>
    <div class="uni_desc">
      <div class="tab-content">
       ...
       <!-- a lot of uni content here-->
  <% end %>

The partial unis/_header.html.erb looks like this:

  ...
<% content_for("breadcrumb") do %>
  <%= uni_breadcrumb(uni, department) %>
  <div class="breakcrumb">
    ...
  </div>
<% end %>

<div class="subcolumns">
  <h1><%=h uni.name %></h1>
  <div class="c08l badge">
  <p>
    <strong>Homepage:</strong>....
   ....
  </p>

So far so good. When we deployed to our staging host we noticed that the university header was not always right. On the first request it was correct and on every subsequent request the header was missing the breadcrumb. Errors like this sounds like caching issues and it took us a while to figure it out.

The problem is the content_for block inside the partial. Once you think about it, it makes a lot of sense. During the first request the show-template gets evaluated. It calls the partial and the partial executes the content_for block. This block access the template binding/variables to inject content. The partial finishes rendering and stores the rendered content in memcached. Everything fine.

On the next request Rails loads up the action and renders the view. During the view rendering the fragment will be loaded from memcached. No bug here.

So why is the breadcrumb not showing up? Because the content_for block will not be executed. It is a side effect of the partial-rendering and the partial will not be rendered once it is cached. Only the resulting HTML-fragment will be loaded from the cache but the code-block is not executed. This means that there is not call to inject extra content to the main view.

So before you wrap some view code in fragment caching remember: do not fragment cache content_for-blocks. They will not be called. Always check cached views for content_for-blocks and move them outside the cache-call.

I know, it is a bit late, but here are my slides from Scotland on Rails:

The slides are also available as a PDF download: Advanced Deployment

Scotland on Rails was again a great conference. A very interesting crowd in a very nice city. I'm looking forward to next year!

As a consultant for Peritor I often review other peoples code. What I often encounter and always address is custom logic in Rake tasks.

Rake is perfect for automating tasks and making calls into your application from the command line. People often use it for doing maintenance, executing periodic jobs, or starting and stopping services. Nothing wrong with that, this is what Rake in Rails apps is for.

What I resent is having all this logic in the Rakefile, either inline in the task body or as a method in the Rakefile.

  # Rakefile
  
  task :ping_stale_users do
    stale_users = User.find(:all, 
                            :conditions => ["last_login < ? ", Time.now - 3.months])
    ...
    ...
    stale_users.each do |user|
      if user.check_something
        user.mail_invite
      end
    end
  end

This is just wrong for the following reasons:

• Testing is hard: methods and Rake tasks are hard to test. The environment is hard to create and you may have to shell-out to call rake.
• Reusing is hard: you cannot easily reuse this functionality from the console or an admin controller.
• POLA: Rakefile are not were people look for domain logic

The solution is simple and yet it seems underused. Just put your domain logic into a model or library and call it from the Rake task. This way it is much easier to test, reuse, and read.

  # Rakefile
  
  task :ping_stale_users do
    User.ping_stale_users
  end
  
  # User.rb
  def self.ping_stale_users
    stale_users = User.find(:all, 
                            :conditions => ["last_login < ? ", Time.now - 3.months])
    ...
    ...
    stale_users.each do |user|
      if user.check_something
        user.mail_invite
      end
    end
  end

Of course the method in the model can be cleaned up a bit more but this is not the point. The point is that we removed domain logic from the Rakefile and made it shorter and easier to understand. The logic is now somewhere where we can test and reuse it.

I just released Webistrano 1.4. Webistrano is a tool for managing Capistrano deployments and offers a rich web UI. It lets you manage projects with their stages and keep track who deployed which version to which servers.

Webistrano 1.4 brings many new features that make deployment easier. The most prominent are:

• Recipe versioning - recipes are now versioned so that you can keep track of changes
• Project cloning - you can now create a template project and clone it over and over again
• Array parameters - support for arrays as values for configuration parameters
• CAS-auth support - Single Sign-On support by delegating authentication to a CAS server. See the documentation
• Enhaced UI - nicer overviews of deployments and many small fixes
• Cancel deployments - a running deployment can now be canceled by Webistrano. The running Capistrano instance will be killed so use this feature with care
• Track deployed revisions - Webistrano will track which revision was deployed. This way you always know which version is running where
• Updated packages - Rails 2.1 and Capistrano 2.5.0

Apart from that some smaller enhancements and fixes went into the 1.4 release. See the CHANGELOG for a complete list.

Further, there is now a Webistrano mailing list at GoogleGroups.

Go get Webistrano from the project homepage as a download or checkout the source:

Download: webistrano-1.4.zip (3.4 MB)

# Development version:

svn co http://labs.peritor.com/svn/webistrano/trunk

# Stable version:

svn co http://labs.peritor.com/svn/webistrano/branches/1.4

RailsRumble 2008 is over - after 48 hours of hacking I'm proud to announce our contestant:

GoodBad.me - Twitter Till Judgement Day!

GoodBad.me is a twitter mashup that Thomas Metschke and I build over the last two days. Once you follow our bot on twitter, we subscribe to your tweets. We collect all your good and bad deeds by searching for #goodme or #badme (or the shorter #gd #bd) in your tweets.

We keep track of your deeds and generate stats about you. Your deed balance decides your character level. Are you just a boy scout or do you strive for beeing friends with Goldfinger?

All those questions can be answered at http://goodbad.me.

The idea is that you keep tweeting as always and by adding the #goodme tag you let the world know that you did something good.

In later iterations we want to allow others to tag your deeds/tweets and maybe allow you to gain points by donating to charity.

The site is online for a couple of hours and we already have several active users. If you want to keep track of your day's work, go join!

If you like the idea and our implementation, please vote for us!

Recently there have been again some articles on block and capture helpers in Rails (e.g. this one).

A block/capture lets you write nicer looking helper functions in Rails, especially if you want to render some HTML before and after a given piece of code. Typical use cases are styled blocks (HTML) that should surround your items.

Assume we have projects that should be displayed on an overview page. You want to render a nice box with the project title and the project description for each project. Often you end up with partials like this:

<div class="box">
  <span class="box_title"><%=h project.title %></span>
  <div class="box_body">
    <%=h project.description %>
  </div>
</div>

When you want to re-use this partial for models other than a project and further, sometimes render different HTML in the box_body you create a helper function. The most common approaches are using a start/end combination of helpers and a block/capture helper.

The idea of the start/end helper combination is that you call one helper to create/print all the HTML before your item rendering, render your item, and then call another helper to close all the tags and finish the box:

# application_heper.rb
def start_box(title)
  out = "<div class='box'>"
  out += "<span class='box_title'>#{h(title)}</span>"
  out += "<div class='box_body'>"
  out
end

def end_box
  "</div></div>"
end

# view

<%= start_box(project.title) %>
  <%=h project.description %>
  <--! add custom HTML here -->
<%= end_box %>

This works fine but is a bit un-elegant. Further, you can forget the end_box-function and nothing would complain. Your HTML would just be broken.

A much nicer looking solution is the block/capture helper:

# application_helper.rb
def box(title, &block)
  out = "<div class='box'>"
  out += "<span class='box_title'>#{h(title)}</span>"
  out += "<div class='box_body'>"
  out << capture(&block) if block_given?
  out += "</div></div>"
  block ? concat(out, block.binding) : out
end

# view
<% box(project.title) do %>
  <%=h project.description %>
  <--! more custom HTML here -->
<% end %>

So we are using a block here to pass our box content. The block is created with the do/end style and then passed as an argument to the helper function. The helper function then creates some output, evaluates the block, and then continues to print some HTML strings . By using the block syntax we can never forget to "close" a box as the Ruby interpreter would complain about the missing end keyword. Further, in the helper method we can choose to not render anything. This technique is especially useful for administrative links.

I'm a big fan of the block/capture helper and favor its syntax any time above the start/end way. I just wanted to post here about a disadvantage of the block/capture way that you should be aware of.

During a recent performance analysis for a client I profiled their root page which is an overview page with many HTML-boxed elements. I noticed that it rendered really slow, espically if you had many items on the overview page. After a bit of digging into the rendering, we found out that their block/capture helpers were eating 70-80% of rendering time. The problem is that creating a block (aka a closure), storing its binding (scope and surrounding variables) and then passing around this closure is expensive compared with the "pure" string output helper.

How much more expensive can be shown by this benchmark. I created a new test Rails project with two actions. Each action displays a project, one uses the start/end helper and the other one the block/capture helper. On the X-axis you see the number of helper calls inside the view and on the Y-axis you see the number of requests per second (a reported by ruby script/performance/request -n 1000 -b).

So the block/capture helper style is a lot slower than the simple start/end helper. But it only matters if you use it a lot on a page. With 250 calls on a page, the block/capture style helper has only 20% of the requests per second that the start/end style helper can deliver. 250 calls may seem like a lot but in my case the page displayed 50 boxed items and the page had several other boxed content (e.g. login, stats, ads). If you then add a lot of variables in the closures scope (that need to be included in the binding), the page rendering can get really slow.

I'm not arguing in general against the block/capture, I really like the resulting syntax and flexibility. But as often with syntactic sugar and nicer looking code, you trade it for performance. Most of the time this should not matter but when it does, it shows!.

DISCLAIMER: Those number are not statistically valid and you should not try to get too much out of them. Use them as a hint were to look for slow rendering.

I'm proud to announce Webistrano 1.3!

Webistrano is a Web UI for managing Capistrano deployments. It lets you manage projects and their stages like test, production, and staging with different settings. Those stages can then be deployed with Capistrano through Webistrano.

The 1.3 adds several new shiny features to Webistrano that make deployment easier:

• Better Git support through Capistrano 2.2
• Support for Phusion Passenger / mod_rails
• Ability to temporary disable hosts for a deployment
• A command-line interface with script/deploy
• A simple permission system

The complete changelog is available through the Webistrano project site.

One very often demanded feature is the ability to temporary disable a host for a deployment. This is helpful when you want to deploy a stage without changing the stage configuration even if one or more hosts are down:

Another scenario is when you want to execute a task only on a limited set of servers.

The script/deploy command is a nice little gem, especially useful if you want to script Webistrano:

$ ruby script/deploy 
Usage: deploy [options] project stage
    -h, --help                       This message
    -e, --environment=ENV            RAILS_ENV for Webistrano (default: production)
    -u, --username=NAME              Webistrano username to use (default: admin)
    -t, --task=NAME                  Capistrano task to invoke (default: deploy)
    -d, --description=TEXT           Deployment comment for Webistrano records

Further, Webistrano now offers built-in tasks for managing mod_rails deployments. It will override the default deploy tasks and ask for the necessary configuration entries so that using mod_rails becomes even easier.

Upgrading from previous releases is very easy, see the Upgrading wiki page.

Webistrano 1.3 can be downloaded here. Webistrano is BSD-licensed and the project site is open for everybody. Please see the project page for more documentation and screenshots. There are even some screencasts.

After being one week on the road, I'm finally back home again. Last week I presented at RubyFools Copenhagen and Scotland on Rails and therefore traveled a lot.

This was my first time in Copenhagen and it seemed like a very nice city. The conference venue was very nice, a brand new university building with a big hall. My talk about Rails on AWS was well received. I've added some information about the new EC2 features like elastic IPs and availability zones. A video of the presentation should hopefully soon be available. After the sessions I've spend some time chatting with Matz about Ruby/JRuby/Rubinius and his work in Japan.

Next I traveled to Edinburgh for Scotland on Rails. Edinburgh feels like my second home town as I've studied and worked there for a while. The conference was very good organized and had a different Ruby crew there. Most of the RubyFools Copenhagen folks went straight to RubyFools Oslo, so there were many new faces in Edinburgh.

I gave a presentation about Rails Patterns, typical problems of real-life Rails production sites and solutions/patterns. Afterwards I had a couple of nice conversations with other developers and their experiences with similar situations.

Apart from the great conference, I had a chance to spend some time in Edinburgh and catch-up with some people there.

My slides can be found here:

RubyFools Copenhagen: Rails on AWS (PDF)

Scotland on Rails: Rails Patterns (PDF)

| View | Upload your own

| View | Upload your own

RubyFools Copenhagen: Rails on AWS (PDF)

Scotland on Rails: Rails Patterns (PDF)

Just a small note for people using Capistrano/Webistrano and the remote_cache deployment strategy.

set :deploy_via, :remote_cache

The remote_cache strategy creates a cached-copy directory in your #{deploy_to}/shared base. It then checks out the coe once and in contrast to the default deployment strategy. After the initial checkout subsequent deployments will do a `svn up` and copy the result over to #{deplot_to}/releases/.

Using remote_cache your deployments are usually a bit faster but there is a catch. If you ever change the repository variable, e.g. because you switch to another tag of move the stable branch, your deployments will either fail or do not completely update.

This is due to the fact, that the new deployment does a

$ svn up -rYOUR_REV http://svn.example.com/svn/branches/my_new_branch

on the cached copy. With a different branch or tag than the one the `svn checkout` command was executed with, this will not work. In order to fix it, just delete the cached-copy directory. It will be re-created on the next deployment.

$ rm -rf /path/to/deploy/shared/cached-copy

The conference season is starting again for me and I wanted to note where I will be/speak during the next couple of weeks.

First, there is Ruby Fools Copenhagen (April 1st and 2nd) where I will speak in the Ruby Performance track about Rails on AWS and how to leverage EC2, S3, and SQS in your application. The lineup at Ruby Fools looks really good with speakers like Glenn Vanderburg, Michael Koziarski, Evan Phoenix, Dr. Nic Williams, Dave Thomas, and Matz himself. Unfortunately I will not have too much time in Copenhagen as I have to leave early for Scotland on Rails in Edinburgh.

I'm really looking forward to be in Edinburgh again. After living, studying, and working there it feels like a second home. At Scotland on Rails (April 4th and 5th) I will talk about Rails Patterns: typical problems and scenarios in Rails applications like asynchronous operations (image processing, calculations, ..), authentication or deployment and common solutions and best practices.

In Mai I will be at Linuxtag 2008 in Berlin and hopefully talk about Ruby on Rails Security, but this talk has not been confirmed yet. Further, there is a chance that I will be speaking a the iX Cebit Forum 2008 about our internal Software Development Process and Agile Development.

Design Patterns in Ruby by Russ Olsen is an introduction to Design Patterns. It covers 14 out of the 23 patterns of the GoF Design Patters: Elements of Reusable Object-Oriented Software book and adds three Ruby-related patterns. The book examines each pattern in general, shows how it applies to a dynamic language like Ruby and explains when to use or not use the pattern. The fact that the book is written in an informal style with lots of examples makes it really easy to read and follow. I really liked the format of the pattern examination and find it an excellent overview of and introduction to Design Patterns. The covered GoF patterns (Template Method, Strategy, Observer, Composite, Iterator, Command, Adapter, Proxy, Decorator, Singleton, Factory Method, Abstract Factory Method, Builder, and Interpreter) are the most important ones and are easy to apply in Ruby. Especially the chapter on Interpreter made a very good job of explaining a widely under-utilized pattern. It even showed how to build a parser and didn't stop at the AST.

Apart from those classic patterns, the book introduces Internal Domain-Specific Languages, Meta-Programming, and Convention Over Configuration as newer patterns that are closely related to dynamic languages like Ruby. Those chapters cover nothing new to Rails programmers but are a nice addition to a general Ruby Patterns book. In my opinion the discussion of Meta-Programming could be a bit longer as it only covers class_eval and Ruby has more to offer.

In general, Design Patterns in Ruby is a very good overview of Design Patterns in the modern, dynamic world of Ruby. The book makes sure that the reader understands where a pattern arises and is very good in explaining its usage by example. Further, I cannot praise enough the fact that the author also tells you when NOT to use a pattern and warns about over-usage of inheritance or patterns.

With 338 pages of informal, easy to read examples and explanations, the book is easy to read in two or three afternoons. If you look for an introduction to Design Patterns or want to know how they apply in Ruby, I really recommend Design Patterns in Ruby.

REST is a nice theoretical concept and Rails is pushing hard in this direction. We all heard about RESTful Rails and map.resources but at least in my surrounding its adoption is still lacking.

At the last Berlin Ruby User Group meeting somebody asked if one of us had a running RESTful application and only one out of 40 responded. Then we got into the whole 'what is REST, when is your application RESTful, and is the edit-view part of REST...' discussion.

Yes, it would be wonderful if the world would be full of REST Web Services instead of SOAP Web Services but in the end we are talking about Web Services. If your application does not expose a service, there is no need for hardcore REST. That's why nobody raised their hand, nobody had a service to expose. Not yet, you could argue. And by doing it the RESTful way they would prepare for the day that their application is so successful that they start to think about exposing some resources. In in this case they would be prepared. Just add some responds_to magic and boom, you got your Web Service.

And I answer YAGNI. Do it when you need it, too much preparation is evil.

At the moment some people (especially in the Rails community) see REST as the new Golden Hammer. Just hammer with it once or twice on your application and boom out comes a Web Service for virtually no effort at all. And this is what scares me. Don't get me wrong. I like the idea of REST and definitely prefer it to the WS-* way of doing things. But making your application a Web Service that will serve anything, from JSON to XHTML and XML, just because you can will not solve your problems. It will create new ones because you couple different views and interaction schemes.

Seeing DHH at the RailsConf Europe keynote hacking custom MIME types in order to render a different view for the iPhone didn't make me jump up and down in a jolly manner. It made me thinking if the Rails community pushed too hard to the RESTful side. The iPhone required it's own view because although running a stock Browser it has a different interaction scheme. A different interaction scheme should force you to have a different logic in the controller. The responds_to blocks start to grow in complexity and size.

This is the point where you should apply the idea of de-coupling, the idea of services. Let the iPhone have it's own Web Application. Let than this new application access the common resources that the Desktop Browser application and the iPhone application share. This access can be done through just loading the same models through svn:externals or through a REST Web Service.

The REST concept should de-couple different clients (controller and views) from each other and from and back-end code (models).

Sometimes the responds_to stuff makes sense. If you mostly expose data, then offering JSON and XML besides XHTML makes sense if by doing this you save your clients the transformation or expose the data to more clients that you could not reach before. But it you have a Web Application with a lot of interaction then coupling the Desktop Browser version with the iPhone version, the mobile version, the JSON version, and the XML version does not make any sense to me.

For me REST is about exposing interconnected resources through a unified interface. And yes you should let the client choose the representation. But different clients will handle and interact differently with the resources. So please think before reaching for your Golden Hammer.

I use Mongrel and mongrel_cluster for all of my Rails apps so it was the obvious choice for our Rails based Knowledge Management system at Peritor.

Our app ran fine in WEBrick on the local development boxes but we had strange errors on the production Mongrels:

NoMethodError: undefined method `find’ for Cluster:Module

Locally it was also running fine with Mongrel. It took some time to find out that the mongrel_cluster gem_plugin that we use on the production boxes installs a global module named Cluster and that this module overshadowed our Rails model named Cluster. I already wrote the author about it and hopefully mongrel_cluster will use a separate namespace in the next release. Until then we manage the Mongrels ourselves. Maybe this will spare somebody a long debugging session…

With the newest Java update (Java 2 Standard Edition 5.0 Release 4) Java 5 is finally the default VM on the Mac:

After installing J2SE 5.0 Release 4, J2SE 5.0 becomes preferred over Java 1.4.2, which will still be installed on your Mac. Applications run with J2SE 5.0 unless they specifically request Java 1.4.2

No need for hacks like this anymore.

Even I do not really want to program in Java anymore after discovering the beauty of Ruby over a year ago, I am a regular listener of the Java Posse.

Python, Ruby and especially Ruby on Rails are common guests on this show as they unsettled the view of many Java Programmers. Every time the guys talk about dynamic scripting languages the story goes:

A: “I really like the features, the productivity gain and the dynamic capabilities. It is amazing what you can do with 10 lines of code.”
B: “Yeah, but Ruby still lacks good tool support and IDEs like Eclipse or IDEA.”
C: “Yes, that’s true.”
A: “Yeah with Java you do not have all these features but you have fabulous tools like Creator, they are just amazing. I can’t imaging working without such an IDE.”
B: “So let’s forget about these until they got an Eclipse.”

And every time I hear this I could just scream that the whole point of having such dynamic and expressive languages is that you can actually do some coding without having to rely on an IDE.

David Heinemeier Hansson gave a similar quote in an recent interiew at uk.builder.com:

Ruby attacks the productivity problem by making the core elements as expressive as possible, so the programmer doesn’t have to rely on IDEs and other tools on top of the language to get stuff done. Few programmers today would even consider doing Java without Eclipse/IDEA or C# without Visual Studio, but lots of programmers happily write Ruby code in good text editors that aren’t born with specialization for one particular language, like TextMate for OS X, which is the big hit in Ruby/Rails circles.

AMEN

Another common one goes like this:

A: “We often read about the so called complexity of Java programming, especially with J2EE.”
B: “I think this comes from people who are used to write their code with Vim.”
A: “Yes, you do not write deployment descriptors or Hibernate configuration files with Vim, you have to use a modern IDE like Eclipse or IDEA, they will generate the code and configuration for you.”
B: “We definitely recommend to use NetBeans, Eclipse, IDEA or Creator, this will ease a lot.”

Dynamic languages like Python and Ruby make programming much simpler as opposed to an IDE which only takes over control and hides the complexity.

Hiding is not the same as removing.

The IDEs bring their own complexity and learning curve. There are whole books published about how to use Eclipse!

How often do you see a novice despairing while just trying to edit a bunch of files in IDEs like Eclipse without creating a workspace or a project. Compare this with Textmate, just drop a folder or a file on its symbol in the Dock and boom, you get a tree view of all the files and you can just start the edit them and get productive.