New inflectors in Rails 3.0.5 may break your ActiveRecord models

I decided to write on this in the hope it will help someone else.  I lost too much time on this problem.

I have a Rails application that I migrated from 3.0.3 to 3.0.9.  Suddenly, all my tests were failing on my Media model with this error:

ActiveRecord::StatementInvalid:
Could not find table 'media'

I went in the Rails console, and just typing Media was giving me this error instead of listing the attributes:

=> Media(Table doesn't exist)

What??? I checked my database, everything was fine.  Looks like the pluralization does not work.  And then, I type:

'media'.pluralize
=> "media"

Ha haaaa….. that’s it.  Rails must have new inflectors, and it does!

To fix this, we have two options:

  1. Force the table name in the Model
  2. Add an inflector of our own

I chose the second option since it would apply to my entire app (views as well).  Just had to modify the inflections.rb initializer:

ActiveSupport::Inflector.inflections do |inflect|
  inflect.irregular 'media', 'medias'
end

There you go. Hope it helps.

Rails 1.2.x + Ruby 1.8.6 + OS X Lion: 10 easy steps

One of my most popular blog posts is this one, where I explain how you can have an old Rails application running on Snow Leopard.

Well now that Lion is out, it’s time to update and again make sure our old projects are still maintainable (yes, I have customers that don’t need to move to the latest and greatest – I wish).

Although this post targets old versions of Rails and Ruby, you can follow the same steps for the latest versions (1.9.2 and 3.0).

Background

Since 2010, things have changed in the Rails world.  Homebrew is now the preferred way to install packages on Mac OS X.  And RVM is now a must for anyone that wants to do serious Rails and/or Ruby development.

So here we go, follow me through the installation of the Rails stack on Lion.

Note 1: I did a clean install of Lion.  Not sure that my recipe will work on a “migrated” Lion.  A colleague of mine had problems setting up ImageMagick after migrating to Lion.  It might work, but why not take some time to clean everything up and all pieces installed properly.  And why not archive some files at the same time? 

Note 2: I’m using Textmate.  If you are using Aptana, you might want to check on the web, I think I’ve seen some blogs describing some issues with RVM and Aptana.  According to this post, starting Aptana from the command line makes it use the RVM interpreter.

Note 3: Phusion Passenger is awesome.  But in the past, I tried using it on my development machine and I didn’t like it so much.  I still prefer starting the server myself and see the output in my terminal window.  RMV supports Passenger, but I didn’t play with it.

Note 4: I’m on a Macbook Pro 17″ Mid-2010.  Your experience might be different if you are on a different architecture.

Prerequisites

  • If you are still on Snow Leopard, prior to migrate to Lion, find an empty drive and use SuperDuper to clone your hard drive.  This is an excellent backup solution (you can boot from the backup drive afterwards).
  • Format your drive and install Lion.  But hey, how do I install Lion on a clean drive?  If you have a newer Mac, when you boot, you will have a choice to install Lion from the internet.  Otherwise, you may download it and “burn” the image on a USB key or external drive (I tried on a DVD, may it is VERY slow when installing).  Follow these instructions.
  • Get the latest version of XCode on the Apple Developer Site and install it.

Step 1: Install Lion

Boot from the image, install and optionally import only the general settings when the migration assistant makes you the offer.  This will you will get some global preferences back, such as your network settings and printers.  You’re doing a clean install after all, and I prefer to move my files over myself so I can clean-up at the same time!

Step 2: Update the software

Once in Lion, click on the Apple and do a software update.  For sure, there are some new updates to apply.

Step 3: Homebrew

Then, let’s install Homebrew (just one line, isn’t that great?).  First, start the Terminal application (you will find it in Utilities in the Launchpad).  Then, copy and paste the following line.

/usr/bin/ruby -e "$(curl -fsSL https://raw.github.com/gist/323731)"

Homebrew uses “live” recipees stored on Github.  The behaviour described hereby might change in the future.  So, if you see a discrepancy, please let me know so I can update them.

Step 4: Xcode

Yes. Install Xcode after Homebrew, as stated on the web page. Download it from the App Store.

Note: At some point during the install, it might ask you to quit iTunes, although it does not seem to be running. This is due do the iTunes Helper that might be still running. Open “Activity Monitor”, search for the process iTunes Helper and kill it. The install will continue.

Step 5: Basic packages

Now let’s install some basic packages that we all love and cherish.

brew install readline git wget

You have to agree that this is cool.  No sources to download and untar, no configure command to run, … Homebrew takes care of doing all that.  Of course, we don’t have as much control, but this is fine with me so far.

Step 6: MySQL

Chances are that your old Rails project is using MySQL and not a popular NoSQL database such as MongoDB.  Let’s ask Homebrew to install it:

brew install mysql

Very important: read the text that Homebrew dumps at the end and follow the instructions.  Otherwise, you will have all kinds of problems.  If you want those messages back, use this command: brew info mysql.

unset TMPDIR
mysql_install_db --verbose --user=`whoami` --basedir="$(brew --prefix mysql)" --datadir=/usr/local/var/mysql --tmpdir=/tmpsudo chown `whoami` /var/mysql
mkdir -p ~/Library/LaunchAgents
cp /usr/local/Cellar/mysql/5.5.14/com.mysql.mysqld.plist ~/Library/LaunchAgents/
launchctl load -w ~/Library/LaunchAgents/com.mysql.mysqld.plist

I then get the following error:

launchctl: CFURLWriteDataAndPropertiesToResource(/var/db/launchd.db/com.apple.launchd.peruser.501/overrides.plist) failed: -10
But if I check the processes running on the Mac (ps -ef | grep mysql), MySQL server seems to be up and running.  Strange!

I also like to apply some changes to the default configuration.  When you import one of your SQL backups, the default max_allowed_packet might be too small.  So I like to boost it to 64M.

Here’s my /etc/my.cnf file:

[mysqld]
#Max packetlength to send/receive from to server.
max_allowed_packet=64M
character-set-server = utf8

#This option makes InnoDB to store each created table into its own .ibd file.
innodb_file_per_table

[mysql]
default-character-set = utf8

[client]
default-character-set = utf8

Then, let’s restart the MySQL server:

launchctl unload -w ~/Library/LaunchAgents/com.mysql.mysqld.plist
launchctl load -w ~/Library/LaunchAgents/com.mysql.mysqld.plist

Step 7: The lovely ImageMagick

Oh well.  This is always the worst thing to install on Mac OS X.  I had so many problems yesterday installing it that it’s worth explaining how I got it running.

First, later on, when you install RMagick, you will require Ghostscript fonts.  So it is a good thing to install it right now, before ImageMagick:

brew install ghostscript

Now you should have the following error (if not, you’re lucky and most likely the ghostcript brew recipe was fixed).

./base/gdevpng.c:283: error: dereferencing pointer to incomplete type
To fix this, follow the instructions here and install once again ghostscript.

And finally, ImageMagick:

brew install imagemagick

Step 8: Ruby Version Manager (RVM)

Now that we have the foundation installed, let’s have fun with the Ruby Version Manager.  RVM keeps multiple ruby environments isolated from each other.  Each one has its own ruby version (compiled from source) and its own gems (which is awesome).  You can also create “gemsets” to isolate even more projects that share the same ruby.

So instead of using the Lion out-of-the-box Ruby , let’s set-up an environment for 1.8.6.

But first, let’s install RVM:

bash < <(curl -s https://rvm.beginrescueend.com/install/rvm)

Important: do not forget to complete the install by performing 3 steps that are shown to you.

Then, let’s install our first ruby.

rvm install 1.8.6

This will install everything in the .rvm folder in your home folder. If you change your mind later, it’s easy to uninstall: rvm implode.

You can confirm the installation by running rvm list.

Step 9: Gems

RVM will install a version of Rubygems for each environment, with the proper gem sources. Let’s install some:

rvm 1.8.6
gem install ruby-debug
gem install rmagick -v 1.15.16
gem install mongrel
gem install mysql
gem install rails -v 1.2.1

If you get this error when you try to install the first gem:

/Users/hugo/.rvm/rubies/ruby-1.8.6-p420/lib/ruby/1.8/timeout.rb:59: [BUG] Segmentation fault

ruby 1.8.6 (2010-09-02) [i686-darwin11.0.0]

you need to recompile the version of ruby so that it uses the old compiler and not the new LLVM one:

rvm remove 1.8.6
CC=/usr/bin/gcc-4.2 rvm install 1.8.6
rvm 1.8.6

That’s it.  You should be good to go now.  I strongly suggest that you read on how to create and use gem sets.

Step 10: Sugar

There is a way with RVM to automatically switch interpreter when you cd to a project’s folder.  Simply provide a .rvmrc file in the project’s folder:

echo "rvm 1.8.6" > ~/projects/project-a/.rvmrc

Also handy is to have the current interpreter displayed in the command line prompt. Simply edit your ~/.bash_profile file and add the following line:

export PS1="\[\e[32;40m\]\$(~/.rvm/bin/rvm-prompt)\[\e[0m\] $PS1"

Have fun, and “long live to Ruby on Rails”!

Please comment if you see an error in here so I can update it.  Thanks!

Let’s have fun: Rails 2.3.5 + nginx + Passenger on a G4 Cube with Tiger

Last week-end, I purchased a used PowerMac G4 Cube, an iconic product in the Mac community.  I was wondering what I could do with it, given its performance limitation.

I thought of using it to rip some of my DVDs so I could watch the TV shows on my iPad.  But the cube does not have the power to encode in X264.  A 24 minutes episode was requiring 8 hours of encoding… That’s insane!

So then I came up with the idea of using it as a torrent downloader.  I would submit download request through a web interface or by placing a torrent file in a shared folder.  The web interface would also show the current and past downloads. I am planning to have Transmission running and using their API to interface with it, using one the following Github projects:

Since I spend almost all my time in Ruby on Rails, why not setting up Rails on the cube, with nginx, Passenger and SQLite.

Preparation

XCode installation.  XCode can be found on the Tiger install disc.  We do not need to have the latest and greates, we simply need the compile and some development libraries installed.  I did not install everything, just the following:

  • Developer Tools Software
  • gcc 4.0
  • gcc 3.3
  • Soware Development Kits

Ruby & Rubygems

I could install Ruby using the great RubyOSX one-click install, but I wanted to make sure I had the full control on what I install.  So I decided to install sources and compile everything.

Tiger comes with a very old version of Ruby: 1.8.2. Let’s upgrade it to a more recent version.  I followed Hivelogic instructions for doing so.

The difference is that I chose to install patch 399 of ruby 1.8.6: 

curl -O ftp://ftp.ruby-lang.org/pub/ruby/1.8/ruby-1.8.6-p399.tar.gz

And for Rubygems, I chose to install the latest version, 1.3.6:

curl -O http://production.cf.rubygems.org/rubygems/rubygems-1.3.6.tgz

Be patient when compiling Ruby… It’s only a 450Mhz PowerPC computer!

SQLite

Now that we have rubygems, let’s install the sqlite3 adapter.

sudo gem install sqlite3-ruby

Git

There is a good chance that we will need git for some plugins for our Rails application.  So let’s install this as well.

cd /usr/local/src
curl -O http://kernel.org/pub/software/scm/git/git-1.7.1.tar.gz
tar xzvf git-1.7.1.tar.gz 
cd git-1.7.1
make configure
./configure --prefix=/usr/local 
make
sudo make install</pre>

Rails

Now, the “piece de resistance”: Rails.  This is pretty straight forward.  Let’s install it, without the documentation.  Development will not be done on the G4 Cube.  We will use Capistrano to remotely deploy our apps on it.

sudo gem install -V rails -v 2.3.5 --no-rdoc --no-ri

Passenger

We need to install Passenger first, because we will need a passenger-nginx module later.

sudo gem install passenger

nginx

We could have installed nginx using Passenger, but let’s take the complex route (Passenger has not been tested on Tiger).

First, let’s install nginx.

cd /usr/local/src
curl -O http://nginx.org/download/nginx-0.8.36.tar.gz
tar xvfz nginx-0.8.36.tar.gz
cd nginx-0.8.36
sudo ./configure --pid-path=/usr/local/nginx/logs/nginx.pid --sbin-path=/usr/local/sbin/nginx --with-md5=/usr/lib --with-sha1=/usr/lib --with-http_ssl_module  --with-http_dav_module --without-http_rewrite_module --add-module=`passenger-config --root`/ext/nginx
sudo make
sudo make install

When running “configure”, at some point, you should get *** Phusion Passenger support files have been successfully compiled ***. That’s a good sign!

Now let’s create a script to automatically start nginx (thanks to Trevor for some hints).

Using a file editor, create file /Library/LaunchDaemons/nginx.plist and put the following content in it:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
        <key>KeepAlive</key>
        <true/>
        <key>Label</key>
        <string>nginx</string>
        <key>RunAtLoad</key>
        <true/>
        <key>LaunchOnlyOnce</key>
        <true/>
        <key>Program</key>
        <string>/usr/local/sbin/nginx</string>
        <key>StandardErrorPath</key>
        <string>/usr/local/nginx/logs/error.log</string>
</dict>
</plist>

Load the daemon description:

sudo launchctl load /Library/LaunchDaemons/nginx.plist

Nginx should start automatically. You can check by looking the processes and see if the nginx server is there:

ps -aux | grep nginx

If nginx did not start, look at the possible errors in /usr/local/nginx/logs/error.log.

If you need to start or stop nginx manually, here’s how to do it:

sudo nginx -s stop
sudo nginx

Test application

Let’s create a dummy Rails app to test our set-up.

cd ~
rails foo
cd foo
script/generate scaffold bar title:string description:text
rake db:create
rake db:migrate
sed 's/# map.root/map.root/g' config/routes.rb | sed 's/welcome/bars/g' > config/routes.rb
rm public/index.html

Now, let’s configure nginx. Instead of creating virtual servers for each Rails app, let’s put them all under the same virtual server. This way, we won’t have to create special hostnames and different server sections in the nginx configuration. And we will be able to access the applications by pointing to the G4 Cube ip address. We just need to make sure Passenger is aware of that by using the passenger_base_uri directive.

So let’s create a symbolic link in the root html directory:

cd /usr/local/nginx/html
sudo ln -s ~/foo/public/ foo

Make sure the nginx configuration file (/usr/local/nginx/conf/nginx.conf) looks like something like this:

worker_processes  1;

events {
    worker_connections  1024;
}

http {
    include       mime.types;
    default_type  application/octet-stream;

    sendfile        on;
    keepalive_timeout  65;

    gzip  on;

    passenger_root /usr/local/lib/ruby/gems/1.8/gems/passenger-2.2.11;
    passenger_ruby /usr/local/bin/ruby;

    server {
        listen       80;
        server_name  localhost;

        passenger_enabled on;
        passenger_base_uri /foo;
        rails_env development;
        root /usr/local/nginx/html;
    }
}

Check the location of your passenger root by running this command:

passenger-config --root

If you loaded the launch daemon script, nginx is most likely running at the moment. Let’s reload the configuration file:

sudo nginx -s reload

Now, the moment of truth. Rush to Safari and go to http://localhost/foo. After a few seconds (Passenger must start the app), you should get the bars scaffold.

Excellent! We now have a fully functional system for hosting utility apps on my home network. The next step is to write the apps!

See ya next time!

Efficiency with Rails

Sometimes, I can’t believe the amount of work I can accomplish with Ruby on Rails. Today, I finished coding a RSS feed aggregator that generates aggregated RSS feeds for a Joomla web site (it merges posts from multiple feeds into one single daily feed for Joomla).

  • Import/export tab-delimited list of channels
  • Fetch RSS posts and converting the content to UTF-8 encoding (if necessary) (all major RSS formats) in background at regular intervals (daemon)
  • Show a list of all posts published today (taking into account the timezone)
  • Present that list in Atom format
  • Manually add/edit/destroy a channel
  • Show the raw channel content in XML format (to spot conversion errors if any)
  • Store the channels and posts in a database
  • Add a “source” element for each post in the aggregated feed that I generate
  • Modify the Joomla renderer to display the source of the post
  • Provide a way to stop/start the posts importer running in background
  • Set-up CentOS server for hosting the Rails application
  • Deploy to production server using Capistrano
  • Integrate a nice CSS template
  • Include basic http authentication for a single administrator
  • List all the posts in the database, with pagination
  • Provide a nice 2-level navigation menu
  • Store source code into Subversion

It took only 19 hours to do all that.  And it could have been faster if rendering the aggregated feed in Joomla would have been a lot easier (SimplePie does not support the “source” tag in RSS 2, but it does in Atom).

Thanks to the following open source contributions which made me save lots of time:

I’m very happy with the result.  I wonder if the same results could be achieved using another framework, such as .Net or PHP, in the same amount of time.  I think it would take more time, but hey, my opinion is biased.

Long live to Ruby on Rails!

Any comment?

Rails 1.2.x + Ruby 1.8.6 + Snow Leopard: the missing link

Yesterday, I migrated to Snow Leopard.  After a day of file transfers, disk formatting, apps installation, password and license keys recovery, I finally was ready to install the Rails stack.  It’s great to do a clean install, we can get rid of the crap that was installed in the last 2 years.

Today, I spent a few hours on the Rails stack.  And I ran into some problems that I need to share with you because I didn’t find an answer on the net.

Background

I had to install the Rails stack, the framework I use in most (if not all) my projects.  The challenge was that I still have a few Rails projects that run under 1.2.x.  And Rails 1.2.x is not a good friend of Ruby 1.8.7, the version of ruby that ships with Snow Leopard.

Back on Tiger, I was using the amazing Locomotive application, which was a way to have dedicated environments for rails applications.  Then, when I moved to Leopard, Locomotive was not supported and I had to deal with all the apps sharing the same gems, same ruby and same everything.  Fortunately, ruby 1.8.6 was handling my Rails 1.2.x projects properly. I was also unsing Capistrano 1.4.2 along side 2.5.2 to deploy those old projects (i.e. cap _1.4.2_ deploy_with_migrations).

Now on Snow Leopard, I wanted to keep this flexibility.  I looked at different ways to set-up Rails on my Macbook Pro (Core 2 Duo).  The one on the Rails wiki was interesting, and I was used to this set-up, but I wanted to do a bit more.  I remembered that a new package manager was borned this year: homebrew.  I heard some good things about it.  And my last wish was to have multiple versions of Ruby on my system. This guide was a really good “instructor” on using RVM, the Ruby Version Manager.  Using RVM, you can have ruby 1.8.6, 1.8.7, 1.9.1 and even jruby and ree (Ruby Enterprise Edition).  This is cool, I will finally have a chance to start testing with 1.9.1 and eventually start new projects with it.

So I installed all that using my own recipee (which follows next), and everything was working fine with Ruby 1.8.7 and Rails 2.3.5.  When I started to use Ruby 1.8.6, my rails apps were hanging so bad, I was forced to “kill -9” the processes (CTRL-C was not working).  Anytime the Rails environment was loading, the process would hang.

After a few hours of investigation, I found out that the problem was with RMagick.  And the problem was due to the use of NSLinkModule functions in dln.c (ruby 1.8.6 source code). Thanks to RVM, we have access to the source code of ruby. After applying a patch to ruby 1.8.6, as described in the bug report, everything started to work.

So here we go, follow me through the installation of the Rails stack on Snow Leopard.

Note 1: I did a clean install of Snow Leopard.  Not sure that my recipe will work on a Snow Leopard migration.  It might, but you will have to reinstall some 32-bit gems that compile some binary stuff.

Note 2: I’m using Textmate.  If you are using Aptana, you might want to check on the web, I think I’ve seen some blogs describing some issues with RVM and Aptana.  According to this post, starting Aptana from the command line makes it use the RVM interpreter.

Note 3: Phusion Passenger is awesome.  But in the past, I tried using it on my development machine and I didn’t like it so much.  I still prefer starting the server myself and see the output in my terminal window.  RMV supports Passenger, but I didn’t play with it.

Prerequisites

  • If you are still on Leopard, prior to migrate to Snow Leopard, find an empty drive and use SuperDuper to clone your hard drive.  This is an excellent backup solution (you can boot from the backup drive).
  • Format your drive and installer Snow Leopard.
  • Get the latest version of XCode on the Apple Developer Site and install it.

Homebrew

We could use MacPorts, but Homebrew takes less space (it uses what we already have on Mac OS X) and it does not require being root.  And Ryan Bates recommends it.

As mentioned on Homebrew’s wiki, you can install it anywhere, but lots of 3rd party solutions expect to have things in /usr/local. And Homebrew’s files are well isolated and the uninstall is well documented.  So there is no danger installing it under /usr/local.

So let’s change the owner of the /usr/local folder (otherwise, we always have to switch to super user).  I know, this is not standard practice, but hey, we’re on a mac and most likely you will be the only one using stuff in /usr/local.

sudo chown -R `whoami` /usr/local

Then, let’s install Homebrew (just one line, isn’t that great?).

curl -L http://github.com/mxcl/homebrew/tarball/master | tar xz --strip 1 -C /usr/local

Basic packages

Now let’s install some basic packages that we all love and cherish.

brew install readline
brew install git
brew install wget
brew install imagemagick

You have to agree that this is cool.  No sources to download and untar, no configure to run, … Homebrew takes care of doing all that.  Of course, we don’t have as much control, but this is fine with me so far.

MySQL

I’m still using MySQL (although Postgresql is on my R&D to-do list for 2010).  It’s installation is well documented.  You can download a disk image from the MySQL web site or compile the source code, or simply use Homebrew:

brew install mysql
mysql_install_db
sudo chown `whoami` /var/mysql

I also like to apply some changes to the default configuration. Here’s my /etc/my.cnf file:

[mysqld]
#Max packetlength to send/receive from to server.
max_allowed_packet=64M
socket = /var/mysql/mysql.sock
character-set-server = utf8
default-character-set = utf8

#This option makes InnoDB to store each created table into its own .ibd file.
innodb_file_per_table

[mysql]
default-character-set = utf8

[client]
socket = /var/mysql/mysql.sock
default-character-set = utf8

Then, let’s start the server and also install the launcher.

cd /usr/local/Cellar/mysql/5.1.41; /usr/local/Cellar/mysql/5.1.41/bin/mysqld_safe &
launchctl load -w /usr/local/Cellar/mysql/5.1.41/com.mysql.mysqld.plist

Ruby Version Manager (RVM)

Now that we have the foundation installed, let’s implement the Ruby Version Manager.  RVM keeps multiple ruby environments isolated from each other.  Each one has its own ruby version (compiled from source) and its own gems (which is awesome).

So instead of using the Snow Leopard out-of-the-box Ruby , let’s set-up an environment for 1.8.7.

But first, let’s install RVM (replace USER with your username):

gem install rvm
echo "if [[ -s /Users/USER/.rvm/scripts/rvm ]] ; then source /Users/USER/.rvm/scripts/rvm ; fi" >> ~/.bash_login
rvm install 1.8.7
rvm use 1.8.7

This will install everything in the .rvm folder in your home. If you change your mind later, it’s easy to uninstall: rvm implode.

You can confirm the installation by running rvm list.

Gems

RVM will install a version of Rubygems for each environment, with the proper gem sources (yes, gemcutter is already in the list). We have to install all the gems we need for each environment. Let’s install the common ones:

gem install ruby-debug
gem install capistrano
gem install rmagick
gem install sqlite3-ruby
gem install mongrel
export ARCHFLAGS="-arch i386 -arch x86_64"
gem install mysql -- --with-mysql-dir=/usr/local --with-mysql-config=/usr/local/bin/mysql_config
gem install rails
gem install manalang-bdoc

Now, to really see the power of isolated environments, launch bdoc from the command line.  See? Only the gems that we just installed.

Then, you can install additional gems, or go into one of your Rails 2.x project and issue rake gems:install.

Ruby 1.8.6

Now that we have ruby 1.8.7 working, let’s install version 1.8.6 so that we can do maintenance in our old Rails projects running under 1.2.1. And let’s make 1.8.7 the default interpreter when we open a new bash session.

rvm install 1.8.6
rvm 1.8.7 --default
rvm use 1.8.6

Now, we have to install a backport from ruby 1.8.7 (patch) so that we can use RMagick under ruby 1.8.6 on Snow Leopard. I suspect that it also fixes some other weird bugs.

rvm use 1.8.6
cd ~/.rvm/src/ruby-1.8.6-p383
wget http://redmine.ruby-lang.org/attachments/download/580 -O dln.patch
patch dln.c dln.patch
make
make install

Finally, let’s install the gems that we need.

gem install ruby-debug
gem install capistrano -v 1.4.2
gem install rmagick
gem install mongrel
export ARCHFLAGS="-arch i386 -arch x86_64"
gem install mysql -- --with-mysql-dir=/usr/local --with-mysql-config=/usr/local/bin/mysql_config
gem install rails -v 1.2.1
gem install manalang-bdoc

And that’s it. We now have 2 isolated interpreters and their gems. We are now in business for eventually migrating some projects to ruby 1.9.

Candies

There is a way with RVM to automatically switch interpreter when you cd to a project’s folder.  Simply provide a .rvmrc file in the project’s folder:

echo "rvm 1.8.6%projecta" > ~/projects/projecta/.rvmrc

Also handy is to have the current interpreter displayed in the command line prompt. Simply edit your ~/.bash_login file and add the following line:

export PS1="\$(~/.rvm/bin/rvm-prompt) $PS1"

Have fun, and continue to enjoy Ruby on Rails.

Permanent 301 redirections using a Rack middleware in Rails

It’s a quarter to midnight and you are just about to deploy a new version of a web site that has a totally new hierarchy.  This happens a lot when migrating a PHP site to Rails.  And then you realize that the structure or the URLs is totally different than the one on the old site.

Maybe the old site has a nice ranking in Google and a good reputation or lots of people have bookmarked pages of the site.  You don’t want to make Google and those users unhappy.  So you want to have the new web site send a 301 reponse code (permanent redirect) instead of the famous 404 (page not found).

One way to do this in Apache is to enable the rewrite engine and add some rewrite rules in the web site configuration.  But these rules are not easy to build and it involves changing the Apache config, which is not trivial for some people.

I found a Rack middleware called Rack::Rewrite that can take care of the redirections.  I find this solution interesting because it is web server agnostic (will work with Apache, Nginx, …).  Although the performance is not as good as having rewrite rules at the web server level.

rails 2.x

First, you need install the gem:

gem install rack-rewrite

Then, you want to include a reference to the gem in config/environment.rb:

config.gem 'rack-rewrite', '~> 0.2.0'

Rails 3.x

First, you need to include the gem in your Gemfile:

gem 'rack-rewrite'

Then, we need to load the middleware in the rails stack of our application.  Instead of putting this in the environment.rb file, I prefer to put it in an initializer (config/initializers/rack_rewrite.rb) since we will also include the redirection rules in this file:

Rails 2.x

ActionController::Dispatcher.middleware.insert_before(Rack::Lock, Rack::Rewrite) do
  r301 '/fr/rates_and_packages', '/fr/services-et-tarifs'
  r301 '/en/rates_and_packages', '/en/services-and-rates'
  r301 '/fr/information', '/fr/renseignements/horaire-et-coordonnees'
  r301 '/en/information', '/en/informations/business-hours-and-location'
  r301 %r{\A/index.php}, '/fr'
  r301 %r{/en/index.php}, '/en'
end

Rails 3.x

YourApplicationName::Application.config.middleware.insert_before(Rack::Lock, Rack::Rewrite) do
  r301 '/fr/rates_and_packages', '/fr/services-et-tarifs'
  r301 '/en/rates_and_packages', '/en/services-and-rates'
  r301 '/fr/information', '/fr/renseignements/horaire-et-coordonnees'
  r301 '/en/information', '/en/informations/business-hours-and-location'
  r301 %r{\A/index.php}, '/fr'
  r301 %r{/en/index.php}, '/en'
end

Remember to include your application namespace in the previous code snippet.

As you can see, the first rules look for a perfect match. The last two rules apply a regular expression to the request URI. This is really cool when a bunch of old URLs have a similar structure and share the same redirection URL.  There is a rewrite command that allows to transform the URL and pass it down the Rails stack.  See the documentation for more options.

Now, how do you find out what are the old URLs you need to redirect?  Well, you can go on the old site and manually crawl the site.  But I prefer to use a simple ruby crawler called Anemone.

gem install anemone

Then write some ruby code and run it.

require 'rubygems'
require 'anemone'

Anemone.crawl('http://www.awebsitetocrawl.ca') do |anemone|
  anemone.on_every_page do |page|
    puts page.url
  end
end

It will give you a list of all the URLs found on the given site. Though it does not work if the site is using Flash menus (normally, such sites should have regular navigation menus somewhere on the page for SEO purposes, but I’ve seen some sites that don’t).

That’s it. A simple way to generate redirection codes. There is a more complete solution named Refraction that you can use as well, but I am satisfied with the straight forward Rack::Rewrite solution.

Here you go, hoping it helps you in your Rails project.

simply_versioned + attachment_fu + aws/s3

After adding simply_versioned support in a model that is using attachment_fu with S3 storage, I was getting this strange error the second time my versions association was used in my application:

NoMethodError (undefined method `quoted_table_name' for "0.5.1":String)

After a few hours, I finally figured out that this was caused by the aws/s3 setting its Version to “0.5.1” (in a module).  That module gets included by attachment_fu.  The simply_versioned plugin uses a Version model.  So I don’t know it this a bug with attachment_fu or Rails, but here’s what I did to fix the problem.

  1. Rename the Version model to DataVersion
  2. add “set_table_name ‘versions'” in the DataVersion model if you don’t want to rename the table in the database
  3. In the simply_versioned.rb file, search for the “has_many” instruction and add :class_name => ‘DataVersion’

And voilà.  Everything should work fine now.

SOLR Presentation at Montreal On Rails

I’m back from a very relaxing week in Mexico.  I strongly recommend this resort: Valentin Imperial Maya in Riviera Maya.  Great place, great food!

Alright.  Here are the show notes of my presentation on SOLR at Montreal on Rails on August 19th.

First, the slides are on SlideShare.

SOLR is a Java-based plugin. It is based on the Lucene technology.  Other possible full-text search engine solutions are: Ferret, Ultra Sphinx, Xapian.

You basically install the acts_as_solr plugin, configure it and start the server using a rake task: rake solr:start. You also have to create temporary folders in the plugin folder.



script/plugin install git://github.com/railsfreaks/acts_as_solr.git
mkdir vendor/plugins/acts_as_solr/solr/logs
mkdir vendor/plugins/acts_as_solr/solr/tmp

Now look at the file config/solr.yml that was created by the plugin.  You can customize it if you want.  Then, generate documentation (very handy) and start SOLR:



rake doc:plugins
rake solr:start

Then, you can test that SOLR is in fact running by going to: http://localhost:8982/solr/. This is a very handy tool to test the searches and verify that model instances have been properly indexed.

In your model, you simply have to add “acts_as_solr” and the model will be fulltext indexed. In my example, my model is named Tip. SOLR will index model instances when they are saved. To reindex existing instances, you can simply go through each of them and call save() or you can call rebuild_solr_index from the script/console:



script/console
> reload!
> Tip.rebuild_solr_index

To do a search, it’s very easy: Tip.search “something”.

Scores

Give the :scores option to the find method and results will have a solr_score attribute.

Tip.find_by_solr('foo', :scores => true)
number_to_percentage( tip.solr_score*100, :precision => 0 )

Additional fields

By default, SOLR indexes all model attributes.  If you want to index a virtual attribute, give the option :additional_fields to acts_as_solr:

acts_as_solr :additional_fields => [:searchable_tags]

Specific fields

If you don’t want all the attributes to be indexed, use the :fields option to specify the attributes you want to have indexed (you can include virtual attributes):

acts_as_solr :fields => [:title, :body, :searchable_tags]

Boost

By default, all attributes have the same weight in the search.  You can boost models/attributes by using the :boost option:

acts_as_solr :fields => [:body, {:title => {:boost => 100.0 }}, :featured, :searchable_tags], :boost => 10.0

Range

You can tell SOLR to treat an attribute as a integer or float range.  This will allow you to search for intervals:

acts_as_solr :additional_fields => [ {:seconds => :range_integer} ]

Then, you can search for an interval:

Tip.find_by_solr('seconds:[0 TO 30]')

Pagination

The find_by_solr accepts pagination and sorting options: :limit, :offset, :order.

Multi-model search

You can search in multiple models by giving :models to the find_by_solr method.  You have to invoke the method on a Model and include the other ones:

Tip.multi_solr_search( “pure”, :models => [Category,Comment] )

Return IDs only

Sometime, you only wanna have instances IDs instead of all their attributes.  You might want to do that in order to perform a SQL query after the full-text search and limit the search to the IDs SOLR returned.

Tip.find_id_by_solr(‘pure’).docs

Facets

Faceting allows you to have statistics on result groups.  For example, you could have the number of results per Tip category.  This is a “advanced” topic and I encourage you to read the faceting article that you will find in my resources list below.

French accents

Now, what about french accents in a field? Boom… out-of-the-box, this SOLR plugin will treat them as whitespaces. So if you have “crédit” in a model, you will not be able to find it with “credit”.  Look at the SOLR analyzer and you will see how it treats the indexing and search: http://localhost:8982/solr/admin/analysis.jsp?highlight=on

There is a way to fix this. You basically have to modify the filtering sequence in the SOLR schema (configuration). This is in the schema.xml file under vendor/plugins/acts_as_solr/solr/solr/conf. Modify the file with the following lines:



<fieldType name="text" class="solr.TextField" positionIncrementGap="100">
  <analyzer type="index">
    <tokenizer class="solr.StandardTokenizerFactory"/>
    <filter class="solr.ISOLatin1AccentFilterFactory"/>
    <filter class="solr.StandardFilterFactory"/>
    <filter class="solr.LowerCaseFilterFactory"/>
    <filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt"/>
    <filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="1" catenateNumbers="1" catenateAll="0"/>
    <filter class="solr.EnglishPorterFilterFactory" protected="protwords.txt"/>
    <filter class="solr.SnowballPorterFilterFactory" language="French"/>
    <filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
  </analyzer>

    <tokenizer class="solr.StandardTokenizerFactory"/>
    <filter class="solr.ISOLatin1AccentFilterFactory"/>
    <filter class="solr.StandardFilterFactory"/>
    <filter class="solr.LowerCaseFilterFactory"/>
    <filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt"/>
    <filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="1" catenateNumbers="1" catenateAll="0"/>
    <filter class="solr.EnglishPorterFilterFactory" protected="protwords.txt"/>
    <filter class="solr.SnowballPorterFilterFactory" language="French"/>
    <filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
  </analyzer>
</fieldType>

The ISOLatin1AccentFilterFactory filter will take into account the french accents and replace them with their equivalent english letter.  The SnowballPorterFilterFactory with french option will take into account the plural versions of some words.  You can add additional filters (such as a HTML stripper one to remove HTML codes).  Have a look at this page, it lists them all.

Caveat: those filters will apply to all the attributes.  Now this works well if you are integrating SOLR in a french-only site, but it will not work so well on a bilingual site.  This is where I want to eventually spend some time creating new field types based on language (i.e. text_fr, text_en).  This would allow having different sets of filters by field type.  I’ll write a blog entry when I get this done.

Resources

Look at the following links for additional information:

Acts as Solr Plugin
acts_as_solr : search and faceting

Advanced acts_as_solr

Solr: Indexing XML with Lucene and REST

acts_as_solr on GitHub

And read recipe 11 “Faceted Search with SOLR” in the Advanced Rails Recipe book.

GoogleMap geocoding challenges

Matawini

Like most of the programmers that have to deal with languages other than english, I wish we were building english-only web applications, and support only english strings. I live in Quebec and almost all my projects have to deal with the french accents. It obviously brings a lot of challenges when dealing with external systems not using the UTF-8 encoding (like Hotmail for instance which is still on ISO-8859-1, or copy & paste from a French Word document). But I will keep this discussion for another post.

In one of my current projects, I need to integrate GoogleMap in order to get the geocode of a physical address. Piece of cake you would say: just use the google-geocode gem. Well, I tried. It worked really great until I tried searching for a Quebec address with french accents. It failed!

I then told myself that it wasn’t that complicated to implement it myself. I just query Google using a URL like http://maps.google.com/maps/geo?q=addressToLookFor&output=json&key=yourGoogleKey and voila, you get JSON results and simply have to parse it. Well, it is not that straight forward. Here’s what I found:

  • By default, Google replies using the ISO-8859-1 encoding. The JSON parser doesn’t like that. To force UTF-8, you have to pass an extra parameter in the URL: oe=utf8. Where did I get this? From the bug list of the API.
  • The results that Google sends back are not the same as the ones you get from a manual search on maps.google.ca. Why? I don’t know. I guess the web site does additional massaging to the results.
  • Results may contain some strange things. For example, it you search for “455 st-pierre, montreal, qc, canada”, you get more than one address and one of them is in the Matawinie sub-administrative area, in the Montreal locality, which doesn’t make any sense. So which one is the right one? Well, the machine can’t figure it out, we have to ask the human in front of the computer to pick the right one.
  • You sometimes get results that do not start with a numeric address (it looks like a generic geocode for the street only). I simply convert the address to an integer (bla.to_i), and if it returns something higher than 0, than I consider this address as a potential good one.

So I hope this saves you some time if you ever have to do some geocoding mapping in your application.

Thanks for listening! 😉

P.S. I just found out that 3 days ago a new version of GeoX got out. I will give this plugin a try to see if it works as I would like it to work. I will post an update soon.

Textmate shortcuts

Oh my god. I finally found the shortcut I was missing for so long: Ctrl-Shift-T. It brings a list of all the methods in the class I’m currently editing. Man, I was dreaming about this feature almost every night. Everyday we learn something, so this is what I learned today (among other things of course!).

By the way, I found this while watching PeepCode’s Textmate Screencast. Lots of interesting screencasts on http://www.peepcode.com. And for free quick and dirty (and very effective) ones: http://www.railscasts.com. If you’ve never looked at those, go there right now! You WILL learn something, garanteed! You can even subscribe to those podcasts in iTunes and have them transferred to your iPod (or iPhone).

That’s it! Long live Textmate! Long live RailsCasts & PeepCode.