WebCreator Pro + jQuery + Moneris

One of my customers created a simple form in WebCreator Pro, a donation form sending an email to their administrators.  Next, they wanted to tie the form with a payment gateway, Moneris in this case.  This is not a out-of-the-box feature of WebCreator Pro, of course.  Here I come to the rescue.

They built all their site with WebCreator Pro, and they didn’t want to switch to another CMS.  They also wanted to integrate the Moneris payment gateway at the lowest possible cost .  My previous experience was with fully embedded payment solutions, using Active Merchant and Ruby on Rails.  So this was a new challenge for me.

Solution

First, the cheapest way to integrate Moneris is to use their eSelect Plus Hosted Payment Page package.  The form submits the data to one of their web servers, and you then configure the page to your needs.  This solution also supports recurrent transactions, which was needed by the customer. Moneris has a great testing environment, so I was able to fully test my solution.

Next, when submitting data, you have to use a naming convention for the fields, as per Moneris integration guide.  And you also have to make sure the data is well formatted for the fields .

So here I am with an HTML page generated by WebCreator Pro.  This tool does not allow the designer to control how the HTML is generated.  You can include some javascript code, but tying it with elements in the DOM is not that smple.

I decided to go with jQuery (I love jQuery!).  It was just a matter of including a “javascript include” in the page and then all my logic would sit in a .js file that I have full control on it.  I was able to do that in WebCreator Pro by adding a Counter object at the top of the page.

In a nutshell, here’s what my script does after it loads:

1. When the DOM is ready, modify the form to change the encoding to ISO-8859-1 (required by Moneris to support french accents – by the way, this won’t work in IE).
2. Add hidden fields for the store ID and key.
3. Add additional hidden field to comply with the integration guide.  I also have some fields that are specific to the recurrent payment (if selected by the user).
4. Change the “onclick” event of the submit tag so that it calls one of my functions.
5. Fill-out some default values.

Then, the script will do some validations and some data manipulations when the user hits the Submit button.  If everything is find, the form will get posted to the Moneris server and the user will then enter her credit information.

This solution of course requires Javascript, but we assume that Javascript is enabled on the browser.  Anyway, most of the WebCreator Pro output requires javascript.

Then it was as simple as deploying my javascript files on the test and production servers.  Fortunately, WebCreator Pro does not wipe out the files when it publishes the site.

Have fun!

Mighty Mouse cleaning tips

Everybody knows that after a while, you loose the ability to scroll down with the mighty mouse.  This is the most used scrolling direction on the mouse.  So if you are in this situation, don’t throw the towel yet.  

Here are the solutions I tried:

• The one from Apple itself.
• Some of the ones in here.
• Cleaning the ball using a cooton swab and rubbing alcohol.  It did the work a few times!
• Blowing pressured air around the ball.

It worked a couple of times.  But lately, nothing was putting a smile on my face.  The mouse was really getting on my nervers.  I was about to “scrap” the mouse and buy a new one.  But hey, I might as well try to open it and clean it.  Worst case, I will break it and it will still go in the garbage.

So I watch this great video from Julian Schrader and decided on a sick day to just do it.  

First, let’s get the tools.  Second, wash the exterior of the mouse.  We don’t want to contaminate the patient during the operation.

Bad news: I cut my finger while trying to remove the ring with my knife.  So a few days after recovering from the cut, I decided to continue the delicate operation and use a small screw-driver to finish removing the ring.

I damaged the ring a little bit, but who cares! You never see it when the mouse is on a surface.  And once again, I was ready to throw this baby away.

I disconnected the cables, unscrewed the scrolling case, and voila! Look at the dirt inside:

 

Now I understand why the mouse was so picky.  Look at the amount of dirt on the right scroll wheel (the one used while scrolling down).  Mystery solved!

Now the easy stuff: cleaning the parts with a cotton swab and rubbing alcohol.  A clean wheel will look like this:

You can also take all four wheel and put them side by side.  They are magnetized, so they love each other and stay together.  Cleaning with the cotton is a lot easier.

Now before screwing the scrolling case back into the mouse, make sure the ball scrolls properly.  Gently put your finger on the ball, scroll and watch the wheels to see if they really scroll.  The scroll down function was still not going as smooth as I wanted.  So I opened the scrolling case again and found the problem:  the magnetic wheels do not have to touch the metal plates.  Otherwise, it puts a pressured on the wheel and you have to apply a lot of pressure on the ball to make it scroll.

Finally, I reassembled the mouse, anxiously wondering if the scrolling experience will have improved.

Once in a while, blow some pressured air all over the place to make sure there nothing left, especially before putting back all the pieces together.

Now the ring was originally glued to the mouse casing. Instead of using some crazy glue and because I guess I’ll hae to repeat the operation in the future, I decided to use rubber cement glue.  You put glue on both sides, way for it to dry and then put the ring back.  The remaining glue can be removed with the fingers.

 

Overall, this was a fairly simple operation.  The worst steps were:

• Removing the ring.  I broke it a bit while forcing with my screw-driver.  So do it very carefully.
• Putting back the cables in their sockets

Now my mouse is like new.  What a great feeling of scrolling down and see the page going up!

UPDATE: This video seems to be a bit better, especially for the tool he’s crafting for removing the ring.

simply_versioned + attachment_fu + aws/s3

After adding simply_versioned support in a model that is using attachment_fu with S3 storage, I was getting this strange error the second time my versions association was used in my application:

NoMethodError (undefined method `quoted_table_name' for "0.5.1":String)

After a few hours, I finally figured out that this was caused by the aws/s3 setting its Version to “0.5.1″ (in a module).  That module gets included by attachment_fu.  The simply_versioned plugin uses a Version model.  So I don’t know it this a bug with attachment_fu or Rails, but here’s what I did to fix the problem.

1. Rename the Version model to DataVersion
2. add “set_table_name ‘versions’” in the DataVersion model if you don’t want to rename the table in the database
3. In the simply_versioned.rb file, search for the “has_many” instruction and add :class_name => ‘DataVersion’

And voilà.  Everything should work fine now.

SOLR Presentation at Montreal On Rails

I’m back from a very relaxing week in Mexico.  I strongly recommend this resort: Valentin Imperial Maya in Riviera Maya.  Great place, great food!

Alright.  Here are the show notes of my presentation on SOLR at Montreal on Rails on August 19th.

First, the slides are on SlideShare.

SOLR is a Java-based plugin. It is based on the Lucene technology.  Other possible full-text search engine solutions are: Ferret, Ultra Sphinx, Xapian.

You basically install the acts_as_solr plugin, configure it and start the server using a rake task: rake solr:start. You also have to create temporary folders in the plugin folder.

script/plugin install git  /github.com/railsfreaks/acts_as_solr.git
mkdir vendor/plugins/acts_as_solr/solr/logs
mkdir vendor/plugins/acts_as_solr/solr/tmp

Now look at the file config/solr.yml that was created by the plugin.  You can customize it if you want.  Then, generate documentation (very handy) and start SOLR:

rake doc:plugins
rake solr:start

Then, you can test that SOLR is in fact running by going to: http://localhost:8982/solr/. This is a very handy tool to test the searches and verify that model instances have been properly indexed.

In your model, you simply have to add “acts_as_solr” and the model will be fulltext indexed. In my example, my model is named Tip. SOLR will index model instances when they are saved. To reindex existing instances, you can simply go through each of them and call save() or you can call rebuild_solr_index from the script/console:

script/console
> reload!
> Tip.rebuild_solr_index

To do a search, it’s very easy: Tip.search “something”.

Scores

Give the :scores option to the find method and results will have a solr_score attribute.

Tip.find_by_solr('foo', :scores => true)
number_to_percentage( tip.solr_score*100, :precision => 0 )

Additional fields

By default, SOLR indexes all model attributes.  If you want to index a virtual attribute, give the option :additional_fields to acts_as_solr:

acts_as_solr :additional_fields => [:searchable_tags]

Specific fields

If you don’t want all the attributes to be indexed, use the :fields option to specify the attributes you want to have indexed (you can include virtual attributes):

acts_as_solr :fields => [:title, :body, :searchable_tags]

Boost

By default, all attributes have the same weight in the search.  You can boost models/attributes by using the :boost option:

acts_as_solr :fields => [:body, {:title => {:boost => 100.0 }}, :featured, :searchable_tags], :boost => 10.0

Range

You can tell SOLR to treat an attribute as a integer or float range.  This will allow you to search for intervals:

acts_as_solr :additional_fields => [ {:seconds => :range_integer} ]

Then, you can search for an interval:

Tip.find_by_solr('seconds:[0 TO 30]')

Pagination

The find_by_solr accepts pagination and sorting options: :limit, ffset, rder.

Multi-model search

You can search in multiple models by giving :models to the find_by_solr method.  You have to invoke the method on a Model and include the other ones:

Tip.multi_solr_search( “pure”, :models => [Category,Comment] )

Return IDs only

Sometime, you only wanna have instances IDs instead of all their attributes.  You might want to do that in order to perform a SQL query after the full-text search and limit the search to the IDs SOLR returned.

Tip.find_id_by_solr(‘pure’).docs

Facets

Faceting allows you to have statistics on result groups.  For example, you could have the number of results per Tip category.  This is a “advanced” topic and I encourage you to read the faceting article that you will find in my resources list below.

French accents

Now, what about french accents in a field? Boom… out-of-the-box, this SOLR plugin will treat them as whitespaces. So if you have “crédit” in a model, you will not be able to find it with “credit”.  Look at the SOLR analyzer and you will see how it treats the indexing and search: http://localhost:8982/solr/admin/analysis.jsp?highlight=on

There is a way to fix this. You basically have to modify the filtering sequence in the SOLR schema (configuration). This is in the schema.xml file under vendor/plugins/acts_as_solr/solr/solr/conf. Modify the file with the following lines:

<fieldType name="text" class="solr.TextField" positionIncrementGap="100">
  <analyzer type="index">
    <tokenizer class="solr.StandardTokenizerFactory"/>
    <filter class="solr.ISOLatin1AccentFilterFactory"/>
    <filter class="solr.StandardFilterFactory"/>
    <filter class="solr.LowerCaseFilterFactory"/>
    <filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt"/>
    <filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="1" catenateNumbers="1" catenateAll="0"/>
    <filter class="solr.EnglishPorterFilterFactory" protected="protwords.txt"/>
    <filter class="solr.SnowballPorterFilterFactory" language="French"/>
    <filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
  </analyzer>

    <tokenizer class="solr.StandardTokenizerFactory"/>
    <filter class="solr.ISOLatin1AccentFilterFactory"/>
    <filter class="solr.StandardFilterFactory"/>
    <filter class="solr.LowerCaseFilterFactory"/>
    <filter class="solr.StopFilterFactory" ignoreCase="true" words="stopwords.txt"/>
    <filter class="solr.WordDelimiterFilterFactory" generateWordParts="1" generateNumberParts="1" catenateWords="1" catenateNumbers="1" catenateAll="0"/>
    <filter class="solr.EnglishPorterFilterFactory" protected="protwords.txt"/>
    <filter class="solr.SnowballPorterFilterFactory" language="French"/>
    <filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
  </analyzer>
</fieldType>

The ISOLatin1AccentFilterFactory filter will take into account the french accents and replace them with their equivalent english letter.  The SnowballPorterFilterFactory with french option will take into account the plural versions of some words.  You can add additional filters (such as a HTML stripper one to remove HTML codes).  Have a look at this page, it lists them all.

Caveat: those filters will apply to all the attributes.  Now this works well if you are integrating SOLR in a french-only site, but it will not work so well on a bilingual site.  This is where I want to eventually spend some time creating new field types based on language (i.e. text_fr, text_en).  This would allow having different sets of filters by field type.  I’ll write a blog entry when I get this done.

Resources

Look at the following links for additional information:

Acts as Solr Plugin
acts_as_solr : search and faceting
Advanced acts_as_solr
Solr: Indexing XML with Lucene and REST
acts_as_solr on GitHub

And read recipe 11 “Faceted Search with SOLR” in the Advanced Rails Recipe book.

GoogleMap geocoding challenges

Like most of the programmers that have to deal with languages other than english, I wish we were building english-only web applications, and support only english strings. I live in Quebec and almost all my projects have to deal with the french accents. It obviously brings a lot of challenges when dealing with external systems not using the UTF-8 encoding (like Hotmail for instance which is still on ISO-8859-1, or copy & paste from a French Word document). But I will keep this discussion for another post.

In one of my current projects, I need to integrate GoogleMap in order to get the geocode of a physical address. Piece of cake you would say: just use the google-geocode gem. Well, I tried. It worked really great until I tried searching for a Quebec address with french accents. It failed!

I then told myself that it wasn’t that complicated to implement it myself. I just query Google using a URL like http://maps.google.com/maps/geo?q=addressToLookFor&output=json&key=yourGoogleKey and voila, you get JSON results and simply have to parse it. Well, it is not that straight forward. Here’s what I found:

• By default, Google replies using the ISO-8859-1 encoding. The JSON parser doesn’t like that. To force UTF-8, you have to pass an extra parameter in the URL: oe=utf8. Where did I get this? From the bug list of the API.
• The results that Google sends back are not the same as the ones you get from a manual search on maps.google.ca. Why? I don’t know. I guess the web site does additional massaging to the results.
• Results may contain some strange things. For example, it you search for “455 st-pierre, montreal, qc, canada”, you get more than one address and one of them is in the Matawinie sub-administrative area, in the Montreal locality, which doesn’t make any sense. So which one is the right one? Well, the machine can’t figure it out, we have to ask the human in front of the computer to pick the right one.
• You sometimes get results that do not start with a numeric address (it looks like a generic geocode for the street only). I simply convert the address to an integer (bla.to_i), and if it returns something higher than 0, than I consider this address as a potential good one.

So I hope this saves you some time if you ever have to do some geocoding mapping in your application.

Thanks for listening!

P.S. I just found out that 3 days ago a new version of GeoX got out. I will give this plugin a try to see if it works as I would like it to work. I will post an update soon.

Textmate shortcuts

Oh my god. I finally found the shortcut I was missing for so long: Ctrl-Shift-T. It brings a list of all the methods in the class I’m currently editing. Man, I was dreaming about this feature almost every night. Everyday we learn something, so this is what I learned today (among other things of course!).

By the way, I found this while watching PeepCode’s Textmate Screencast. Lots of interesting screencasts on http://www.peepcode.com. And for free quick and dirty (and very effective) ones: http://www.railscasts.com. If you’ve never looked at those, go there right now! You WILL learn something, garanteed! You can even subscribe to those podcasts in iTunes and have them transferred to your iPod (or iPhone).

That’s it! Long live Textmate! Long live RailsCasts & PeepCode.

Concatenating strings

I was looking at some Railscasts.com podcasts and Bryan is using an interesting way to concatenate 2 strings, using an array and a join operation.

def full_name
  [first_name, last_name].join(' ')
end

Now, I was wondering if this was a bit overkill since it has to allocate an Array, then loop through it and build the string. So I ran Benchmark on three different ways to concatenate.

require 'benchmark'
include Benchmark

def using_array(a,b)
  [a,b].join(' ')
end

def using_format(a,b)
  "#{a} #{b}"
end

def using_concat(a,b)
  a+' '+b
end

Benchmark.benchmark " "*7 + CAPTION, 7, FMTSTR do |x|
  a = 'We all'
  b = 'love Ruby on Rails'

  n = 300000

  x.report("Array: ") { n.times do; using_array(a,b); end }
  x.report("Format:") { n.times do; using_format(a,b); end }
  x.report("Concat:") { n.times do; using_concat(a,b); end }
end

And the results speaks for themselves. As in other languages, the less memory allocations you do, the more performance you will get.

             user     system      total        real
Array:   1.810000   0.020000   1.830000 (  2.395285)
Format:  1.450000   0.010000   1.460000 (  1.634995)
Concat:  1.670000   0.020000   1.690000 (  1.974156)

As you can see, using the format string is really a better approach and I find it easier to read as well.

Railscasts rocks!

Another great discovery: railscasts.com. Over 73 small video podcats on different hints for Ruby on Rails. If you are a beginner or a senior RoR developper, you will for sure learn something from those podcasts. They are also accessible through iTunes, so you can subscribe to the podcast and have them sent to your iPod or iPhone. Thanks to Francois Beausoleil for the hint.

iPhone review

It’s been a week now that I have my iPhone. Like oncle Tom would say, it is simply “unnnnnbelievable”. Like mac OS X, everything is really well integrated. You see a phone number on a web page, in your adress book, in an SMS message and you simply click on it to make a call. Same thing with an e-mail address.

Talking about email: you can set up multiple accounts (POP, IMAP) and it just works! If you configure them to send your emails through AuthSmtp.com instead of your ISP relay, you can then send e-mails from anywhere in the world, as long as you are on a WiFi network.

The YouTube app is really addictive: easy to navigate and use, and the video quality matches the one on a regular PC.

The web browser (Safari) is a lot better than I thought. The screen being smaller than my Nokia n800, I had a fear that it would be really hard to read and interact with the web site. But the browser has an amazing intelligent zooming feature: when you double-click on a page area, it analyses the HTML code and will zoom just enough so that the element is properly shown within the screen. So if you double-click on a table cell, the browser will zoom that cell. You can then swipe your finger up and down to scroll the page, and this is also well done: the movement is processed so that the scrolling follows an axis. Good job Apple!

But the best is the sync feature. If you are on a Mac, it is transparent. You can sync ALL your contacts (not just a few of them), same thing with your iCal appointments. And if you have long notes in an appointment, it will transfer! And if you want, it can even synchronize your Safari bookmarks.

And….the screen does not scratch…. finally!

But, there are a few things that I miss from my previous Sony Ericson cell phone:

• Video recording (only pictures)
• French language (auto-completion is english only and it sometimes gets in the way)
• SMS templates (you always have to write a message from scratch
• No games! But I guess this will change soon.
• Bluetooth access (it only supports earphones, so you can’t exchange contacts or access its content from your notebook)
• Custom ringtones. You have to buy ringtones iTunes, you just can’t take an MP3 and use it as a ringtone.
• No WAP support. I know, WAP might be dead right now, but it was a great way to have a very light web site.

So at 399$, it is a must for anyone who wants to stay in touch with his or her digital life when on the road.

Refactor my code

Marc-André Cournoyer had a great idea: allow developpers to exchange on pieces of code, in a friendly matter so that everyone can be proud of their piece of code. The web site is called Refactor My Code. Go have a look. You can share your knowledge of Ruby, PHP, Javascript, Java, C, C#, Python, Perl, Lisp and Bash.

I briefly discussed with Marc-André at the last Montreal on Rails gathering and he was really happy that people at helping each others in a friendly manner.