Tuesday, 22 December 2009

The named_scope in Rails

The named_scope method allows you to set up shortcuts to narrow searches within set parameters. It is invoked in the model, something like this:
class Computer < ActiveRecord::Base
named_scope :active, :conditions => {:in_use => true}
named_scope :networked, :conditions => "(network_id <> \"\")"
named_scope :recent, lambda { {:conditions => ["created_at > ?", 1.months.ago ] } }

Note the use of the lambda. If you simply compared the created_at to the date, you would be comparing it to the date the named_scope was invoked, rather than the current date (you might object that Rails reloads the model each time, so the difference might not be so much, but that does not seem to affect your named_scopes). You might also want to do this if you want to access another model. Say your database has a list of types, and this model has a field for the ID of one of those types. You want to collect all the records by the name of the type, so you need to access the table for the other model to get the ID. But that model might not be loaded when this one is, so you have to delay getting the ID. Or perhaps you want to send parameters to a named_scope through the lambda.
# Use a lambda to access a later loading model
named_scope :blue, lambda {
{ :conditions => ["(colour_id = #{Colour.find_by_name('blue').id})"] }
# Use a lambda to allow parameters
named_scope :located, lambda { |loc|
{ :conditions => {:location_id => loc } }

You can also use named_scopes for other things, such as ordering.
named_scope :ordered, :order => 'created_at ASC'
Now to get an array of computers in use, just do this:

You can chain named_scopes together, and also with find. Now I can list all the blue computers at location 5, in ascending order of record creation, or all the networked computers running WinXP just like this:


A named_space can be used with will_paginate without a problem:
@samples = Sample.outstanding.ordered.paginate :page => params[:page], :per_page => 16

I found that I had to restart my web server when trying these out to get the named_scopes to reload.

Also, if named_scope does not like your condition, it just fails to define a new method. There is no warning or hint about what could be wrong. All you get is a method_missing complaint when you try to invoke it.

Struggling with Ruby: Contents Page

Wednesday, 16 December 2009

Using Sub-directories in Rails Projects

If you have a big project, you are going to want to break it up into parts, grouping, say, controllers for a certain part in one sub-directory. I found a couple of blog pages saying how to do this (basically you set up a name space in routes.rb, and prefix the controller class names with that name space, with the views in a similarly-named subdirectory):


However, they paint it rather simpler than it really is.

The Namespace for Controllers
Okay, so I have a number of controllers relating to a sample logging system, and I want to put them all inside a directory called sample_log. This corresponds to a Ruby namespace (because I might have a controller called TopController in each part of the system, so Rails needs a way to guarantee they are distinct). All the views need to be in their own subdirectory too, with the same name. Each of my controllers' class name needs to be prefixed with the name of the namespace:
class SampleLog::SamplesController < ApplicationController

Then you need to set up your routes, so that Rails knows you are using a namespace:
map.namespace :sample_log do |submap|
submap.resources :samples
# other controllers

At this point, you should be able to get pages, with a URL something like this:

So far so good. The tricky part (especially if you already have a project that you want to do this to) is handling links in and out of the subdirectory. The standard link_to method invocation looks like this:
link_to 'Cylinders', :controller => 'cylinders'

This will generate a link within the sub-directory. How do you link to other subdirectories, or to the top level? Append a slash to your controller name, like this:
link_to 'Cylinders', :controller => '/cylinders'
link_to 'Samples', :controller => '/sample_log/samples'

The various helper methods like new_samples_path and edit_samples_path seem to work fine, but require the directory name to be appended to the method name (run rake routes to see the helper methods listed):
link_to 'List', sample_log_generic_samples_path
link_to 'Show', sample_log_generic_sample_path(@sample)
link_to 'Edit', edit_sample_log_generic_sample_path(@sample)
redirect_to sample_log_samples_path

However, Rails does not seem to be able to cope with links like this (use the helper methods just mentioned instead):
link_to 'Show', @sample
redirect_to @sample

For some reason, Rails does not provide a helper method for destroy, so you will need to given that link through the action:
link_to 'Destroy', { :action => :destroy, :id => sample.id },
:confirm => 'Are you sure?',
:method => :delete

If you use the form_for functionality, or polymophic methods (presumably with STI), you need to send the directory as a parameter (as a symbol or string), wrapped up in an array:
form_for([:sample_log, @sample]) do |f|
link_to 'Edit', edit_polymorphic_path([:sample_log, @samples[1]])

Pointing to templates and partials
Any time you explicitly invoke a template in a controller, you will obviously need to change that so it points to the correct directory, and in your views, your partials will need to be adjusted similarly if you are using the full directory path (which you might do if the partial is in the directory of another controller).
# This works (as does the implicit version, i.e., no render statement at all)
render :action => 'show'

# This
render :template => 'samples/show'
# ...becomes this
render :template => 'sample_log/samples/show'

# This
render :partial => 'samples/list_table_row',
:collection => @samples
# ... becomes this
render :partial => 'sample_log/samples/list_table_row',
:collection => @samples

# But this stays the same
render :partial => 'list_table_row',
:collection => @samples

Functional Testing
You will need to modify your functional tests. Just as with the controllers, they need to go into an identically name directory, and put into the correct name space. Rake will find the tests in subdirectories without any prompting.
class SampleLog::SamplesControllerTest < ActionController::TestCase

Besides that, the other major change is to make sure all your paths are defined using helper methods, as in the controller.

Integration Testing
These tests will need to be modified so your HTTP requests point to the right URL, and the templates you expect are in the right folder
# This
get "/samples/home"
# ... becomes
get "/sample_log/samples/home"

# This
assert_template "samples/home"
# ... becomes
assert_template "sample_log/samples/home"

Controllers and views are closely coupled, so if you want your controllers in a subdirectory, your views must be in an identically named subdirectory. On the other hand, your models can be handled independantly (when I was trying this out, I moved the controllers and views of one section first, and had the project working fine with the corresponding models still in the top app/model directory, then I moved all the models for all the sections, and again had it working fine, then moved the remaining controllers and views). That said, it would seem to me that best practice has to be to have your directory structure identical for controllers, views and models.

You have two choices with the models. The first is to use the same namespace concept as the controllers. In this case, the database name also needs to have the nampespace prepended.
sample_log/sample.rb    # The file name
SampleLog::Sample # The class name
sample_log_samples # The database

That is probably the best way to go if you are starting from scratch, but if you are modifying an existing project, you could find that there are a lot of changes required (and so a lot of potential for errors).

The alternative is to forget the name spaces, and just make sure Rails can find your files. Two (nearly identical) approaches can be seen here:


The basic idea is that you tell Rails about the location of your models. Rails keeps an array of paths that it loads from, so you need to add your new paths to that in config/environment.rb. This code snippet adds three folders, sample_log, computer_log and user_log and would go inside the Rails::Initializer.run do |config| block.
ary = %w(sample computer user)
ary.each do |dir|
config.load_paths << "#{RAILS_ROOT}/app/models/#{dir}_log" end

That is all you need to do for your models. The unit tests can be shifted into their own subdirectory if you want (and I think you should), but none of the unit test or model files need to be changed at all.

One last point. If you put your ActionMailer templates in a subdirectory, you need to tell ActionMailer where to find them. You do that in config/environment.rb, inside the big Rails::Initializer.run block, like this:
config.action_mailer.template_root = "#{RAILS_ROOT}/app/views/#{my_sub_dir}"

Struggling with Ruby: Contents Page

Saturday, 7 November 2009

Ruby Arrays

An array is a group of values in a certain order. You can mix-and-match what you put in the array (it is all objects with Ruby), including hashes and other arrays.
a = ['one', 2, 3.0]

A quick way to create an array of strings (if each string is a single word) is like this (you can use any matching brackets, or indeed more punctuation):
a = %w(one two three)
# => ["One", "Two", "Three"]

To add an element to an existing array do this:
array << "new element"
You can join two arrays using the addition operator.
b = [4, 16]
c = a + b
# => ["One", "Two", "Three", 4, 16]

Use include? to determine if the given object is in the array. To access an array member use [], or at or fetch. The [] and at methods return nil if the index is out of range, while fetch throws an exception, or a default value of given. A negative index counts back from the end, while a range returns a subset of the array.
ary = %w(zero one two three four five six)
p ary[2]
# => "two"
p ary.at(3)
# => "three"
p ary[-1]
# => "six"
p ary[2..4]
# => ["two, "three", "four"]

Ruby has some very neat tricks with arrays. Want to delete all the elements of an array of hashes that have a body that is nil?
array.delete_if { |x| x.body.nil? }

The delete_if removes from the array any elements that evaulate to true in the block. Note that the delete_if method seems to ignore the convention of naming methods that affect the object itself with an exclamation mark.

The join method concatenates each member of an array into a long string. The supplied parameter is used to separate each item. The * operator does the same.
a = %w(one two three four)
# => ["one", "two", "three", "four"]
a * ', '
# => "one, two, three, four"

map, select and reject
The map method (aka collect) constructs a new array by processing each element in the array as per the block, while select returns a new array containing only those elements where the block evaluates to true. The reject method gives an array for the elements where the block is not true.
people = [
{:name => 'Fred', :age => 19},
{:name => 'Boris', :age => 23},
{:name => 'Mary', :age => 27},

p people.map {|e| e[:name]}
# => ["Fred", "Boris", "Mary"]

p people.select {|e| e[:age] <> [{:name=>"Fred", :age=>19}]

p people.reject {|e| e[:age] <> [{:name=>"Boris", :age=>23},
{:name=>"Mary", :age=>27}]

The sort method will, as the name suggests, sort the array, using the <=> relationship. Alternatively you can supply block to have it sorted by a custom comparison.
people.sort { |a, b| a[:age] <=> b[:age] }
# youngest to oldest

Note that your comparison must return -1, 0, or +1. In this example, I therefore could not use <.

If you use the Enumerable mixin discussed later you also have a sort_by method. This is not as fast to run, but can be quicker to code. In this method, the block determines a value for the element, for ranking purposes. Here is the previous example re-written.
people.sort_by { |a| a[:age] }

pop and push, shift and unshift
You can use push and pop to add or remove the last element, and unshift and shift to add and remove from the start.
a = %w(one two three)
# => ["one", "two", "three"]
a.push 'four'
# => ["one", "two", "three", "four"]
# => "four"
# => ["one", "two", "three"]
a.unshift 'zero'
# => ["zero", "one", "two", "three"]
# => "zero"
# => ["one", "two", "three"]

Extending Array, part 1
You can, of course, add your own methods to Array. Here are some examples.
class Array

# Shuffle an array
# from http://snippets.dzone.com/posts/show/2994
def shuffle
sort_by { rand }
def shuffle!
self.replace shuffle

# Randomly pick one element of the array.
def pick
fetch rand(length)

# Returns a total over each element in the array
# where the value for an element is determined
# by the given block. This example will look
# through an array of hashes
# and return the total of the square of values
# with the key :value
# ary.total { |e| e[:value] * e[:value] }
def total &prc
val = 0
each do |e|
val += prc.call(e)

# Returns an element that best fits the criteria
# given by the block. This example will look
# through an array of hashes and return the element
# with the highest value with the key :value
# ary.find_best { |x, y| x[:value] < y[:value] }
def find_best &prc
best = fetch(0)
each { |e| best = e if yield(best, e) }

# Returns the elements of the given array as
# string with each element listed
# in the form "one, two and three".
def list
join(', ').reverse.sub(' ,', ' dna ').reverse

Extending Array, part 2
Another way to extend Array is to "mixin" the Enumerable module.
class Array
include Enumerable

This has several interesting methods (and also some that are already in Array). Use any? and all? to determine if at least one element evalauates to true on the block, or all of them do.
a = %w(one two three)
a.any? {|e| e.length == 5}
# => true
p a.all? {|e| e.length == 5}
# => false

You can find an element in an array using find (aka detect). This method returns the first element for which the block evaulates to true. The method takes an optional parameter, this is returned if no element is found that fits.
people.find { |e| e[:age] == 23 }

The inject method combines each element of the array.
total_age = people.inject(0) { |memo, e| memo += e[:age] }
# => 69

The supplied parameter is the initial value (in this example, zero). This is technically optional, but going to be necessary in most cases (including this example) for the calculation to work in the first iteration. The memo is the ongoing value, so the increment is added to this.


Struggling with Ruby: Contents Page

Thursday, 22 October 2009


Let us say I want to be able to output a variety of objects (say strings, floats and dates) on a web page. With Java I would create a new class with over-loaded methods, like this (warning: my Java is getting rusty, and this is untested):
class Formatter {
SimpleDateFormat sdf = new SimpleDateFormat("MM/dd/yy");

public static String show(String s) { return s; }
public static String show(Date d) { return sdf.format(d); }
public static String show(double x) { return x < 0.05 ? '<0.1' : "" + (Math.round(x * 10) / 10.0); }
public static String show(Object o) { return o.toString(); }

To invoke, I would use this:

Java will select the method based on the class I send. Note that there is a method for object to catch anything unexpected.

In Ruby, I would approach this quite differently. There is no need for a new class, just modify the existing classes. This is not possible in Java; I could extend Date, but I would have to ensure that every date I sent was of my date class. String and float cannot be extended at all.
class String
def show

# Remember the require 'date.rb'
class DateTime
DATE_FORMAT = '%d/%b/%y'

def show

class Float
def show
self < 0.05 ? '<0.1' : (self * 10).round / 10.0

class Object
def show

That is more verbose, but the result is much neater, much more object-orientated, as now the method can be invoked like this:

I have a library of useful Java methods. It is a collection of static methods that do various operations on arrays and strings. In Ruby, I am building up a library that changes how arrays and string behave. Just occasionally, that is not the best way - again because of duck-typing, as it happens. Say I have a method to format dates consistently. All it does is invoke strftime with a certain format string. I could define my method as part of the DateTime class, however I would then be unable to use it with Time objects. In this case, I am better adding the method to the Object class, then DateTime, Date and Time objects would all be able to use it, as they all have strftime methods (and duck-typing allows this to work).

Wednesday, 21 October 2009

The Case Statement and Relationship Operator

Ruby supports a case statement, in which the value of something is matched against a set of options
case value
when 1, 2, 5
when 3

In this example the first when will catch three different values. Note that unlike the C family of languages, there is no break statement used. Options cannot fall though to the next one.

You do not need to give a parameter to the case statement, as seen here.
when @t == 7
p 't is 7'
when @s == :this
p 's is :this'
p 'none of the above'

In this form, the case is like an if/elsif chain.
if @t == 7
p 't is 7'
elsif @s == :this
p 's is :this'
p 'none of the above'

So why use case? Well, case returns a value, so instead we could do this:
s = case
when @t == 7
't is 7'
when @s == :this
's is :this'
'none of the above'

The Relationship Operator

The case statement uses the relationship operator, === (aka triple equals or trequals operator) when comparing the value to each when. The relationship operator is really a method, and in Object the relationship operator is defined to do the same as the equals operator. However, the important point here is that it can be overridden as required. Patterns override it to check for a match, and the Range class overrides it to check if the value is within the range. That allows you to do things like this:
mark = 56

grade = case mark
when 1..25
when 26..50
when 51..75
when 76..100
'Out of range!'

Here grade is set to 'B' because (51..75) === 56 evaulates to true. Note that this is calling the === method on 51..75. Write it the other way around, 56 === (51..75), and the === method of 56 is invoked, and the expression evaluates to false.

See more here:

Struggling with Ruby: Contents Page

Tuesday, 22 September 2009

Gotchas for Models

I have hit a few issues using Rails, some very frustrating. Here are a selection that relate to the models.

Constructors with Arguments
The first is how to handle a constructor that will take a parameter. In Ruby, when you call new for a class, the object is created, then the initialize method is invoked. If you want to do that for your model, then you need to invoke the initialize method in the superclass to ensure everything is set up, and that is done through the super keyword.

The problem is that super takes with it all the parameters that were sent to the method. If your initialize takes two parameters, both parameters get sent to the initialize in ActiveRecord::Base, which throws an exception, because it expects just one; a hash. The solution is to use brackets with super; super().

On a kind of related note, see this page about why overriding initialize may not be the best solution, as Rails sometimes creates objects another way.

Column Values
The second gotcha is accessing column values within the model. When you are not in the model, it is very simple:
@post.title = 'My new title'
s = @post.body

Ruby and Rails work together to make this seem as though you are accessing a variable in a class. However, the reality is that you are invoking methods, and this runs into problems when you do it from inside the class. You might imagine that this would work:
class Post < ActiveRecord::Base
def initialize params = {}
body = params[:body]
s = body
title = "Post: #{s[0..100]}"

It does not. The s = body line is fine; I guess Ruby notes that no variable called body exists, so invokes the method_missing method, and everything is good. Not so with title = 'My new title'. Ruby assumes that this is an assignment to a new variable, rather than a method invocation (why does the interpretor not check in the method exists first? I can only assume this is a performance issue). This has lead to some obscure bugs where values are mysteriously failing to get assigned. The solution is to prefix with self., which gives Ruby the hint that this is a method call.
class Post < ActiveRecord::Base
def test
self.title = 'My new title'
s = body

One of the most common reason my unit tests do not work is that I have done a migration on the development database, and forgotten the testing database (rake db:test:prepare).

Fixtures and Migrations
When you create a model, some default fixtures are also created for testing. If you subsequently modify the migration file, those fixtures might not work, and you will get an error like this when you run your test:
ActiveRecord::StatementInvalid: Mysql::Error: #42S22Unknown column

Reverting to Rails 2.2.2
Rails 2.3.2 adds a new helper file for new models, in units/test/helpers. If you then revert to 2.2.2, this will crash rake when you run your unit tests (but other tests will be okay).

Saving records
When you save a record (and it is not a new record), Rails will work out what has changed, and only update those fields that have actually changed. However, it is not that reliable at spotting a change. In this example, no change is made.
r = MyRecord.find 19
r.description.sub! 'this', 'that'

The problem is that Rails uses a flag on each attribute, and if you do not set the flag, the attribute is not updated. Assignment automatically sets the flag, so it is easy to over-look. The following will work fine:
r = MyRecord.find 19
r.description = r.description.sub 'this', 'that'

The alternative is to tell Rails explicitly that the attribute is being changed:
r = MyRecord.find 19
r.description.sub! 'this', 'that'

First may not be first
I have database that has been in use for some ten months ago. I wanted to retrieve the very first sample. Should be easy:

Apparently not. This was retrieving a record from only a couple of weeks ago, with an ID of 1269. And in my development database, when I tested this, it worked fine. To get the right sample, I had to specify what to order by (I have no idea what ordering brings record 1269 to the front).
Sample.find(:first, :order => "created_at ASC")

Erroneous commas in hash assignments
This one is not limited to models. If you are assigning values to a hash, an extra comma can wreak havoc! Usually extraneous punction is ignored (extra semi-colons, for example), or throws an error. Not in this case.
h = {}
h[:name] = 'Fred'
h[:desc] = 'Big'

p h.inspect
# => "{:name=>\"Fred\", :desc=>\"Big\"}"

h[:name] = 'Fred',
h[:desc] = 'Big'

p h.inspect
# => "{:name=>[\"Fred\", \"Big\"], :desc=>\"Big\"}"

Changes to a file not always noted
While you are developoing your system, chances are you will have your local web server running, and will be looking at how the web site behaves and looks as you make changes. Generally, this is fine. However, some changes that you make are not going to change the web site, for example, changes to files in config. If you have the console open, and type reload! you might find that even changes to your model (I think this may be when connected to the production database, so it is going to be very rare, admittedly). You may need to shut down the console or web server, and restart it to get the changes to have an effect.

Struggling with Ruby: Contents Page

Tuesday, 16 June 2009

Displaying Usage Statistics on a Bar Graph

My last post was about creating images of line graphs in Rails. Now I am going to have a go at histograms (bar graphs), using HTML to draw the graph, rather than creating an image.

The Data
I recently wanted to see how frequently entries were added to a table in my database. Each entry has a created_at column, so it should be easy to break this down by week or by month. Or so I thought. It was not as easy as I had hoped. There is no simple Rails way to group records within a set time interval. Instead, I had to use some actual SQL. And as my development PC is running MySQL and my server has PostgreSQL, I had to use two different dialects of SQL. It is not pretty, but this is what worked for me. First, I set up a SQL string (I did this in the controller while trying it out, but moved it to config/initializers/constants.rb later; you need to restart the web server after changing constants.rb):
module Sql
# SQL to get usage stats by week

# Different SQL for different dialects
# Both will generate the number of days since the year dot
# when the records was created
# MySQL has year and dayofyear functions
MYSQL_YEAR = 'year(created_at)'
MYSQL_DAY = 'dayofyear(created_at)'
# PostgreSQL uses the extract function for both
POSTGRESQL_YEAR = 'extract(year from created_at)'
POSTGRESQL_DAY = 'extract(doy from created_at)'
# Select the correct SQL based on the name of the database adapter
SQL_YEAR = ActiveRecord::Base.configurations[RAILS_ENV]['adapter'].match(/mysql/) ?
SQL_DAY = ActiveRecord::Base.configurations[RAILS_ENV]['adapter'].match(/mysql/) ?

# The rest of the SQL is the same
# The second line calculates the week number for the record
select count(sample_reference) as count,
floor((#{SQL_YEAR} * 365 + #{SQL_DAY}) / 7) as week,
#{SQL_YEAR} as year,
floor(#{SQL_DAY} / 7) as week_no
from samples
group by week
order by week desc


You do have to careful when invoking SQL directly that you guard against SQL injection attacks. Not a problem in this case as the SQL command is in no way altered by the end-user.

SQL does have functions for getting the week number, however, funny things happen just before the new year, with records created in week zero, but in the previous year, so I abandoned using these functions. Months would have easier, but I wanted weeks.

The Controller
After all that, the method in my controller is fairly simple. Some sites I looked at used establish_connection before connection. As I understand it, establish_connection tells Rails that you want to potentially make a connection to a database, allowing you to create actual connections as oftyen as you like in the future. Rails does that automatically for ActiveRecord, so there is no need in this situation for me to. The map method adds a new key-value pair to each hash in the array; this will be used for the labels (eg "2009 week 3").
def graph
ary = ActiveRecord::Base.connection.select_all(Sql::SQL)
@result = ary.map do |e|
e['label'] = "#{e['year']} week #{e['week_no']}"

A Generic Partial
I created a partial to display the results in the bar graph. This is a general thing that could be used in any number of web pages to display this sort of data. It uses HTML to extend a small image across a table, according to the data. Note that this uses the find_best method I described on the blocks page; this pulls out the entry with the highest count, against which all the others are scaled.
max = data.find_best { |x, y| x['count'].to_i < y['count'].to_i }['count'].to_i
data.each do |element|
count = element['count'].to_i
size = "#{count * 800 / max}x20"
<td><%=h element['label'] %></td>
<td><%=h count %></td>
<td><%= image_tag 'bar.gif', :size => size %></td>
<% end %>

The View
The partial is invoked like this:
<%= render :partial => 'bargraph',
:locals => { :data => @result } %>

In the locals hash, :data is mapped to the array that holds the data. Each entry in the array must have a 'count' value, used for the size of the bar, and another value, the key for which is 'label'.

Struggling with Ruby: Contents Page

Monday, 15 June 2009

Displaying Trends on a Graph Image

This builds on my last two posts on drawing images for Rails with Java.

My objective here was to display values from a series of records on a graph to show possible trends (specifically analytical results for samples submitted to a lab). There are two quite separate issue here involving firstly extracting data from the database table, and secondly creating an image of a line graph on-the-fly.

Getting the Data
Step one, then, is to grab the data. The easy way is to just use find all:
ary = Worksheet.find :all

However, that could involve a lot of data coming across from your database that you do not need. We are only interested in the contents of one column. Better to just extract out the data from that column.
ary = Worksheet.find :all, :select => 'result'

This gives us an array of Worksheet objects, but each object has only the specific column set to a value. We need to grab those values, and the map method gives an easy way to convert to an array of numbers. This can all go inside a class method in Worksheet (in this case the name of the column is in a variable, column_name, to keep it general).
def self.collect_data column_name
Worksheet.find(:all, :select => column_name).map do |e|

Draw the Graph
Step two is drawing the graph (see also here for drawing with Java on Rails). I created a new Ruby class in its own file in the models directory, using class methods. I wanted the graph to be able to scale itself, but for the lower and upper bounds to be round numbers; the graph_limits methods handles that.

Here is the basis of the class. Java needs to be included, and for convenience I have imported all the Java classes I will be using. Then I set up some constants:
include Java

class Graph

java_import java.io.ByteArrayOutputStream
java_import java.awt.image.BufferedImage
java_import java.awt.Color
java_import java.awt.RenderingHints
java_import java.awt.geom.GeneralPath
java_import java.awt.geom.Line2D
java_import java.awt.Font
java_import java.awt.BasicStroke
java_import javax.imageio.ImageIO

WIDTH = 800
HEIGHT = 600
FOREGROUND = Color.new 0.2, 0.2, 0.2
BACKGROUND = Color.new 0.95, 0.95, 0.95
PLOT = Color.blue
FONT = Font.new("SansSerif", Font::PLAIN, 22)


I wrote three helper methods (all protected). Note that the second uses the find_best method I described on the blocks page;
  # Converts the data point to a vertical
# position on the image,
# scaled between min and max.
def self.calc y, min, max
19.0 * HEIGHT / 20 - (y - min) *
18 * HEIGHT / (max - min) / 20

# Determines the minimum and maximum
# values from the data, and rounds them
# down and up respectively to give lower
# and upper display limits.
def self.graph_limits data
# See my blocks page for find_best method
min = data.find_best { |x, y| x > y }
max = data.find_best { |x, y| x < y }
modifier = 10 **
-(Math.log10(max - min).floor)
min = (min * modifier).floor *
1.0 / modifier
max = (max * modifier).ceil *
1.0 / modifier
# Ruby allows a method to return
# multiple values
return min, max

# Draws a line and label for the y-axis
def self.graph_line g2, text, h
g2.drawString(text, 0, h)
# Note that Line2D uses inner
# classes, accessed via ::
g2.draw(Line2D::Double.new(WIDTH / 10,
h, 9 * WIDTH / 10, h));

Now that we have the foundations, we can draw the graph.
  def self.line_graph data
min, max = graph_limits data

# Set up image object
off_image = BufferedImage.new(WIDTH, HEIGHT,
g2 = off_image.createGraphics()

# Set up background
g2.fillRect(0, 0, WIDTH - 1, HEIGHT - 1)
# Note: You get wierd results if you do not
# fill your background
g2.drawRect(0, 0, WIDTH - 1, HEIGHT - 1)

# Set up the y-axis (no x-axis drawn)
graph_line g2, min.to_s[0..5], HEIGHT * 9 / 10
graph_line g2, max.to_s[0..5], HEIGHT / 10
graph_line g2, ((min + max) / 2).to_s[0..5],

# Convert data to image coordinates
x_points = Array.new(data.length) do |i|
(i * 8.0 * WIDTH / (data.length - 1) / 10) +
WIDTH / 10
y_points = Array.new(data.length) do |i|
calc(data[i], min, max)

# Generate a GeneralPath object from the
# coordinates
polygon = GeneralPath.new(
polygon.moveTo(x_points[0], y_points[0])
(1...x_points.length).each do |index|

# Draw the GeneralPath object

# Convert for web image
os = java.io.ByteArrayOutputStream.new
ImageIO.write(off_image, "gif", os)
Ptting it Together
Step 3 is to put it all together. In my controller I put in a new method (this needs to be mentioned as a collection in your routes, as it has no specific record associated with it):
  def graph
respond_to do |format|
format.gif do
send_data Graph.line_graph(
:type => "image/gif",
:disposition => "inline"

The important part is the bit starting with send_data. The collect_data method is invoked on Worksheet, returning an array of values from the "result" column of the database table. This is sent to the line_graph method of Graph, which generates the image.

Finally, we need a view. This must be called graph.html.erb, of course, and should include the following:
<%= image_tag "/Worksheet/graph/x.gif", :size => "800x600" %>

If this was an image for a specific record in your table, you would need the id of the record. As this image is for a collection of records, no id is applicable. Nevertheless, you need something there, so I have called in "x".

Struggling with Ruby: Contents Page

Tuesday, 9 June 2009

Creating Images For Rails With Java

Some time ago I had a look at RMagick to create images on-the-fly in a Rails application. The problem with RMagick is that it is not compatible with JRuby, and if you are going to be using TomCat as your webserver, you need to be using JRuby. However, JRuby does present a ready alternative through Java.

To look at this I created a new controller, images_controller, in an existing Rails project. In hindsight, images might have been a bad name, as there is a folder called public/images already. However, Rails was able to work out whether the image should come from public/images or via my images controller, so the issue is really only whether it is confusing to the developer.

The first thing you need to do is set up the new MIME types. There are various places you can do this, but the correct one, I believe, is in config/initializers/mime_types.rb. This is what I added:
Mime::Type.register "image/jpg", :jpg
Mime::Type.register "image/png", :png
Mime::Type.register "image/gif", :gif

Now let us create the controller. The first thing you need to do is include Java. Then you should import all the classes you will need. The second step is optional, but the alternative is writing out the full name of he Java classes every time you use them, which leads to longer and harder to read code. Note that unlike in Java you cannot use the * wildcard to import a whole bunch of classes in one line. Here is the start of my controller:
class ImagesController < ApplicationController
include Java

java_import java.io.ByteArrayOutputStream
java_import java.io.File

java_import java.awt.image.BufferedImage
java_import java.awt.Color
java_import java.awt.RenderingHints
java_import java.awt.geom.GeneralPath
java_import java.awt.Font

java_import javax.imageio.ImageIO

Now I need an action defined by a method, image. A clever feature of Rails is that you can respond differently depending on the file extention, which is what I do here. The respond_to block directs the action to the correct format (as long as the format is a known MIME type).

The HTML response is blank. Rails will do the default action, which is to return a page derived from the view images/image.html.erb. The other three responses follow the same format. Each invokes the send_data method, with the data (the image) as the first parameter, followed by a hash of options. The :type is simply the MIME type. The :disposition determines if the data is to be displayed ("inline"), or downloaded ("attachment"). For attachments, you can set a :filename parameter too, and there is also a :status parameter, which conveniently defaults to 200, success.

API for send_data:
def image
@name = params[:id]

respond_to do |format|
format.jpg do
send_data get_jpeg(@name), :type => "image/jpeg",
:disposition => "inline"
format.png do
send_data get_png(@name), :type => "image/png",
:disposition => "inline"
format.gif do
send_data get_gif(@name), :type => "image/gif",
:disposition => "inline"

So that just leaves the methods that generate the image data. These should all be protected, by the way.

This first method simply grabs a file from public/images and displays it. Once it has the file, it creates an output stream, and writes the image file to that stream in the specified format. It then converts from the stream to a Ruby string, via a Java byte array.
def get_jpeg filename
imagefile = File.new("#{RAILS_ROOT}/public/images/#{filename}.jpg")
os = ByteArrayOutputStream.new
ImageIO.write(ImageIO.read(imagefile), "jpeg", os)

Converting to another format is trivial (though noticeably slower).
def get_png filename
imagefile = File.new("#{RAILS_ROOT}/public/images/#{filename}.jpg")
os = ByteArrayOutputStream.new
ImageIO.write(ImageIO.read(imagefile), "png", os)

This version does an operation, converting the image to its negative.
def get_jpeg filename
imagefile = File.new("#{RAILS_ROOT}/public/images/#{filename}.jpg")
bi = ImageIO.read(imagefile)
big = bi.getGraphics

# Create a look-up table as an array
lut = Array.new(256) { |j| 256 - j }
# Convert to Java byte array
jlut = lut.to_java :byte
# Convert to Java byte look-up table
blut = java.awt.image.ByteLookupTable.new(0, jlut)
# Convert to Java operation look-up table
op = java.awt.image.LookupOp.new(blut, nil)
# Apply the operation
dest = op.filter(bi, nil)
# Draw the new image
big.drawImage(dest, 0, 0, nil);
# Create an output stream, and write the image to it
os = ByteArrayOutputStream.new
ImageIO.write(dest, "jpeg", os)

The next method actually generates a GIF image on-the-fly, putting the filename in the middle of a simple image. The basic methodology is to create a blank BufferedImage object (rather than from a file), use that to create a Graphics2d object (just as before). The image is built by calling methods on the Graphics2d object, and finally the last three lines create the output stream and write the image to it (just as before).
WIDTH = 400
HEIGHT = 300

def get_gif filename
off_image = BufferedImage.new(WIDTH, HEIGHT,

g2 = off_image.createGraphics()

# Set up background
g2.draw3DRect(0, 0, WIDTH - 1, HEIGHT - 1, true);
g2.draw3DRect(3, 3, WIDTH - 7, HEIGHT - 7, false);

x = 7;
y = 7;
x3_points = [x, WIDTH - 2 * x, x, WIDTH - 2 * x]
y3_points = [y, HEIGHT - 2 * y, HEIGHT - 2 * y, y]
filled_polygon = GeneralPath.new(GeneralPath::WIND_EVEN_ODD,
filled_polygon.moveTo(x3_points[0], y3_points[0])
(1...x3_points.length).each do |index|
filled_polygon.lineTo(x3_points[index], y3_points[index])
g2.setFont(Font.new("Helvetica", Font::PLAIN, 22))
g2.drawString(filename, WIDTH / 2, HEIGHT / 2)

os = java.io.ByteArrayOutputStream.new
ImageIO.write(off_image, "gif", os)

Okay, so we can generate images; how do we get them on to a web browser? The easy way (to test the above work) is to invoke it though the address bar. Say I have an image saved in public/images called sheep.jpg, I can use the following:
# => Gives a negative of the image
# => Gives the PNG converted image
# => Gives the generated GIF with "sheep" written on it

What we really want to to have those images embedded in a web page. So here is a view, views/images/image.html.erb, that will do just that:
<h1>My <%= @name %> image</h1>

<%# This image has been generated on the fly and served via my controller %>
<%= image_tag "/images/image/#{@name}.gif", :size => '400x300' %>

<%# This image has been processed and served via my controller %>
<%= image_tag "/images/image/#{@name}.jpg", :size => '400x300' %>

<%# This image has been processed and served via my controller; uses HTML tags directly %>
<img alt="Sheep" height="300" src="/images/image/<%= @name %>.jpg" width="400" />

<%# This image has been processed and served via my controller as a PNG %>
<%= image_tag "/images/image/#{@name}.png", :size => '400x300' %>

<%# This image came directly from public/images %>
<%= image_tag "/images/#{@name}.jpg", :size => '400x300' %>

<%# This image has been processed and served via my controller, va a helper method %>
<%= images_image_tag @name, :jpg, :size => '400x300' %>

One of those methods uses a helper method, and that would be my prefered way. Here is that method:
def images_image_tag name, type, options = {}
image_tag "/images/image/#{@name}.#{type.to_s}", options

Struggling with Ruby: Contents Page

Monday, 8 June 2009

JRuby and Java

If you are using JRuby you can access Java functionality in your Ruby code pretty easily. Two possible reasons to do this are to create an involved user interface (for simple GUIs the Shoes toolkit is excellent, but it is somewhat limited), and to create images on-the-fly for your Rails application (RMagick offers an alternative, but cannot be used with Tomcat, which requires JRuby). I may be posting on both these later, but first, I want to explore the basics of using Java from Ruby.

I am using NetBeans, which has a particular file structure for projects, and also makes the creation of a JAR file very easy (just press F11 and the main project gets built and compressed into a JAR file inside the dist folder). For testing purposes, I created a project called JavaForRuby (and so a package name javaforruby), with a simple test class, TestObject, which has two instance variables, with setters and getters, and a static method, getStatus, which returns a String object.

And so to the Ruby side. You need to be running JRuby, of course. I had some problems getting my Java classes to work, and this seemed to be resolved by using a more recent version of JRuby (1.3.0RC1), though I am sure it should work with older versions.

The first thing to do is introduce Java to Ruby, then you need to get your JAR linked up, and then your class loaded. Once that is done, the class can be accessed:
include Java

require "#{File.dirname(__FILE__)}/../../JavaForRuby/dist/javaforruby.jar"

include_class "javaforruby.TestObject"

p TestObject.getStatus

Creating and using an instance of a Java object is simple:
tobj = TestObject.new 'My Tester', 5
p tobj.class # => Java::Javaforruby::TestObject
p tobj.java_class # => class javaforruby.TestObject
p tobj.get_name # => "class javaforruby.TestObject"My Tester"
p tobj.get_value # => 5
tobj.set_name 'My Renamed Tester'
p tobj.get_name # => "My Renamed Tester"

Note that I used the conventions of Java to define the object, but the conventions of Ruby to access it from Ruby; JRuby has associated get_name in Ruby with getName() in Java.

Let us try using an array. JRuby adds a to_java method to Array, and this takes a single parameter, the class of the objects for the array.
ja = %w(one two three).to_java :string
p ja.java_class # => [Ljava.lang.String;
p ja[0] # => "one"
p ja[0].class # => String
ja.each { |e| p e } # => "one"

What happens if you do not tell it the object type? Well, the each method still works the same; the system works out what each object is. Accessing elements by their index is not so good (I did wonder if fetch might work, but throws an NoMethodError exception).
ja = %w(one two three).to_java
p ja.java_class # => [Ljava.lang.Object;
p ja[0] # => #<Java::JavaLang::String:0x155d3a3 @java_object=#<Java::JavaObject:0x57e787>>
p ja[0].class # => "Java::JavaLang::String
ja.each { |e| p e } # => "one"

We can combine the array and the custom class to make a Java array of Java objects in Ruby.
toja = [
TestObject.new('My First Tester', 5),
TestObject.new('Second', 93),
TestObject.new('Last Tester', 42),

toja.each { |e| p e.get_name }

Hashes are no problem either.
hash = {:name => 'Fred', :age => 27}
java_hash = java.util.HashMap.new(hash)
p java_hash.get :name # => "Fred"

Rather than using the full path name in the method call, you can import each class. You should be able to import a complete package, though I could not get it to work with JRuby 1.3.0RC1 (but I could with 1.1.6).
java_import "java.util.HashMap"
# import "java.util"
# include_package "java.util"

hash = {:name => 'Fred', :age => 27}
java_hash = HashMap.new(hash)
p java_hash.get :name

You can use import rather than java_import, and many tutorials indeed do this. However, import conflicts with Rake, and so, although your system will work as expected, your tests will fail with a NameError and complaints about a 'const_missing' (thanks to Charles Oliver for pointing this out).

Also, I had a problem with import java.io.File. Ruby warns that the constant File already exists ("warning: already initialized constant File"), and I think that this gets overwritten, and the code that is expecting the original value gets very confused... You need to restart your web server after that. Again, java_import seems to fix this.

See also:

Struggling with Ruby: Contents Page

Tuesday, 12 May 2009

Date and Time in Rails

Rails adds some extra time and date functionality to Ruby, and so you can do cool things like 10.years. However, when you scratch below the surface, it all becomes a bit mysterious. I put this into a view to investigate:
[1.seconds, 1.minutes, 1.hours, 1.days, 1.months, 1.years].each do |period|
<% end %>

The output for seconds looked like this:

  • value=1

  • inspect=1 second

  • class=Fixnum

  • Duration?=true

  • to_i=1

  • to_i.class=Fixnum

  • to_i.inspect=1

It would seem that what we are using here are ActiveSupport::Duration objects, which are pretending to be Fixnum objects. Their value is the duration in seconds, but an inspect will add the units, and for days, months and years will give the value in those units (but minutes and hours are given in seconds). I would guess the point here is to fool other methods into treating these objects as Fixnum objects, while retaining the added functionality. Curiously, the years method returns an object that pretends to be a Float, even though it does admit to being an ActiveSupport::Duration.

By the way, each of these methods has a singular alias; year can be used instead of years, etc.

Something else a little odd here (I am multiply the first by 1.0 to convert to float arithmetic):
<p>Number of days in a month: <%= 1.0 * 1.months / 1.days %></p>
<p>Number of months in a year: <%= 1.years / 1.months %></p>
<p>Number of days in a year: <%= 1.years / 1.days %></p>

The result:
Number of days in a month: 30.0
Number of months in a year: 12.175
Number of days in a year: 365.25

So the Rails system quietly ignores anomolous leap years (the last was in 1900, the next in 2100 so is a reasonable approximation), but more bizarrely has slightly more than 12 months in a year. Would it not have made more sense to have 30.4375 days in a month?

Rails also introduces the ActiveSupport::TimeWithZone object. This is an object that "acts like" a Time object, but can handle different time zones.

The created_at and updated_at fields of an ActiveRecord return objects of this type. You can also convert an ActiveSupport::Duration to an ActiveSupport::TimeWithZone using ago (and its alias since) and from_now (and its alias until). Both these statements produce ActiveSupport::TimeWithZone objects:
first_sample = Sample.find(:first).created_at
now = 0.seconds.ago

Arithmetic with ActiveSupport::TimeWithZone objects will give a Float object representing a number of seconds (surely an ActiveSupport::Duration masquerading as a Float would make more sense).
elapsed = now - first_sample

You can use ActiveSupport::TimeWithZone objects in your database searches.
# Find all samples changed in the last week
Sample.find :all :conditions => ["updated_at > ?", 1.week.ago]
# Count all samples created between 1 and 2 years ago.
Sample.count :conditions => ["created_at > ? AND created_at < ?", 2.years.ago, 1.year.ago]

See also:

Struggling with Ruby: Contents Page

Saturday, 9 May 2009

Regular Expressions in ruby

A Regexp is a Ruby object representing a RegEx or "regular expression". So what exactly is a "regular expression"? It is a sort of string that can be used to match against another string. You could think of it as a template or a set of rules that a string can be compared to. Creating a Regexp object is much like creating a string, except that you use the forward slash to delimit it, rather than quote marks.
r = /my regular expression/

Alternatively, you can use this notation (you seem to be able to use any punctuation, just like %q for a string and %w for a word array):
r = %r|my regular expression|
r = %r
r = %r=my regular expression=

That regular expression will just match the string "my regular expression", anywhere in a string. The power of regular expressions lies in their use of wild cards, as we will see later.

Several standard Ruby methods take Regexp objects, but the most basic use is a simple comparison. There are two ways to do that; using the =~ operator or the match method in String. Both can be used either way around:
s = 'Here is my string'
r = /s my/
s.match r
r.match s
s =~ r
r =~ s

The difference is that the match method returns a MatchData object if a match is found, while the =~ operator gives the position of the match. However, the special variable $~ holds the MatchData for the last Regexp comparison performed, so this information is still available (personally, I do not like the built-in globals; if you want the MatchData object, use the match method, and everyone else with have a better idea of what you are doing). See later for more on MatchData.

So what can we put into a regular expression? There is a variety of options allowing you to specify your template as broadly or as narrowly as you want.
.             any character except newline
[ ] any single character of set
[^ ] any single character NOT of set
* 0 or more previous regular expression
*? 0 or more previous regular expression (non-greedy)
+ 1 or more previous regular expression
+? 1 or more previous regular expression (non-greedy)
? 0 or 1 previous regular expression
| alternation
( ) grouping regular expressions
^ beginning of a line or string
$ end of a line or string
{m,n} at least m but most n previous regular expression
{m,n}? at least m but most n previous regular expression (non-greedy)
\1-9 nth previous captured group
\A beginning of a string
\b backspace(0x08)(inside[]only)
\b word boundary(outside[]only)
\B non-word boundary
\d digit, same as[0-9]
\D non-digit
\S non-whitespace character
\s whitespace character[ \t\n\r\f]
\W non-word character
\w word character[0-9A-Za-z_]
\z end of a string
\Z end of a string, or before newline at the end
\/ forward slash

Some simple examples
Here are some examples to get us going.
# Simple pattern matches to dog
p1 = /dog/
p (p1 =~ 'cat-dog') # => 4
p (p1 =~ 'cat-doggy') # => 4
p (p1 =~ 'cat-dig') # => nil
p (p1 =~ 'cat-fox') # => nil

# Pattern matches to d, any letter, then g
p1 = /d\wg/
p (p1 =~ 'cat-dog') # => 4
p (p1 =~ 'cat-doggy') # => 4
p (p1 =~ 'cat-dig') # => 4
p (p1 =~ 'cat-fox') # => nil

# Pattern matches to d, any vowel, then g
p1 = /d[aeiou]g/
p (p1 =~ 'cat-dog') # => 4
p (p1 =~ 'cat-doggy') # => 4
p (p1 =~ 'cat-dig') # => 4
p (p1 =~ 'cat-fox') # => nil

# Pattern matches to dog at end of string
p1 = /dog\Z/
p (p1 =~ 'cat-dog') # => 4
p (p1 =~ 'cat-doggy') # => nil
p (p1 =~ 'cat-dig') # => nil
p (p1 =~ 'cat-fox') # => nil

# Pattern matches to d, anything other than o or u, then g
p1 = /d[^ou]g/
p (p1 =~ 'cat-dog') # => nil
p (p1 =~ 'cat-doggy') # => nil
p (p1 =~ 'cat-dig') # => 4
p (p1 =~ 'cat-fox') # => nil

The MatchData object
If you bracket sections of your Regexp, you can then "capture" these subsections. Each subsection can be accessed as though the MatchData object as an array, with the first element being the entire matched string (though methods like each cannot be used). Use the offset method to determine the position in the string for each group. Here it is in action:
s = "Here is a string with http://www.mydomain.com/path/to/mypage.html in it"
r = /http:\/\/([a-z.]*)(\/[a-z]*)*(\/[a-z]*.html)/i
m = r.match s
p m.string
p m.pre_match
# => "Here is a string with "
p m.post_match
# => " in it"
p m[0]
# => "http://www.mydomain.com/path/to/mypage.html"
p m.offset(0)
# => [22, 65]
p m[1]
# => "www.mydomain.com"
p m.offset(1)
# => [29, 45]
p m[2]
# => "/to"
p m.offset(2)
# => [50, 53]
p m[3]
# => "/mypage.html"
p m.offset(3)
# => [53, 65]
p m[4]
# => nil
#p m.offset(4)
# => IndexError
p m.length
# => 4
p m.size
# => 4

If you want an actual array, use to_a or captures (the latter includes only the capture groups, the former also has the entire match as the first element).
m.captures.each { |e| p e }
# => "www.mydomain.com"
# => "/to"
# => "/mypage.html"
m.to_a.each { |e| p e }
# => "http://www.mydomain.com/path/to/mypage.html"
# => "www.mydomain.com"
# => "/to"
# => "/mypage.html"

I was surprised to find that you can only capture as many subsections as you have brackets. Even though the Regexp matches one subsection to two parts of the URL ("/path" and "/to"), only the last one appears in the array.

MatchData API

Shortcut to capture groups
If you only want to pick one section out from a string, there is a quick way to do it. Both of these will pick out a number that follows a space, but the second way is much more conmcise.
# The usual way
md = s.match(/ ([0-9]+)/)
p md.nil? ? nil : md[1]

# The quick way
p s[/ ([0-9]+)+/, 1]

Note that for the first method we have to check for nil (no match is found), otherwise you will throw an error, as you are calling [] on nil. The quick way just returns nil if there is no match.

Back-references to capture groups - or not
A captured group can be refered to later in the pattern. Here is an example:
pattern = /aa(\d+)-\1/
pattern =~ 'aa1234-1234' # => 0
pattern =~ 'aa1234-1233' # => nil

The pattern requires at least one digit inside the brackets. This is the capture group. The backslash-one refers back to this group, and requires that the exact same number is repeated.

Note that capture groups number from one, rather than zero.

You may not want to have back-references to your capture group (remembering that you are limited to only 9 back-references). In the next example, question-mark-colon is used to indicate that we want to capture a group, but not to count it for back reference. We are looking for three groups of numbers in the pattern. The third should be identical to the second, but by marking the first as not counted, we can use \1 instead of \2. This trick allows you to have any number of captures, despite being limited to only nine back references.
pattern = /(?:\d+)-(\d+)-\1/
s = 'bird-cat-12-654-654-otter'
match = pattern.match s
match.to_a.each { |e| p e }
# => "12-654-654"
# => "654"
# => "done"

If you use the String.scan method, it splits a string into an array, each member of which matches the given pattern. If the pattern includes a capture group, then it is the part that is captured that goes into the array. However, if you use ?: yoiu can stop that behavior, to get the whole match (or another capture).

Multiple matches
Often you want to match multiple occurances.
\d       Match exactly one digits
\d? Match one or zero digits
\d* Match zero or more digits
\d+ Match 1 or more digits
\d{2,5} Match between 2 and 5 digits
aeiou* Match "aeio" followed by any number of "u"
[aeiou]* Match any number of vowels
(aeiou)* Match any number of sequences of "aeiou"

Greedy vs non-greedy
A greedy match will try to match against as many characters as possible, while a non-greedy will match against as few as possible. Here is a simple example to illustrate:
s = "Here another string"
greedy = /[a-z]* [a-z]*/
non_greedy = /[a-z]*? [a-z]*?/
p greedy.match(s)[0] # => "ere another"
p non_greedy.match(s)[0] # => "ere "

The * will match against a number (or zero) of the preceding, so in the two Regexp objects, they will look for a match against a group of letters, then a space, then a group of letters. The difference is the second has the ?, which makes the * non-greedy.

In both cases they ignore "H" as it does not fit, then they find a match for "e". The match continues, as both are allowed a variable number of letters, and they then match the space. Finally each can have a variable number of lower case letters. The non-greedy version aims for the fewest - in this case zero. The greedy version grabs all it can, so gets "another".

For a set of alternative characters, put them inside square brackets. For sequences, use curved brackets, separated by vertical bars.
[aeiou]   Match any one vowel
(dog|cat) Match either "dog" or "cat"

Building Regexp objects dynamically
You can use #{} when defining a Regexp, just as you can for a double-quoted string. Here is a real example that adds two new methods to the String class (the Rails API already adds them, by the way):
class String
def starts_with? sub

def ends_with? sub

The argument sent to the method gets incorporated into the Regexp. Note how ^ and $ are used to anchor the match to the start of the end of the string respectively.

Case sensitivity and other options
You can change the way the pattern matches either by appending a control code, to change the whole pattern, or using extended patterns (borrowed from Perl). These are things you can insert into a pattern inside brackets, following a question mark. For example, you can use i and -i to turn case sensitivity on and off.
# Case sensitive by default
pattern1 = /fox-cat-dog/
pattern1 =~ 'fox-cat-dog' # => 0
pattern1 =~ 'fox-CaT-dog' # => nil
pattern1 =~ 'fox-CaT-doG' # => nil

# Whole pattern modified, case insensitive
pattern2 = /fox-cat-dog/i
pattern2 =~ 'fox-cat-dog' # => 0
pattern2 =~ 'fox-CaT-dog' # => 0
pattern2 =~ 'fox-CaT-doG' # => 0

# Pattern behavior modified within the pattern
# case sensitivity turned off then back on
pattern2 = /fox-(?i)cat-(?-i)dog/
pattern2 =~ 'fox-cat-dog' # => 0
pattern2 =~ 'fox-CaT-dog' # => 0
pattern2 =~ 'fox-CaT-doG' # => nil

# Pattern behavior modified within the pattern
# case sensitivity turned off for substring
pattern2 = /fox-(?i:cat)-dog/
pattern2 =~ 'fox-cat-dog' # => 0
pattern2 =~ 'fox-CaT-dog' # => 0
pattern2 =~ 'fox-CaT-doG' # => nil

The full list of options is:
/i         case insensitive
/m multiline mode - '.' will match newline
/x extended mode - whitespace is ignored
/o only interpolate #{} blocks once
/[neus] encoding: none, EUC, UTF-8, SJIS, respectively

The last two can, I think, only be used to modify the whole pattern.

There are various other options using the brackets-question-mark notation. You can embed a comment:
pattern2 = /cat(?#comment)dog/
pattern2 =~ 'catdog' # => 0

This makes more sense with the x option just mentioned, which causes the pattern to ignore whitespace, and so allow formatting and comments like this:
pattern1 = /\d\d\d   (?# Looking for three digits  )
- (?# followed by a hash )
\d\d\d (?# and abother three digits )
p pattern1.match('578 123-678ref 567')[0]
# => "123-678"

Looking ahead
You can also look ahead at what follows, without getting the next bit including in your match. You can check that pattern is either there or is absent, as show in this example. In the first instance, pattern1 looks for three numbers follwed by "ref", but the resultant match has only the three numbers. Then pattern2 looks for three numbers not followed by a space.
pattern1 = /\d\d\d(?=ref)/
pattern2 = /\d\d\d(?! )/
pattern3 = /\d?(?! )/
p pattern1.match('578 123 678ref 567')[0]
# => "678"
p pattern2.match('578 123 678ref 567')[0]
# => "678"
p pattern3.match('578 123 678ref 567')[0]
# => "57"

And also...
One final option:
(?>)          nested anchored sub-regexp. stops backtracking.

Means nothing to me, but I mention it for completeness.

Struggling with Ruby: Contents Page

Tuesday, 21 April 2009

Using JavaScript with Ruby on Rails

Okay, this is a blog about Ruby and Rails, however, there are occasions when you want to do some processing at the client end, and JavaScript is the usual language to do that with. It is pretty straightforward to add JavaScript to your Rails web application. The strategy I employ is to write a static web page with the script on it, and get that working, and then transfer that to the Rails view.

One imortant time to use JavaScript is for user-defined code. You can save the code to your database, for use later. But rather than invoke it in Ruby on your server (which is going to be a big security risk), use JavaScript on the client machine instead. Here are a couple of examples that I have used.

Moving the focus
This script will more the focus to the first text field or text area form element on the page (after checking that there is a form). I have this in my application-wide layout, so the focus gets set for every form in my application.
// Sets the form focus to the first element found in forms[0] that
// is a textfield or text area
function setFocus() {
// Bail if no form on page
if (document.forms[0] == null) return;

// Iterate though elements
for (var i = 0; i < document.forms[0].elements.length; i++) {
e = document.forms[0].elements[i];
if ((e.type == "text") || (e.type == "textarea")) {


<body onload="setFocus()">

Two buttons to add default values
First some background: This was for a table of laboratory samples. Occasionally, analysis is not required, and I wanted a quick way to note that in the comments text area. This helper method adds a link; click on it and the text is added to the text area.
def not_required_button
js = "sample_comments.value += \'Not required.\'"
"<a onclick=\"#{js}\" class=\"not-button\">[n/r]"

The method builds a little string of JavaScript, and then uses that in some HTML. I set up the CSS class like this:
a.not-button {
color: silver;
font-size: 10px;
cursor: pointer;

The next example is a bit more complicated. Again this was for the table of laboratory samples, and there are several different types of samples requiring different fields. To handle that, I created an array of hashes; each hash held the information for one field (for example, how to display it). The edit and show actions iterate through the array, and display the fields based on the values in the hash. Several fields usually require the next number in a sequence, so what I wanted was a button the user could click to add that number. This helper method does just that:
def count_button field
return '' if !field[:count]
old_value = Sample.maximum field[:column_name]
js = "#{sample_#{field[:column_name]}}.value = \'#{old_value.nil? ? 1 : old_value + 1}\'"
"<input type=\"button\" onclick=\"#{js}\" value=\"\#\"}>"

The method accepts the hash for the field as a parameter. If the value of count is not set to true, then no button is wanted, and the method returns. Otherwise, it searches the Sample table for the highest value already there, and adds one to that (or uses one as the value if there are none there yet). Then it builds the JavaScript, in the string js. The tricky bit is working out what JavaScript will expect the element to be called; the convention is class_column, all in lowercase and underscored (you can confirm that by looking at the HTML source of your output page, of course). The JavaScript code is in turn is put into the HTML code for the button.

Struggling with Ruby: Contents Page

Monday, 23 March 2009

The Model Part 5 - Find and other ActiveRecord methods

The find method
ActiveRecord provides a lot of functionality through the find method and its derivatives. Here are some examples:
Post.find 45
# Retrieves the post with id=45
Post.find 2, 4, 5
# Gets the posts with id equal to 2, 4 and 5 (as an array)
Post.find :first
# Retrieves the first post in the database
Post.find :last
# Retrieves the last post in the database
Post.find :all
# Retrieves all the posts (as an array)
Post.find_by_author 'F2Andy'
# Retrieves the first post in the database authored by F2Andy
Post.find_all_by_author 'F2Andy'
# Retrieves all the posts in the database authored by F2Andy
Post.find_all_by_author_and_title 'F2Andy', 'ActiveRecord find'
Post.find :all, conditions => { :title => 'ActiveRecord find',
:author => 'F2Andy'}
# Both these retrieves all the posts in the database with
# the given author and title

There are various options you can include in your search, as illustrated here, which gets the 21st through to 30th posts, ordered by the title:
Post.find_all_by_author 'F2Andy', :order => 'title',
:offset => 20, :limit => 10

You can determine whether the ordering goes up or down using ASC and DESC (defaults to ASC). The offset is ignored if limit is not present.

When you know what you are searching for, ActiveRecord has it covered. However, often we do not. Do a search of Google, and you are looking for one or more words within the text. To do that for ActiveRecord you need to delve into SQL a little.

Dipping a Toe into SQL
Rails helps you to some degree, and to see how, let us perform that last search again.
Post.find :all, :conditions => ['author = ? AND title = ?',
'F2Andy', 'ActiveRecord find']

Notice the :conditions key. This maps to an array. The first element is the base string, some SQL code including question marks. All the other elements are substituted for the question marks in order. The SQL command that is used will be something like:
SELECT * FROM Post WHERE author = "F2Andy" AND
title = "ActiveRecord find"

You can use symbols rather than question marks. It is a little more long-winded, but makes it clear what goes where, and you can go in any order, so would be preferable in all but the simplest of cases.
Post.find :all,
:conditions => ['author = :author AND title = :title?',
:author => 'F2Andy', :title => 'ActiveRecord find'}]

ActiveRecord does some kind of sanitising to protect against SQL injection attacks. While the following will work, it is a bad idea, as it will bypass that process:
Post.find :all,
:conditions => 'author = "F2Andy" AND title = "ActiveRecord find"'
Post.find_by_sql 'SELECT * FROM Post WHERE author = "F2Andy" AND title = "ActiveRecord find"'

However, for complex searches, this might be your only option (an interesting page on that can be seen here).

Using Wildcards
So now we are ready to search for a fragment within a field. In SQL, you use the LIKE keyword, rather than the equals sign, and use % as a wildcard. Let us suppose I have a database of literature references, and I am searching for one author, Smith.
refs = Ref.find :all,
:conditions => [ "authors LIKE ?", "%Smith%" ]

This will return any record where "Smith" appears in the authors field.

Case Insensitive
There is no standard for doing case sensitive/insensitive seaches in SQL (and no help from Rails either), nor any standard about which should be the default (indeed, that depends on how the database is set up).

On PostgreSQL, you can use ILIKE to do case insensitive searches:
refs = Ref.find :all,
:conditions => [ "authors ILIKE ?", "%smith%" ]

On MySQL, you can change the collaton method (this seems to be the usual SQL strategy, however as each database system has its own collations, it still varies between databases; SQL Server uses SQL_Latin1_General_CP1_CI_AS I think).
refs = Ref.find :all,
:conditions => [ "authors LIKE ? COLLATE utf8_general_ci",
"%smith%" ]

What this all means is that the only safe way is to convert to all lower (or upper) case. The only way I could get this to work was to change the search term before hand (possibly due to how Rails sanitises the SQL).
term = "Smith".downcase
refs = Ref.find :all,
:conditions => [ "LOWER(authors) LIKE ?", "%#{term}%" ]

Complicated searches
So what if we want to search for muliple words in multiple fields? What you need to do is build a search string that each term with be put into, and an array of terms. Here is an addition to ActiveRecord::Base that will do just that.
class ActiveRecord::Base
# Retrieves all ActiveRecords that contain the
# user supplied keywords.
# The hash parameter should contain column name
# mappings to strings of keywords. For example:
# refs = Ref.search :authors => 'smith jones',
# :body => 'ruby search'
# This will retrieve any record containing both
# "smith" and "jones" in the authors field, and
# "ruby" and "search" in the body field.
# The search is case insensitive.
# Note that simply sending the params hash from
# the controller will not work, as this includes
# values for :action and :controller.
def self.search params
# This will becomes the base for the SQL string
sql_fragments = []
# This is the array of search terms. The first
#entry is just a place holder; this will be
# replaced by the SQL string at the end.
search_terms = ['']
params.each_pair do |k, v|
v.split.each do |e|
sql_fragments << "LOWER(#{k.to_s}) LIKE ?"
search_terms << "%#{e.downcase}%"
# Now assemble the SQL string and put it at
# the start of the array of terms.
search_terms[0] = sql_fragments.join(' AND ')
find :all, :conditions => search_terms

Other methods

The delete and destroy methods
These take an id or an array of ids to delete a set of records, returning the number of records set. The destroy method actually creates a new instance, populates it with data from the table, then calls the destroy method on the object, so will be slower, but will ensure any custom code in the destroy method and any callbacks and filters is invoked.
Post.delete 19
Post.destroy [23, 67, 103]

The delete_all and destroy_all methods
These are similar to find :all, and will accept the same sort of conditions. As with delete and destroy, destroy_all is slower as the objects are instantiated first.
Post.delete_all :conditions => [ "user_name = ? AND category = ?",
user_name, category ]

The exists? method
This method returns true if one or more records matching the condition exists. Note that it accepts either an id, or the condition itself (rather than :condition mapping to the condition)
Post.exists? 45
Post.exists? :user_name => user_name, :category => category
Post.exists? ["user_name = ? AND category = ?",
user_name, category]

As well as retrieving records, you can perform calculations, using average, maximum, minimum and sum. Examples from the API:
Person.calculate(:count, :all) # The same as Person.count
Person.average(:age) # SELECT AVG(age) FROM people...
:conditions => ['last_name != ?', 'Drake'])
# Selects the minimum age for everyone with a last name other
# than 'Drake'
Person.minimum(:age, :having => 'min(age) > 17',
:group => :last_name)
# Selects the minimum age for any family without any minors
Person.sum("2 * age")

You can also use count. Again, examples from the API:
# returns the total count of all people
# returns the total count of all people
# whose age is present in database
Person.count(:conditions => "age > 26")
Person.count(:conditions => "age > 26 AND job.salary > 60000",
:include => :job)
# because of the named association, it finds the DISTINCT
# count using LEFT OUTER JOIN.
Person.count(:conditions => "age > 26 AND job.salary > 60000",
:joins => "LEFT JOIN jobs on jobs.person_id = person.id")
# finds the number of rows matching the conditions and joins.
Person.count('id', :conditions => "age > 26")
# Performs a COUNT(id)
Person.count(:all, :conditions => "age > 26")
# Performs a COUNT(*) (:all is an alias for '*')

See also:


Struggling with Ruby: Contents Page

Wednesday, 18 March 2009

Ruby dates and times

Ruby has three classes for handling time, Date, DateTime and Time.

Time is part of the core language, while Date and DateTime are part of standard Ruby; to use Date and DateTime you will need to load in date.rb. Here is some code that initialises each with the current date/time.
t = Time.now
# => Fri Feb 06 08:56:27 +0000 2009
require 'date' # Needed for Date and DateTime
# => true
d = Date.today
# => #<Date: 4909737/2,0,2299161>
dt = DateTime.now
# => #<DateTime: 14140044704711/5760000,0,2299161>

As you can see, despite its name, the Time class holds the date as well as the time. The inspect method of Time gives a convenient output; not so Date or DateTime. However, the to_s method is a little more useful. The best option is to use the strftime method for all of them, which gives you full control over how time and dates are formated.
p t.inspect
# => "Fri Feb 06 08:56:27 +0000 2009"
p d.inspect
# => "#"
p t.to_s
# => "Fri Feb 06 08:56:27 +0000 2009"
p d.to_s
# => "2009-02-06"
p dt.to_s
# => "2009-02-06T08:56:10+00:00"
p d.strftime('%H%M on %d/%b/%y')
# => "0000 on 06/Feb/09"
p t.strftime('%H%M on %d/%b/%y')
# => "0856 on 06/Feb/09"
p dt.strftime('%H%M on %d/%b/%y')
# => "0856 on 06/Feb/09"

The full list of options (from here):
  %a - The abbreviated weekday name ("Sun")
%A - The full weekday name ("Sunday")
%b - The abbreviated month name ("Jan")
%B - The full month name ("January")
%c - The preferred local date and time representation
%d - Day of the month (01..31)
%H - Hour of the day, 24-hour clock (00..23)
%I - Hour of the day, 12-hour clock (01..12)
%j - Day of the year (001..366)
%m - Month of the year (01..12)
%M - Minute of the hour (00..59)
%p - Meridian indicator ("AM" or "PM")
%S - Second of the minute (00..60)
%U - Week number of the current year,
starting with the first Sunday as the first
day of the first week (00..53)
%W - Week number of the current year,
starting with the first Monday as the first
day of the first week (00..53)
%w - Day of the week (Sunday is 0, 0..6)
%x - Preferred representation for the date alone, no time
%X - Preferred representation for the time alone, no date
%y - Year without a century (00..99)
%Y - Year with century
%Z - Time zone name
%% - Literal "%" character

You can also create Date, Time and DateTime objects using the new method. Date and DateTime default to midnight on the 1st of January 1988, while Time defaults to the current date and time. With Date and DateTime you can set specific values. The parameter list starts with the biggest units, years, and gets smaller. Successive arguments are optional.
p Date.new.strftime('%H%M on %d/%b/%y')
# => "0000 on 01/Jan/88"
p DateTime.new.strftime('%H%M on %d/%b/%y')
# => "0000 on 01/Jan/88"
p Date.new(2006).strftime('%H%M on %d/%b/%y')
# => "0000 on 01/Jan/06"
p Date.new(2006, 4).strftime('%H%M on %d/%b/%y')
# => "0000 on 01/Apr/06"
p Date.new(2006, 4, 7).strftime('%H%M on %d/%b/%y')
# => "0000 on 07/Apr/06"
p DateTime.new(2006, 4, 7).strftime('%H%M on %d/%b/%y')
# => "0000 on 07/Apr/06"
p DateTime.new(2006, 4, 7, 8).strftime('%H%M on %d/%b/%y')
# => "0800 on 07/Apr/06"
p DateTime.new(2006, 4, 7, 8, 23).strftime('%H%M on %d/%b/%y')
# => "0823 on 07/Apr/06"

Using date and time examples.
d2 = d1 >> 2  # d2 will be two months later than d1
d2 = d1 << 2 # d2 will be two months earlier than d1
d.wday # Day of week, Monday = 1
d.yday # Day of the year
d.zone # Time zone
d.leap? # Leap year?

You can determine the different between two dates just be taking one from the other. The complication is that the result is a Rational.
d1 = Date.new 2004
d2 = Date.new 2005
d2 - d1
# => Rational(366, 1)

A Rational object consists of two numbers. Basically it is a fraction; the first number goes on top, the second number of the bottom (in mathematics, a rational number is one that can be expressed as a faction with finite digits; as opposed to, for example, pi, which is an irrational number). The number of days between the 1st January 2004 and 2005 is 366 divided by 1. You can freely use Date and DateTime objects together; the result is always the number of days expressed as a fraction, as a Rational object. You can convert your Rational object to an array cotaining the hours, minutes, second and the second fraction with Date.day_fraction_to_time.

According to here, Time is written in C, and is therefore some 20 times faster than Date/DateTime. However, it can only handle dates from 1970 to 2039 (Unix epoch)

Humanize time/date:

Struggling with Ruby: Contents Page

Sunday, 15 March 2009


In Ruby, variables are always private; they cannot be directly accessed from outside the object (constants are always public). However, it is often the case that you do need to allow some access. One way would be to define methods that set and get the values. For example:
class TestClass
def initialize id, name
@id = id
@name = name

def id

def name

def name= s
@name = s

# Test it works properly
tc = TestClass.new 12, 'Boris'
p tc.id
p tc.name
tc.name = 'Alfie'
p tc.name

Note that the id cannot be set after the object is created; it is a read-only attribute.

Ruby offers a short cut for getters and setters. The above class can be re-written like this:
class TestClass
def initialize id, name
@id = id
@name = name

attr_reader :id
attr_accessor :name

The class behaves just the same, so the test code will work here as well, but all that clutter has been removed.

There is also a method for write-only attributes, and several attributes can be listed, separated with commas:
attr_reader :size, :address, :dir

What is happening is that attr_reader is a method (in the Module class), that takes the parameter :id, and dynamically defines the id method.

Having said that, here is an interesting article (written for Java, but applicable to any object-orientated language) about why getters and setters are evil (sometimes):

Struggling with Ruby: Contents Page

Saturday, 14 March 2009


The Ruby interpreter will take anything that has a name beginning with a capital letter as being a constant.
# Variables
n = 67
s = 'my string'

N = 89
TITLE = 'My great program'
class MyClass end

Not really constant?
Variables and constants are really pointers to objects. This has some practical consequences that may be unexpected. Let us set some up:
S = s = 'My string'
N = n = 23
X = x = 12.6

Then we can see what happens what we modify the object:
s << ' is longer'
n += 4
x += 3.5

Perhaps the odd thing here is that modifying s also modifies the constant, S. But is that so odd? Both s and S point to the same string. Modify the string, and naturally what they both point to has changed. So what is surprising is that the others have not changed (as an aside, this is also the situation in Java, but in Java, numbers are primitives, not objects; in Ruby everything is an object). Numbers have one object each to represent each value (though they will not all exist in te virtual machine t any one time of course). Change the value, and it will point to a new instance that stands for the new value.

Note that for the string, I used << rather than +=. Although they both concatenate strings, The former changes the existing string, while the latter creates a new string. If I had used += in the above, s would change to point to the new string, while S would still point to the old string.

So you can readily modify the object that a constant points to, but you cannot change what object it points to... Can you? Well, yes you can. The only issue is that the interpreter gives a warning. Here is an IRb session:
irb(main):011:0> Constant = 'My string'
=> "My string"
irb(main):012:0> Constant = 56
(irb):12: warning: already initialized constant Constant
=> 56

It turns out that constants are only really constant by convention, and there is nothing to stop you changing them and no guarantee that they will remain the same (just as setting a method as private is no guarantee that it cannot be invoked from anywhere).

Struggling with Ruby: Contents Page