Thursday, 25 November 2010

Printing Labels

I recently had a need to print labels from our database. The factory produces samples; the idea is that they enter the samples into the database, which spits out a label to stick on it. Analysts then update the record as the results come in.

After some rather lengthy discussions with my IT supplier we went for a Zebra GK420t label printer. The accompanying CD has a manual for programming in a language called ZPL, allowing me to set up my labels with complete flexibility.

Configuring a Printer Port

I am in a Windows environment, and some of this is specific to Windows.

The general strategy is to use Ruby to create a file, and then copy the file to the serial port. Most PCs nowadays do not have a serial port, but you can pretend the printer is on a serial port using the "net use" command at the DOS prompt. It is first necessary to share your printer on the network. Then, at the server where your project is run from, type something like this (comp01 is the computer's network name, zebra is the share name for the printer):
net use lpt4 \\comp01\zebra /persistent:yes

You can check if the printer is still assigned to that port using "net use".

If only it were that simple. Windows runs services in their own environment, so setting the printer at the prompt is great for when you are developing the project, as you run your web server from the command prompt. Then you go live, and now the project is run as a service on Tomcat or whatever, and has no clue about what you have done at the command prompt.

The way I got around this was to put code into a file in config/initial, like this:
printer ='config/printer.cfg').strip
command = "net use lpt4 \"#{printer}\""
`net use lpt4 /delete`
result = `#{command}`

Everytime Rails starts up, this reads the printer name from a file called config/printer.cfg, deletes the old setting and applies the new.


Using the "copy" command, you can now send a file to the printer. I think it is useful to try that before getting into Ruby to ensure that works, so here is an example ZPL file (for labels 100x50 mm).
^FO50,60^ADN,36,40^FDMy first line^FS
^FO50,110^ADN,36,20^FDA second line^FS
^FO50,160^ADN,36,20^FDThe third line^FS
^FO50,220^ADN,18,10^FDMy company^FS
^FO50,245^ADN,18,10^FDCreated at 1847 on 13/Sep/10^FS

Here is some Ruby code that will generate a new file on the fly, and send it to the printer. Note that ZPL does not require that data is sent in the order in which it appears on the label.
require 'date'

class Printer
PRINTER = 'lpt4'
COMPANY = 'My Company Ltd'

def self.print_label lines
# Accept either a string with line breaks embedded,
# or an array of strings
# (assume each line is not too long)
lines = lines.split("\n") if lines.is_a? String

# A "here document" defines the basic label,
# With a box, date and company name
text = <<-END
^FO50,#{70 + 50 * lines.length}^ADN,18,10^FD#{COMPANY}^FS
^FO50,#{95 + 50 * lines.length}^ADN,18,10^FDCreated at #{'%H%M on %d/%b/%y')}^FS
^FO35,50^GB680,#{10 + 50 * lines.length},2^FS

# Each line sent is now added
(1..lines.length).each do |i|
text << "^FO50,#{60 + 50 * i}^ADN,36,20^FD#{lines[i]}^FS\n"

# The file terminator is tagged on the end
text << "^XZ"

# The file is created"print.txt", "w") { |file| file << text }

# In Rails, stop at this point if in a unit test
return 'not printed' if RAILS_ENV == "test"

# Create a DOS command
command = "copy #{RAILS_ROOT}/print.txt #{PRINTER}"

# Usually Ruby will convert forward slashes to backslashes
# in Windows, but not here
command.gsub! '/', '\\'

# Use backticks for the system command
# so we can capture the output
result = `#{command}`

# Printed okay
return 'label printed' if result =~ /1 file\(s\) copied/

# Problem, so return the error message and command
"Label failed with error \"#{result}\", command was \"#{command}\""

Printer.print_label ["My first line", "A second line", "The third line"]

You may want to consider whether two people might try to print at the same time. If the second writes a new file before the first has printed it, both users will get the second label. My understanding is that Rails handles one request at a time, so I do not think this is a problem, and for me, uses will only be printing from a single computer anyway.

Struggling with Ruby: Contents Page

Saturday, 6 November 2010

Monkeybars, a UI framework

Finding a decent UI for use with Ruby is something of an on-going quest. I blogged a couple of years ago about Ruby Shoes, which is a very neat idea, but is limited, and not really suited to big projects. I am not too sure how well it is being supported nowadays, though there is some activity at GitHub.

So recently I was looking at Monkeybars. Monkeybars is not a UI as such, but a framework for using Swing inside an IDE. It attempts to hide all the underlying Java, so you just design your interface through your IDE, and the rest is Ruby (specially, JRuby of course, for Swing).

There is a problem with Monkeybars; it does not work with recent versions of JRuby. I found an older version in an example file, and that works old (I have reported this as a bug here).


Installing is simple (this will also install the "rawr" gem):
gem install monkeybars

Create a project

To create a new project:
monkeybars my_project
cd my_project
rawr install

You than need to set it up in your IDE. Monkeybars was designed with NetBeans in mind, and that is the IDE I use too (it obviously needs both Ruby and Java included). It needs to be a Java project, as you are actually running a Java application that uses Ruby, so your new NetBeans project will be a "Java Project with Existing Sources". You need to point NetBeans to the source packages in the "my_project/src" folder. Once the project is created, right click on "Libraries" in the project browser, select "Add JAR/Folder" and select the .jar files in "my_project/lib/java".

When you first run your project, NetBeans will ask for the main file; select "org.rubyforge.rawr.Main"

Create your model-view-controller

Just like Rails, you can use a rake task to generate these (in ths example, called "main"):

rake generate ALL=src/main

This will generate the files:
* src/main/main_controller.rb
* src/main/main_view.rb
* src/main/main_model.rb

Create the UI

You actually need a fourth file, which is the UI itself. In the IDE, right click on the folder (in this case "main") in the project browser, and select New - JFrame Form, and give it a suitable name (I chose MainJFrame). You can now add components graphically, using your IDE. The important point to remember is that any component you want your Ruby code to interact with should have a Ruby-friendly name. A "Quit" option on the file menu might be "quit_menu_item", but the menu itself you can leave to the default.

To keep it simple, drag a label on to the dialog box, and set the variable name (found under the code tab) to "message".

Edit the view

The main_view.rb file is the glue between the Java UI and your Ruby code. The first thing it needs is the name of your Java UI. Then it needs to know how the components on the UI map (or more accurately, properties of the components) to the model. In this example, then we need only two lines.
class MainView < ApplicationView
set_java_class 'main.MainJFrame'
map :model => :message, :view => "message.text"

You can have as many map statements as you need, one for each component that displays data (not required for buttons, etc. that generate events only). The :view part means that one end of the link is to the text property of the JLabel that we called "message". The other end of the link is the message property in the model.

You can set a mapping to be one way, by adding a :using value. In our example, the label cannot be edited directly by the user, so we only want the data to go from the model to the UI component, not the other way around (you can flip "nil" and ":default" to have the data go the other way only, but your model needs to provide read and write access to the property even so, because of the way Monkeybars creates a new model with the view data, then transfers data from that to the real model).
map :model => :message, :view => "message.text", :using => [:default, nil]

You can also use the ":using" value to specify a method to convert the data; give the method name instead of :default.

Edit the controller

The controller needs to be told the name of the model, and the name of the view. You can also set an action for when the dialog close is clicked. All of this is done for you, and is all we need for this simple application.
class MainController < ApplicationController
set_model 'MainModel'
set_view 'MainView'
set_close_action :exit

However, if you have any way for the user to interact with your dialog box, that gets captured here. Let us suppose you have a "Quit" option on yor file menu, and you have set the variable name to "quit_menu_item". This method will respond to that menu item being selected, and ask for confirmation.
def quit_menu_item_action_performed
r = javax.swing.JOptionPane.showConfirmDialog(nil,
"Do you really want to quit?",
exit if r == javax.swing.JOptionPane::YES_OPTION

Here is another example for a button; the user presses the button and the text in two JTextAreas is used to update the model (the first JTextArea is called "text_area_1", but mapped to "text1" in the model, using map :model => :text1, :view => "text_area_1.text" in the view). The update_model method is used to transfer data to the model. Then the update method in the model is called (this is a method I have written, to do what I need in my model). Finally, update_view is called so the UI is updated to reflect the new state of the model.
def update_button_action_performed
model.text1 = view_state.model.text1
model.text2 = view_state.model.text2

What happens is that calling view_state creates a new instance of the model, and this is populated with the values from the UI. You can then copy across the values you want into the real model. A convenience method, update_model, can be used instead.
def update_button_action_performed
update_model(view_state.model, :text1, text2)

This is how the controller will handle most events, first transfer the data from the UI to the model, then call a method in the model, then update the view, so you could have one method to create a whole set.
%w(up down edit cut paste insert).each do |action|
%w(button menu_item).each do |type|
class_eval <<-"END"
def #{action}_#{type}_action_performed
model.text = view_state.model.text

You can set the value for set_close_action to :nothing, :exit, :close, :dispose or :hide. If you want other functionality, override one of those methods in the controller (I found I could override close, but not exit).

Edit the model

The model needs to include accessor methods for all the properties you mapped to in the view, and methods for all the actions. In our simple example, like this:
attr_accessor :message

You can set properties through properly defined methods, but that is something of a minefield, I have found, and better avoided.

There is no more to say about the model; Monkeybars makes no assumptions about it, and it has no parent class to inherit from (other than Object). This is where you do the real work.

Struggling with Ruby: Contents Page

Tuesday, 1 June 2010

Iterating through records

Occasionally, you want to be able to go through the records in a table one by one, i.e., from a page showing one record, you want to be able to jump to the next or previous record. One way I have seen to do this is with named_scopes. i like named_scopes, but really, they are not the solution in this case; a simple method for the model is more elegant. Compare:

Note that a named_scope returns an array. Also, I have called the method forward, as next is a keyword in Ruby. Here are the two methods to go either way.
def forward
Post.find(:all, :conditions => ["id > ?", id], :limit => 1, :order => "id ASC")[0]

def back
Post.find(:all, :conditions => ["id < ?", id], :limit => 1, :order => "id DESC")[0]

If you want a looped list, you can use these two methods, which will loop around to other end of the list. They use the fact that nil is taken as false, and that Ruby uses lazy evaluation for Boolean arithmetic. he scond part of the expression is only evaulated if the first works out to be nil. There is also a random method, so you can jump around the list. I have made this a class method so you can also jump into a random point in the list.
def forward
Post.find(:all, :conditions => ["id > ?", id], :limit => 1, :order => "id ASC")[0] || Post.find(:first, :order => "id ASC")

def back
Post.find(:all, :conditions => ["id < ?", id], :limit => 1, :order => "id DESC")[0] || Post.find(:first, :order => "id DESC")

def self.random
Post.find(:first, :offset =>rand(Post.count :all))

On your show view, add links like this:

<%= link_to 'Previous', :id => @post.back %> |
<%= link_to 'Random', :id => Post.random %> |
<%= link_to 'Next', :id => @post.forward %>

Struggling with Ruby: Contents Page

Friday, 28 May 2010

Moving to Rails 2.3.8

I have recently decided to move to Rails version 2.3.8. I have tried rails 2.3 a while ago, but it did not agree with JRuby. As JRuby 1.5 is now available, I thought I should give it another go. Though I had the same problem, I fixed it this time.

The first change is that app/application.rb must be renamed to app/application_controller.rb, which certainly makes it more consistent.

JRuby and Rake
I think this was the issue that stopped my moving to 2.3.2. Running any rake command that accesses the database (and, for example, the test tasks invoke database tasks, so will do this as well) generates this:
rake aborted!
Task not supported by 'jdbcpostgresql'

The problem seems to have been know about two years ago (see here; more recently here). When Rake has to interact with the database, it checks the hash generated from your database.yml file, and finds the adapter is called jdbcpostgresql. It then compares this against the adapters it knows, trying - and failing - to match postgresql. The solution is to edit the rakes file that handles database interactions (rails-2.3.8\lib\tasks/databases.rake) and change every occurance of "postgresql" or 'postgresql' (with quotes) to /postgresql/. Now it will sucessfully match the jdbc adapter. Obviously if you are using another database, you need to change it for that.

Is test_helper loaded twice?
I have a couple of constants defined in test/test_helper.rb, and this causes warning that they have already been defined. I can only assume this is because the file is getting loaded twice, but I could not see why that might be the case.

Also in test/test_helper.rb, you need to change:
class Test::Unit::TestCase

... to:
class ActiveSupport::TestCase

Otherwise, you will get "undefined method `use_transactional_fixtures=' "

Deprecation warnings

"Giving :session_key to SessionStore is deprecated, please use :key instead."

In config/environment.rb, change :secret_key to :key. It will look something like this:
config.action_controller.session = {
:key => '_AuthSampleLog_session',
:secret => #some big hex number

"Using assert_redirected_to with partial hash arguments is deprecated"

Usually, your redirect will specify a controller and an action, but occasionally you specify something else as well, such as an id. Now in Rails assert_redirected_to checks each of the key-value pairs that you specify (this is confusing because the API says they are all optional; considered with redirect_to they are option; they just have to match up).

So this is fine, because the test specifies the controller and the action.
# In the controller
redirect_to :controller => :audits, :action => :edit
# In the controller test
assert_redirected_to :controller => :audits, :action => :edit

In this case, the test must also specify the id, because it is given in the redirect_to.
# In the controller
redirect_to :controller => :audits, :action => :edit, :id => @audit_section.audit_id
# In the controller test
assert_redirected_to :controller => :audits, :action => :edit, :id => as.audit_id

Integration testing
I found some odd things happened with logging out in my integration tests; eventually isolated this to a known bug:

Struggling with Ruby: Contents Page

Sunday, 2 May 2010

Ruby Unit Testing

Ruby has a built-in unit testing facility, Test::Unit. To use it, create a new file, and define a class that inherits from Test::Unit::TestCase. This is best done in a separate folder, by the way, so you can easily release your project without the unit tests (Rails sets this all up for yu by the way).
# Load in the unit test classes
require 'test/unit'
# Load in your class to be tested
require 'things'

class ThingsTest < Test::Unit::TestCase

def setup
# Set up some test conditions here
# This will be invoked before each test

def teardown
# This will be invoked after each test
# Use to close connections, etc.

You will also need to define your tests. Each test goes in its own method, the name of which must begin "test" (as Ruby will search your class for such methods for testing).
def test_thing_creates_okay
# assertions

There are several assertions you can use to test the results. Here are some of the more common ones:
assert_equal      # Compares two objects
assert # Tests for true
assert_nil # Tests for nil
assert_raise # Tests an exception is thrown
assert_in_delta # Test two floats are within a given amount

Rails (since about 2.2) uses a different format for tests:
test "Thing creates okay" do
# assertions

The end result is the same. In Rails, test is a method that dynaally defines a method (in this case called "test_thing_creates_okay") with the given block.

Running Tests

If you are using NetBeans you can run a test simply by right clicking on the file being tested, and selecting test. NetBeans can work out which the relevant test is as long as it is named conventionally (eg the test for things.rb would be things_test.rb). Alternatively, you can right click on the test file, and selecting test or run.

To test from the command prompt, simply run the test file through ruby (eg ruby things_test.rb; if you are using JRuby, do jruby things_test.rb).

For a big project, you can collect all your tests together to form a test suite. Just create a Ruby file that requires each of the tests in turn, and run that through Ruby. Alternativerly, put this in your test suite, and it will automatically go though every file in that folder that has a name ending "_test.rb".
files = Dir.entries(File.dirname(__FILE__)).select do |file|
file =~ /_test.rb$/
files.each { |file| require file }

Testing private methods and variables

Here is a trivial class with an instance variable and a private method.
class UnitTestee
def initialize x
@val = x


def do_stuff y
@val * @val * y

I can test the private method using send, like this:
def test_do_stuff
@ut = 12
assert_equal 12 * 12 * 5, @ut.send(:do_stuff, 5)

Note that I have left Ruby to calculate 12 * 12 * 5; not only does this save me doing it, it makes it clearer where the number comes from (even better to use named constants, of course).

To access the instance variable, I can define a new method. Doing that in my test file means that the new method is not part of the API. Here is one way to do it (there are others):
def test_variable
@ut = 12
eval "def @ut.get_val; @val; end"
assert_equal 12, @ut.get_val


Test files often include something like this at the start:
$:.unshift File.join(File.dirname(__FILE__),'..','lib')

This simply adds the code folder (called "lib" here) to the load path, so Ruby can find the file to be tested.

Struggling with Ruby: Contents Page

Saturday, 1 May 2010

Using partials as methods

Most of my models have an associated view that lists the records on the index page. Rails generates the code, so generally I do not worry about it, but if I am doing the same thing across a dozen models/controllers, surely it would be better to just do it once? Actually, I am not sure. You do not save any typing, as Rails generates the views (assuming you are in the habit of specifying all your columns from the start), and it is less readable this way. Anyway, let us look at how it can be done, and you can decide for yourself if it is worth while or not.

The way to achieve this is through a partial. Conceptually a partial is just a method that returns a chuck of HTML code You send it a few parameters mapped to the :local key, it processes your code, and returns the HTML. I want my partial accessible from any view, so I created a new folder, app/views/shared, with a partial called _table.rhtml.

I will be invoking my table from within a view with something like this:
render :partial => 'shared/table', :locals => {
:list => SamplesHelper::WORKSHEET_LIST,
:data => @worksheets,
:links => check_role?('analyst') ? :edit : :show,

The render method is sent a hash with a :partial key that maps to the location of the file (without the underscore), and a :locals key with my parameters. This is the standard procedures for partils. Within locals I have chosen to require three parameters that will define how the table is drawn. The first, :data, is simply an array of ActiveRecord::Base objects, that is, the database records that Rails got for me.

The :links parameter determines whether the user sees links to show, to show and edit, or to show, edit and destroy. In this case, I am checking if the user has the "analyst" role; if he does, I want the edit and show links, otherwise just the show links.

Finally, the :list parameter is an array of hashes, which I chose to define in a helper file, and which might look like this:
{:heading => "Date", :column => "created_at.format_date"},
{:heading => "Type", :column => :name},
{:heading => "Sample", :column => :sample.number},

This will give me three columns, with the given headings, and the values from the given columns (or rather, method calls). Note that the first is a symbol, the others are strings. This will be explained later.

My partial looks like this:
<table align="center">
<% list.each do |item| %>
<th><%= item[:heading] %></th>
<% end %>
<th colspan="<%= [:none, :show, :exit, :destroy].index(links) %>"> </th>

<% for datum in data %>
<tr class="<%= cycle('odd', 'even') %>">
<% list.each do |item| %>
<td><%= item[:column].is_a?(Symbol) ? h(datum.send(item[:column])) : eval("datum.#{item[:column]}") %></td>
<% end %>
<td><%= link_to 'Show', { :action => :show, :id => } %></td>
<% unless links == :show %>
<td><%= link_to 'Edit', { :action => :edit, :id => } %></td>
<% end %>
<% if links == :destroy %>
<td><%= link_to 'Destroy', { :action => :destroy, :id => }, :confirm => 'Are you sure?', :method => :delete %></td>
<% end %>

<% end %>

For the headings, it iterates through the list, pulling out the :heading value. Then it iterates over the records, and for each record again iterates through the list, this time pulling out the column value. If the :column value is a symbol, the send method is invoked, and the output HTML-escaped. If the :column value is a string, an eval is performed, allowing you to do something more involved (in the example, format a date). Note that this is not HTML-escaped; I have methods that return HTML strings to, for example, highlight values a certain colour, so this preserves that feature. However, you should consider carefully if this is safe in your situation. Note that I coud have defined a formated_created_at method in my model, and invoked that method using a symbol, rather than "created_at.format_date".

The table pads out the headings over the links at the right, and adds only those links that are requested.

As an aside, I was surprised to find that the parameters you send in the :locals hash really are local variables; I expected them to be method calls, like the supposed variables for columns in ActiveRecord.

I added a section at the top of the page that verifies the parameters are there. It just throws an exception if a required parameter is missing, and defines a default for the :links value should that one be missing.
raise"list not set for _form") unless defined? list
raise"data not set for _form") unless defined? data

links = :show unless defined? links

I think it is important to document your partial, so anyone using it knows what he has to supply in the way of parameters.
This partial must be sent:
- an array of hashes called "list"
- an array of ActiveRecords in "data"; the records from the database

It can also be sent:
- a value, "links" set to one of: :edit, :show, :destroy (defaults to :show)

The hashes in the array "list" must have a :heading key and a :column key.
The :heading key should map to a string, giving the column name.
The :column key should map to a symbol or string.
If a symbol, then that will be used the send method on the record, and the
output with be HTML-escaped; use this for model column names.
If a string is supplied, it will be used in an eval
method call, and the output will not be HTML-escaped.

The last thing to do is to test your partial. This needs to be done as a functional test, because you need the infrastructure that that implies. This means we need a new action, let us call it _render, which can be defined in the ApplicationController class (but in the test_helper.rb file, so it only exists in your tests). The action firstly executes a string, params[:eval], which would set up any instance variables required for your test. The page is then rendered, using the partial as defined in params[:args].
class ApplicationController
def _render
eval(params[:eval]) unless params[:eval].nil?
render :inline => "<%= render #{params[:args]} %>"

You should create a new file with your other functional tests. You need to tell Rails which controller it will use (should be okay to use any of them), using the tests method. You may need to load in some records (or you can use fixtures), and you may need to log in.

The meat of the test is the get command, invoking the _render action defined before, with two arguments, the parameters for the partial, and the code for grabbing some ActiveRecords to show (both as strings, please note).
class PartialTest < ActionController::TestCase
tests ChembaseRefsController

def test_table
login_as 'librarian', @request

get :_render, :args => ":partial => 'shared/table', :locals => {
:list => ChembaseRefsHelper::REFS_LIST,
:data => @chembase_refs,
:links => :destroy,
}", :eval => "@chembase_refs = ChembaseRef.find :all"

doc = @response.body
assert_equal ChembaseRef.count(:all) + 1, doc.elements.to_a("table/tr").length#, "#{@response.body}\n"
assert_equal ChembaseRefsHelper::REFS_LIST.length + 1, doc.elements.to_a("table/tr/th").length#, "#{@response.body}\n"

The method ends by creating an XML document from the respoonse, and testing the number of columns and rows are what they should be.

Struggling with Ruby: Contents Page

Saturday, 24 April 2010

Operator Overloading

Ruby permits operator overloading, allowing you to define how operators will work with your own (or indeed any) classes. Here is a quick example, showing how to define the addition operator. Note that you get the += for free.
class Tester1
def initialize x
@x = x

def +(y)
@x + y

a = 5
puts(a + 3)
# => 8
a += 7
puts a
# => 12

The definition is not commutative, i.e., trying to do 3 + a would fail. To get that to work you would need to override the addition method in Integer - and I think that would be a bad idea.

The next example shows how to override comparison operators. Note that by overriding the equality operator, you get the inequality operator for free. Also note that you are not restricted to returning a boolean; in this example <= returns a string.
class Tester2
def initialize ary
@ary = ary

attr_reader :ary

def ==(y)
@ary.length == y.ary.length

def <(y)
@ary.length < y.ary.length
def <=(y)
'@ary.length < y.ary.length'

b = %w(zero one two three)
puts b ==[1, 2, 3, 4, 5])
# => false
puts b !=[1, 2, 3, 4, 5])
# => true
puts b <[1, 2, 3, 4, 5])
# => true
puts b <=[1, 2, 3, 4, 5])
# => "@ary.length < y.ary.length"

You need to be careful when overriding operators, as it has the potential to make your code unfathomable. One time I think it is particular useful, however, is to add array-like properties to a class, for example to access an underlying array, as in this example. Note how the += operator has to be defined via the + operator. Also, the index operator needs two methods if you want to be able to both read from and write to it.
class Tester3
def initialize ary
@ary = ary

def [](y)

def []=(y, value)
@ary[y] = value

def <<(y)
@ary << y

def +(y)
@ary << y

c = %w(zero one two three)
puts c[3]
# => three
c << 'four'
puts c[4]
# => four
c += 'five'
puts c[5]
# => five

As an aside, be aware that << and += are not the same for an Array object (though they are for Tester3). The += operator requires an array, the members of which are added to the existing array. The << operator appends the new object to the array; if the new object is an array, you have the new array nested inside the existing array.

You cannot override keywords, such as "or", nor can you override:
&& & || | () {} :: . ~ .. ...

Struggling with Ruby: Contents Page

Tuesday, 20 April 2010

XML and Ruby

When you come back to XML after using YAML, it is a real pain in the neck. However, sometime we have to do it.

REXML offers a comprehensive set of functions for negotiating an XML document, so this is what I used to read an XML document into a Ruby data structure.

First, the setting up. Load in the REXML library, and include it for convenience. Load in the XML file.
require "rexml/document"
include REXML

file ="materials.xml")
doc = file

In my XML, materials in the root node, and this has a number of children, msds, which in turn have a number of children, material. Each material node has a set of attributes, plus some child nodes of its own.

Each element has an attributes attribute and an elements attribute. You can iterate though these using each, but you can also select specific nodes by sending a path to the each method. I want to start by iterating through the msds nodes:
doc.elements.each("materials/msds") do |msds|
# do stuff

For each iteration though the loop, I then need to go though that elements nodes with an inener loop.

For the material nodes, I need to go though the attributes. The each method of attributes has two parameters for the block, the name and value. Easy to add these to a hash (material_data).
material.attributes.each do |name, value|
material_data[name.to_sym] = value

To extract specific values from a node, into a hash, I did this:
h = { :name => element.text,
:type =>,
:file => element.attributes["file"] }

The text method gets the inner text from the element, name gives the tag name, and attributes["file"] gets the value of the "file" attribute.

require "rexml/document"
include REXML

file ="materials.xml")
doc = file

data = []

doc.elements.each("materials/msds") do |msds|
msds_data = []
msds.elements.each("material") do |material|
material_data_ary = []
material.elements.each do |element|
h = { :name => element.text,
:type =>,
:file => element.attributes["file"] }

material_data_ary << material_data =" {" msds =""> material_data_ary }
material.attributes.each {|name, value| material_data[name.to_sym] = value }

msds_data << msds =""> msds_data, :file => msds.attributes["file"] }


Struggling with Ruby: Contents Page

Thursday, 1 April 2010

Testing for Valid HTML

There is a gem for testing valid HTML called RailsTidy, however, when I tried to use it I got "RuntimeError: can't find the symbol `tidyCreate' ", which seems to relate to calling non-Ruby code (possibly failing because I am using JRuby?). So anyway, I looked at doing my own.

The best way to approach this is to create a method that works like the usual assert_x methods, so it can be invoked like this:
def test_should_get_index
get :index
assert_response :success
assert_not_nil assigns(:posts)

The skeleton of my method is going to look like this:
def assert_validates message = ''
clean_backtrace do
msg = build_message(message, "Invalid HTML found")
assert_block(msg) do
# Code here

In common with other assert_x methods, it will accept an optional message. The clean_backtrace method does what it says. It catches AssertionFailedErrors, cleans the backtrace, and rethrows the error. I am using it to ensure backtraces from my method are consistent with the backtrace from other such methods.

Inside that block, I use the assert_block method. This is the workhorse of all assertions; it catches and counts failures and errors, based on what happens inside its block. If it returns true, the test passes. One limitation here is that it seems impossible to modify the message from within the block, so we can report back what the problem was.

Now to check the HTML is valid. I am going to cheat here, and actually check that it is valid XML, as then I can palm the work off on REXML (remember the require "rexml/document").
begin @response.body
rescue REXML::ParseException => ex

The code attempts to create an XML document from the body of the response. The block returns true if this was fine, and false if REXML threw an exception. You could put a print statement in there tto indicate where the problem is; I prefered to run any offending page through a proper validator (eg here or href="">here) once a problem is found, as it tells you exactly what it is.

Here is the whole thing, which should be inside ActiveSupport::TestCase, in test_helper.rb. I have added a constant, so you can turn validation on or off.
require "rexml/document"

# Do you want to validate?

def assert_validates message = ''
# Do not bother if validation is turned off
return unless VALIDATE
# Do not bother unless it is actually HTML
return unless @response.content_type == "text/html"

clean_backtrace do
assert_block(build_message(message, "Invalid HTML found")) do
begin @response.body
rescue REXML::ParseException => ex
puts ex

You can check a whole bunch of tests in one file by adding a teardown method, like this.
def teardown

If an action results in a redirect, this will only test that the redirect directive is valid HTML, not that the page the user is sent to is. That page is not generated during a functional test (though it will be if another action returns that page, and you test that action).

You could put the assert_validates method into a module, and include that module in both ActiveSupport::TestCase and ActionController::Integration::Session. This would allow you to validate in your integration tests too. I am not sure that that actually has any benefit; you would seem to be testing the same thing twice.

Struggling with Ruby: Contents Page

Wednesday, 31 March 2010


In Ruby (as in most OO languages) there are four sort of variables; global, class, instance and local. In languages like Java and C++, you must declare a variable before use, and where you do that determines what sort it is. In Ruby, variables are not declared, so we need another way to indicate this; a prefix.

Variables can contain any object in Ruby; you can readily assign a string to a variable that previously held an integer. Variable names must begin with a lower-case letter or underscore (after the prefix). By convention, they are all lower-case, with words separated by underscores (eg my_variable).

Local Variables

Local variables have no prefix. They exist only within a block of code, such as a method. Once the block completes, the variable disappears.
def test2
local1 = 2
local1.times do
local2 = 3
p "l1=#{local1} l2=#{local2}"
p "l1=#{local1}"
#p "l2=#{local2}" # Out of scope

Instance Variables

An instance variable belongs to that one instance of the class. It is denoted by a single at symbol (eg @var), and can be accessed from any instance of a class from within a method. They have private access... however, you can access them anyway through these methods:
instance_variables    # Array of strings, listing instance variables
remove_instance_variable # Private method

You can also use attr_accessor, attr_reader and attr_writer to allow others to access your instance variables. Note that while the above methods do require the at sign, these three methods do not.

You can also access instance variables using self. These two statements are equivalent.
self.var = 19
@var = 19

Class Variables

A class variable belongs to the whole class (and indeed subclasses), and is accessible from anywhere inside the class, or from any instance of the class. If you change the value in one instance, it will be changed in every instance. It must start with two at symbols (eg @@var).

In Rails, you can access a class variable like a class method (eg MyClass.var) for a descendant of ActiveRecord::Base (analogous to how you access column names). Outside Rails class variables have private access.

Class variables have some perhaps surprising behavior when you look at inheritance, as a value set in a subclass can affect other classes. Let us see that in action. SubClass1 has a parent, SuperClass, a sibling, SubClass2, and a child SubSubClass1.
class SuperClass
@@var1 = 11
@@var2 = 21
def self.out1; @@var1; end
def self.out2; @@var2; end

class SubClass1 < var1 =" 12" var3 =" 31" var1 =" 13" var3 =" 32" var2 =" 22"> 13
p SubClass1.out1
# => 13
p SubClass2.out1
# => 13

# @var2 is in the top class and its grandchild class. Despite "skipping
# a generation", the value set in the sub-sub-class still affects the
# super-class
p SuperClass.out2
# => 22
p SubSubClass1.out2
# => 22

# @var3 is in the sibling classes only, not in a common super-class. In this
# case setting the value in one has no effect on the other.
p SubClass1.out3
# => 31
p SubClass2.out3
# => 32

There are a set of methods available to use class variables, analogous to those for instance variables.
class_variables    # Array of strings, listing instance variables
remove_class_variable # Private method

In Rails, you can define setters and getters just as you can for instance variables; cattr_accessor, cattr_reader and cattr_writer.

Global Variables

A global variable must start with a dollar (eg $var). It accessible from any class at any time. They are generally regarded as something to avoid; better to keep sections of code isolated from each other as far as possible.

Ruby has a number of predefined global variables. I do not like to use them myself; they make code that much harder to understand, and where there is a more explicit alternative, I would always use that. However, there is a list here.

There is an interesting trace feature for global variables. The trace_var method takes either a string or a symbol representing the global variable (with the dollar sign), together with a proc object (the documentation claims it will take a string or block as well; I do not believe that that is true).
trace_var "$global_var", proc do |x|
puts "$global_var is now #{x}"

You can set more than one trace on a global variable, and you can remove them (all) with untrace_var.

The global_variables method will return an array of strings, the names of all the global variables (complete with dollar sign).

Special Variables

Ruby includes a handful of special variables:
self     # "this" in Java, etc.
nil # "null" in Java, etc., also counts as false
__FILE__ # the current file
__LINE__ # the current line number.

All Together Now

Let us them all in action. Here is some sample code.
$global_var = 32

class TestClass
@@class_var = 14

def initialize
@instance_var = 20

def test
local_var = 7
$global_var += 1
@@class_var += 1
@instance_var += 1
local_var += 1
puts "$global_var=#{$global_var}"
puts "@@class_var=#{@@class_var}"
puts "@instance_var=#{@instance_var}"
puts "local_var=#{local_var}"

Both $global_var and @@class_var are set when the file is loaded. While $global_var is available to everything, @@class_var can only be accessed from within TestClass. Looking at @instance_var, this will be set to 20 for a specific instance of TestClass when the instance is created. It will be incremented whenever test is invoked on that instance, in contrast to $global_var and @@class_var, both of which will get incremented every time test is invoked on any instance. Finally, local_var is created when it is set in the test method, and destroyed when the method terminates.

Struggling with Ruby: Contents Page

Thursday, 18 March 2010

Capturing File Uploads

Here is the requirement: The web page asks the user to upload a file. The data in the file is then used to populate a new record in the database.

The first thing I want, then, is for the view to have a file upload button. Here is what worked for me:
<% form_tag 'create', :multipart => true do %>
<%= hidden_field_tag :sample_id, %>
<%= file_field_tag 'datafile' %>
File names can only contain numbers, letters, hyphens and underscores, and must end ".csv".
Files must be on your C: drive to be successfully uploaded.
<%= submit_tag "Okay" %>

You could also use the form_for method instead, with file_field, rather than file_field_tag.

Some points to note:

You need to flag the form as "multipart", so that the file gets submitted as one part, and other data on the form as a second part. My experience is that uploaded files are a little restricted, and will not upload from a networked drive, or if there are strange characters in the name, and I warn the user of this. This may be dependant on your system. In this particular case, I am looking for a .csv file. The file type may need to be defined as a MIME type (CSV is registed by default).

The file received is of the UploadedTempfile type, and seems to be useable like any other file. As far as I can tell, Rails closes and deletes it for you. It is accessible though params in the normal way (so as params[:datafile] from the above code).
params[:datafile].each_line do |s|
# do stuff

I could not find away to process a CSV file using the CSV library; it seems to demand a filename, rather than the file itself. I was obliged to do it by hand (in my case I knew every field would be surrounded by quotes, so it is pretty easy).

In the functional test, just send the file as a parameter (with a test file in the fixtures folder).
post :create, :sample_id => Sample.find(:first).id,
:datafile =>"#{RAILS_ROOT}/test/fixtures/results.csv")

Struggling with Ruby: Contents Page

Thursday, 11 March 2010

Rails: Singularize and Pluralize

Rails has functions buit-in that will turn a word into its plural or singular. It uses this when generating models, etc. to create names that conform to the standard (so user.rb but users_controller.rb). The rules are set up in a file called:

The action all happened inside a block like this:
module ActiveSupport
Inflector.inflections do |inflect|
# definitions

There are four types of rules. The first sets a general rule for making a plural, like this:
inflect.plural(/([^aeiouy]|qu)y$/i, '\1ies')

If the regular expression in the first parameter matches, then it is replaced by the the second parameter (but note that a capture group is used, so a part of the discarded ending is still used). This rule will match a word ending in y, but not preceded by a vowel, replacing the "y" with "ies"

The second sets a rule for making a singluar, like this (which is the reverse of the previous):
inflect.singular(/([^aeiouy]|qu)ies$/i, '\1y')

Then there are the irregulars, defined like this:
inflect.irregular('person', 'people')

And those that do not change, like this:
inflect.uncountable(%w(equipment information rice money species series fish sheep))

Unfortunately, it is not perfect (not in 2.3.3 anyway), which is why I was obliged to learn about it. For example:
inflect.plural(/(octop|vir)us$/i, '\1i')

The plural of virus is viruses, not viri; octopus can use either form, though octopuses is prefered (see here).

The particular one I ran up against was this, for "metal analyses":
inflect.singular(/((a)naly|(b)a|(d)iagno|(p)arenthe|(p)rogno|(s)ynop|(t)he)ses$/i, '\1\2sis')
inflect.singular(/(^analy)ses$/i, '\1sis')

The system runs through the rules starting with the last defined, until it finds a match. For "analyses", it hits the second of the lines shown above, and returns "analysis" (why this was not put in as an irregular I cannot imagine). There is no match there for "metal analysis" as the pattern specifies the start of the string. So it looks at the previous rule. Now a match is found. However, this match defines several capture groups, the first is "analy", the second is just the "a", and both these are used in the replacement, so the resulting plural goes like this:
<whatever was before the pattern> <"analy"> <"a"> <"sis">

"metal analyses -> metal analyasis

Because the replacement uses only the first and second capture groups, the other words in that rule work fine. For "theses", for example, the "t" is in capture group 8, which is not used. And as analysis on its own gets caught by the previous rule, the error can easily be missed.

This issue was bought up as a bug, but dismissed (see here). The simple workaround is to define your own rule. I have done this in a file inside config/initializers (I have called mine called initial.rb).
ActiveSupport::Inflector.inflections do |inflect|
inflect.singular(/(analy|ba|diagno|parenthe|progno|synop|the)ses$/i, '\1sis')

You can add as many of your own rules as you like. As they get added later, they will take precedence over the existing rules.

Struggling with Ruby: Contents Page

Thursday, 4 March 2010

Adventures in programming...

Back in the day I used to love text-based adventure games (the one that stands out is Leather Goddesses of Phobos). I have made several attempts to write my own, in various languages from BASIC, to C, to Java. The use of an object-orientated language like Java was a big help, but I was recently wondering how Ruby might work (whether it will even come to anything is dubious).

Ruby has a number of apparent advantages, and a consideration of how the data file might be illustrates that.
def setup
locations = [ => :start, :name => 'The Start', :south => :hall,
:north => :lobby), => :hall, :name => 'The Great Hall', :north => :lobby), => :lobby, :name => 'The Swanky Lobby', :south => :hall),
items = [ => :sword, :name => "Widowmaker Sword", :loc => :hall), => :robes, :name => "Magical Robes", :loc => :lobby,
:wearable => true), => :health, :name => "Healing Potion", :loc => :hall,
:consume => "He gained 5 hit points.<% @char[:health] += 5 %>"),
characters = [ => :boris, :name => 'Boris', :location => 'Home',
:inv_limit => 2, :health => 5),
commands = [
{:verb => :take_from, :alias => [/^take ([\w ]+) from (\w+)$/,
/^take ([\w ]+) out of (\w+)$/,
/^remove ([\w ]+) from (\w+)$/],
:item_not_here => "It's not there."
{:verb => :get, :alias => [/^get ([\w ]+)$/,
/^take ([\w ]+)$/,
/^pick up ([\w ]+)$/],
:okay => "+char+ picked up +item+.",
:item_not_here => "It's not there.",
:inv_limit => "You're holding enough already.",
:cannot_get_static => "You can't get that!"
{:verb => :examine, :alias => [/^examine ([\w ]+)$/,
/^look at ([\w ]+)$/,
/^describe ([\w ]+)$/],
:item_not_present => "It's not here.",
{:verb => :simple_com, :alias => [/^look$/],
:script => "<%= @location.describe %>",
] locations, items, characters, commands

Data files are code

In all my previous attempts, my data files have been simple text files (perhaps in XML). This time I took the example of Rails; migrations, the routes files and rake files are all Ruby code, so there is far more you can put in there when you configure. Previously I have had to write code just to load my data; not so this time.

It should be easy to extend the system, even in run time. Create a new data file, perhaps with code to modify existing locations, and run it as a script.


Ruby just asks for you to use hashes for pretty much everything (again, Rails does this a lot). They are so easy to create, to use and to pass around. Sure, Java has hashes too, but it never encouraged me to use them (so really this is only an apparent advantage).

At its simplest, you just make everything, whether the player, an item or the location, a hash, and give it the values appropriate. Want to add a new property? Just add a new value to the relevant hashes and away you go.

Want to save the data? In one line you can save an array of hashes to YAML, and load it back in in another line.

Regular Expressions

For understanding what the user is saying, regular expressions are so useful. An input like "put the cat in the bag" can readily be matched against /^put ([\s ])+ in ([\s ])$/, and straight away you can pull out what went in where (regular expressions have been in Java since 1.4; too late for when I was trying to use it).

Dynamic Method Calls

Once the command has been recognised, the appropriate method can be invoked simply by calling send, with the :verb value from the hash. Could be done using reflection in Java, but not as easy. Earlier languages... no way.


With Ruby a scripting language, it is easy to just put scripts into the hashes. In the healing potion above, an ERB script is used to restore health.

Wednesday, 3 March 2010

Ruby Sockets

I was messing around with Ruby sockets, and came up with a simple chat-server. Testing proved to be rather more complicated... If you run this program, you can connect to it using "telnet 6606".
require 'socket'
require 'thread'

# Boardcaster maintains a list of users.
class Broadcaster
def initialize; @users = []; end
def add user; @users << socket ="" broadcaster ="" lock ="">")
user = {:name => s.gets.strip, :socket => s }
b.add user
print("#{user[:name]} is accepted\n")
s.write("Hello #{user[:name]}\n\rUsers on-line: #{b.list}\n\r>")
while true
st = s.gets.strip
#p st
break if st == 'bye'
lock.synchronize do
b.broadcast "#{user[:name]} says \"#{st}\"\n\r>"
lock.synchronize do
b.broadcast "#{user[:name]} has left\n\r>"
b.remove user
print("#{user[:name]} is gone\n")


This was my first experience of both threads and sockets on Ruby, and with regards to threads, I have to admit to being pretty clueless! However, it does seem worthwhile locking the shared resource, b, when used on a thread.

Sockets seems straightforward enough. A new socket is opened using the TCPServer class. Data is collected with gets, and sent with write. At the end it is closed. I suspect there should be some exception handling in there, but it certainly proves the concept.

Testing Stream-Handling Methods

Okay, so now I want to test my methods that handle streams. Let us suppose that you have a method that accepts data from some stream and outputs to another, like the broadcast method above, and you want to test it. How do you do it?

First, let me simplify, and instead consider this method:
def get_data source, sink
print "\n>"
name = source.gets.strip
print "\n>"
age = source.gets.strip
sink.print "Name: #{name}, age: #{age}"

This could be invoked for use with the keyboard like this
get_data $stdin, $stdout

Or across a network, like this:
require 'socket'
socket =
get_data socket, socket

If I want to test that method the trick is to use StringIO objects.
def test_get_data1 { |sink|
get_data("Boris\n32\n"), sink)
assert_equal "Name: Boris, age: 32", sink.string

Actually, Ruby would happily let you use the same StringIO object for both input and output, but the output would be appended to the input string, so your assertion would need to check for both the input and the output.
def test_get_data2"Boris\n32\n") { |io|
get_data(io, io)
assert_equal "Boris\n32\nName: Boris, age: 32", io.string

That is bad; if we change the get_data method to accept different input, we would need to change the test in two places, and that is clearly a bad thing. Well, okay, we change it so the input gets inserted into what we expect. The problem now is that Ruby is modifying that string during the test, so we need instead to give Ruby a duplicate of the input string for it to play with, so we still have the orignal for comparison at the end.
def test_get_data3
input = "Boris\n32\n" { |io|
get_data(io, io)
assert_equal "#{input}Name: Boris, age: 32", io.string

Then again, perhaps we need to rethink. The whole thing can be generalised into a new method, which can test any method against any input. The test itself can then be reduced to a single line.
def test_get_data4
stream_test("Boris\n32\n", "Name: Boris, age: 32") do |io|
get_data(io, io)

def stream_test input, output { |io|
yield io
assert_equal "#{input}#{output}", io.string

A serious problem with all of these is that errors do not get caught by the test regime. A message is sent to the output, but the error is not counted in the totals (failures, on the other hand, are). I guess this is because the redirect is capturing the exception. A way around this is to capture the exception inside the block, and then flag this as a failure:
def stream_test input, output { |io|
yield io
rescue Exception => ex
assert false, "ERROR: #{ex.inspect}\n#{$!.backtrace[0..12] * "\n"}"
assert_equal "#{input}#{output}", io.string

Testing Multiple Threads

Those tests are all very well, but my Broadcast object sends messages to multiple users. How do I test that? Now I need threads in my tests!

Here is a test method that worked for me:
def test_broadcast

# The number of threads to spawn
number = 100
test_string = 'teststring'
b =
# Define strings outside the blocks so we can access them
# later on
string_ary =, '')
thread_ary =
main_s = nil

# A number of threads are spawned, they listen for
# messages for 0.2 seconds, then write their StringIO
# string to string_ary, before terminating.

number.times do |i|
# Spawn a new thread
thread_ary[i] = Thread.start(b, i) do
# Create a StringIO object to collect the string do |sink|
# Create a new user, and add it to the Broadcaster
user = {:name => 'test1', :socket => sink }
b.add user
# Wait a short time for the message to be broadcast
# Choose wisely, 0.2 on my system led to failures
# Set s1 to a copy of the sink string, so it
# is still around outside the block
string_ary[i] = sink.string.clone

# On the main thread, this sends the message, then waits
# for all the other threads to terminate.

# Create a StringIO object to collect the string do |sink|
# Create a new user, and add it to the Broadcaster
user = {:name => 'test2', :socket => sink }
b.add user
# Broadcast the test string
# Wait for the other threads to finish
# by which time the broadcast should have been received.
number.times { |i| thread_ary[i].join }
main_s = sink.string.clone
rescue Exception => ex
# Flag any exceptions as a failure
assert false, ex.inspect

# Test all the threads received the test_string
assert_equal test_string, main_s
number.times { |i| assert_equal test_string, string_ary[i] }

Struggling with Ruby: Contents Page

Saturday, 30 January 2010

Using Java Applets

I had a system that I wanted to create where the user could do some complicated manipulation of data, before sending the results to the database. It seemed to me that a Java applet would be the best way to interface with the user, but how to interface the applet with Rails?

The applet itself was very straightforward. It picks up the initial data from two parameters, rt and pc (each a list of floats, combining to make pairs of data). I also needed a public class that would return the results in a string.
package railsapplet;

public class MyApplet extends javax.swing.JApplet {
MyData data;

public void init() {
String rt = getParameter("rt");
String pc = getParameter("pc");
data = new MyData(rt, pc);
// Set up UI

// etc.

public String getOutput() {
return data.getOutput();

I packaged my Java in a jar file, insides public/applets. My view needed to reference that applet.
<applet codebase="http://<%= ApplicationController::SITE %>/applets"
width="400" height="400"
code="railsapplet.MyApplet.class" archive="railsapplet.jar"
<param name="rt" value="<%= @rt.join(" ") %>">
<param name="pc" value="<%= @pc.join(" ") %>">
<hr />
If you were using a Java-enabled browser,
you would see an applet right now.
<hr />

The code base points to the public/applets folder (using a constant, SITE, for the domain and port). The two variables @rt and @pc are arrays of floats, which are compiled into strings. These can then be picked up by the applet.

Okay, so I have got the data from the database, and into my applet. The uses plays around with it, then wants to send the results back to the database. There are number of ways to get data out of an applet. One such is to use a JSObject in the applet to communicate with elements on the web page. However, the easiest way is to use JavaScript.

This JavaScript function will search the page for the "myapplet" element, then call the getOutput() method on it (retrieving the data from the applet). The string is placed in the "output" element.
<script language="JavaScript">
function getOutput() {
document.getElementById('output').value = document.getElementById('myapplet').getOutput();

To get it all to work, you need a little form on the web page, with a button and a hidden input. Click the button and the results will go on the hidden input, and the form then submitted.
<% form_tag :action => :glc_update do %>
<input type="hidden" name="output" id="output" />
<button type="button" onclick="getOutput(); submit();">Okay</button>
<% end %>

After that, it is up to Rails to examine the string, extracting the results.

Struggling with Ruby: Contents Page