Wednesday, 3 March 2010

Ruby Sockets

I was messing around with Ruby sockets, and came up with a simple chat-server. Testing proved to be rather more complicated... If you run this program, you can connect to it using "telnet 6606".
require 'socket'
require 'thread'

# Boardcaster maintains a list of users.
class Broadcaster
def initialize; @users = []; end
def add user; @users << socket =" TCPServer.open(6606)" broadcaster =" Broadcaster.new" lock =" Mutex.new">")
user = {:name => s.gets.strip, :socket => s }
b.add user
print("#{user[:name]} is accepted\n")
s.write("Hello #{user[:name]}\n\rUsers on-line: #{b.list}\n\r>")
while true
st = s.gets.strip
#p st
break if st == 'bye'
lock.synchronize do
b.broadcast "#{user[:name]} says \"#{st}\"\n\r>"
end
end
lock.synchronize do
b.broadcast "#{user[:name]} has left\n\r>"
end
s.close
b.remove user
print("#{user[:name]} is gone\n")
end

end

This was my first experience of both threads and sockets on Ruby, and with regards to threads, I have to admit to being pretty clueless! However, it does seem worthwhile locking the shared resource, b, when used on a thread.

Sockets seems straightforward enough. A new socket is opened using the TCPServer class. Data is collected with gets, and sent with write. At the end it is closed. I suspect there should be some exception handling in there, but it certainly proves the concept.


Testing Stream-Handling Methods

Okay, so now I want to test my methods that handle streams. Let us suppose that you have a method that accepts data from some stream and outputs to another, like the broadcast method above, and you want to test it. How do you do it?

First, let me simplify, and instead consider this method:
def get_data source, sink
print "\n>"
name = source.gets.strip
print "\n>"
age = source.gets.strip
sink.print "Name: #{name}, age: #{age}"
end

This could be invoked for use with the keyboard like this
get_data $stdin, $stdout

Or across a network, like this:
require 'socket'
socket = TCPServer.open(6606)
get_data socket, socket

If I want to test that method the trick is to use StringIO objects.
def test_get_data1
StringIO.open { |sink|
get_data(StringIO.new("Boris\n32\n"), sink)
assert_equal "Name: Boris, age: 32", sink.string
}
end

Actually, Ruby would happily let you use the same StringIO object for both input and output, but the output would be appended to the input string, so your assertion would need to check for both the input and the output.
def test_get_data2
StringIO.open("Boris\n32\n") { |io|
get_data(io, io)
assert_equal "Boris\n32\nName: Boris, age: 32", io.string
}
end

That is bad; if we change the get_data method to accept different input, we would need to change the test in two places, and that is clearly a bad thing. Well, okay, we change it so the input gets inserted into what we expect. The problem now is that Ruby is modifying that string during the test, so we need instead to give Ruby a duplicate of the input string for it to play with, so we still have the orignal for comparison at the end.
def test_get_data3
input = "Boris\n32\n"
StringIO.open(input.clone) { |io|
get_data(io, io)
assert_equal "#{input}Name: Boris, age: 32", io.string
}
end

Then again, perhaps we need to rethink. The whole thing can be generalised into a new method, which can test any method against any input. The test itself can then be reduced to a single line.
def test_get_data4
stream_test("Boris\n32\n", "Name: Boris, age: 32") do |io|
get_data(io, io)
end
end

def stream_test input, output
StringIO.open(input.clone) { |io|
yield io
assert_equal "#{input}#{output}", io.string
}
end

A serious problem with all of these is that errors do not get caught by the test regime. A message is sent to the output, but the error is not counted in the totals (failures, on the other hand, are). I guess this is because the redirect is capturing the exception. A way around this is to capture the exception inside the block, and then flag this as a failure:
def stream_test input, output
StringIO.open(input.clone) { |io|
begin
yield io
rescue Exception => ex
assert false, "ERROR: #{ex.inspect}\n#{$!.backtrace[0..12] * "\n"}"
end
assert_equal "#{input}#{output}", io.string
}
end



Testing Multiple Threads

Those tests are all very well, but my Broadcast object sends messages to multiple users. How do I test that? Now I need threads in my tests!

Here is a test method that worked for me:
def test_broadcast
# SETTING UP

# The number of threads to spawn
number = 100
test_string = 'teststring'
b = Broadcaster.new
# Define strings outside the blocks so we can access them
# later on
string_ary = Array.new(number, '')
thread_ary = Array.new(number)
main_s = nil

# LISTENING THREADS
# A number of threads are spawned, they listen for
# messages for 0.2 seconds, then write their StringIO
# string to string_ary, before terminating.

number.times do |i|
# Spawn a new thread
thread_ary[i] = Thread.start(b, i) do
# Create a StringIO object to collect the string
StringIO.open do |sink|
# Create a new user, and add it to the Broadcaster
user = {:name => 'test1', :socket => sink }
b.add user
# Wait a short time for the message to be broadcast
# Choose wisely, 0.2 on my system led to failures
sleep(0.3)
# Set s1 to a copy of the sink string, so it
# is still around outside the block
string_ary[i] = sink.string.clone
end
end
end

# SENDING THREAD
# On the main thread, this sends the message, then waits
# for all the other threads to terminate.

# Create a StringIO object to collect the string
StringIO.open do |sink|
begin
# Create a new user, and add it to the Broadcaster
user = {:name => 'test2', :socket => sink }
b.add user
# Broadcast the test string
b.broadcast(test_string)
# Wait for the other threads to finish
# by which time the broadcast should have been received.
number.times { |i| thread_ary[i].join }
main_s = sink.string.clone
rescue Exception => ex
# Flag any exceptions as a failure
assert false, ex.inspect
end
end

# TESTING
# Test all the threads received the test_string
assert_equal test_string, main_s
number.times { |i| assert_equal test_string, string_ary[i] }
end



Struggling with Ruby: Contents Page

Saturday, 30 January 2010

Using Java Applets

I had a system that I wanted to create where the user could do some complicated manipulation of data, before sending the results to the database. It seemed to me that a Java applet would be the best way to interface with the user, but how to interface the applet with Rails?

The applet itself was very straightforward. It picks up the initial data from two parameters, rt and pc (each a list of floats, combining to make pairs of data). I also needed a public class that would return the results in a string.
package railsapplet;

public class MyApplet extends javax.swing.JApplet {
MyData data;

@Override
public void init() {
String rt = getParameter("rt");
String pc = getParameter("pc");
data = new MyData(rt, pc);
// Set up UI
}

// etc.

public String getOutput() {
return data.getOutput();
}
}

I packaged my Java in a jar file, insides public/applets. My view needed to reference that applet.
<applet codebase="http://<%= ApplicationController::SITE %>/applets"
width="400" height="400"
code="railsapplet.MyApplet.class" archive="railsapplet.jar"
name="myapplet"
id="myapplet"
align="center">
<param name="rt" value="<%= @rt.join(" ") %>">
<param name="pc" value="<%= @pc.join(" ") %>">
<hr />
If you were using a Java-enabled browser,
you would see an applet right now.
<hr />
</applet>

The code base points to the public/applets folder (using a constant, SITE, for the domain and port). The two variables @rt and @pc are arrays of floats, which are compiled into strings. These can then be picked up by the applet.

Okay, so I have got the data from the database, and into my applet. The uses plays around with it, then wants to send the results back to the database. There are number of ways to get data out of an applet. One such is to use a JSObject in the applet to communicate with elements on the web page. However, the easiest way is to use JavaScript.

This JavaScript function will search the page for the "myapplet" element, then call the getOutput() method on it (retrieving the data from the applet). The string is placed in the "output" element.
<script language="JavaScript">
function getOutput() {
document.getElementById('output').value = document.getElementById('myapplet').getOutput();
}
</script>

To get it all to work, you need a little form on the web page, with a button and a hidden input. Click the button and the results will go on the hidden input, and the form then submitted.
<% form_tag :action => :glc_update do %>
<input type="hidden" name="output" id="output" />
<button type="button" onclick="getOutput(); submit();">Okay</button>
<% end %>

After that, it is up to Rails to examine the string, extracting the results.


Struggling with Ruby: Contents Page

Tuesday, 22 December 2009

The named_scope in Rails

The named_scope method allows you to set up shortcuts to narrow searches within set parameters. It is invoked in the model, something like this:
class Computer < ActiveRecord::Base
named_scope :active, :conditions => {:in_use => true}
named_scope :networked, :conditions => "(network_id <> \"\")"
named_scope :recent, lambda { {:conditions => ["created_at > ?", 1.months.ago ] } }
end

Note the use of the lambda. If you simply compared the created_at to the date, you would be comparing it to the date the named_scope was invoked, rather than the current date (you might object that Rails reloads the model each time, so the difference might not be so much, but that does not seem to affect your named_scopes). You might also want to do this if you want to access another model. Say your database has a list of types, and this model has a field for the ID of one of those types. You want to collect all the records by the name of the type, so you need to access the table for the other model to get the ID. But that model might not be loaded when this one is, so you have to delay getting the ID. Or perhaps you want to send parameters to a named_scope through the lambda.
# Use a lambda to access a later loading model
named_scope :blue, lambda {
{ :conditions => ["(colour_id = #{Colour.find_by_name('blue').id})"] }
}
# Use a lambda to allow parameters
named_scope :located, lambda { |loc|
{ :conditions => {:location_id => loc } }
}

You can also use named_scopes for other things, such as ordering.
named_scope :ordered, :order => 'created_at ASC'
Now to get an array of computers in use, just do this:
Computer.active

You can chain named_scopes together, and also with find. Now I can list all the blue computers at location 5, in ascending order of record creation, or all the networked computers running WinXP just like this:
Computer.blue.located(5).ordered
Computer.networked.find_by_os("winxp")


References
http://ryandaigle.com/articles/2008/8/20/named-scope-it-s-not-just-for-conditions-ya-know
http://snippets.aktagon.com/snippets/210-How-to-use-named-scope-in-Rails
http://jitu-blog.blogspot.com/2009/07/looking-into-rails-namedscope.html
http://stackoverflow.com/questions/137630/encapsulating-sql-in-a-namedscope

Notes
A named_space can be used with will_paginate without a problem:
@samples = Sample.outstanding.ordered.paginate :page => params[:page], :per_page => 16

I found that I had to restart my web server when trying these out to get the named_scopes to reload.

Also, if named_scope does not like your condition, it just fails to define a new method. There is no warning or hint about what could be wrong. All you get is a method_missing complaint when you try to invoke it.


Struggling with Ruby: Contents Page

Wednesday, 16 December 2009

Using Sub-directories in Rails Projects

If you have a big project, you are going to want to break it up into parts, grouping, say, controllers for a certain part in one sub-directory. I found a couple of blog pages saying how to do this (basically you set up a name space in routes.rb, and prefix the controller class names with that name space, with the views in a similarly-named subdirectory):

http://myles.eftos.id.au/blog/2005/11/15/sub-directories-on-rails/
http://www.purpleworkshops.com/articles/grouped-controllers

However, they paint it rather simpler than it really is.

The Namespace for Controllers
Okay, so I have a number of controllers relating to a sample logging system, and I want to put them all inside a directory called sample_log. This corresponds to a Ruby namespace (because I might have a controller called TopController in each part of the system, so Rails needs a way to guarantee they are distinct). All the views need to be in their own subdirectory too, with the same name. Each of my controllers' class name needs to be prefixed with the name of the namespace:
class SampleLog::SamplesController < ApplicationController

Then you need to set up your routes, so that Rails knows you are using a namespace:
map.namespace :sample_log do |submap|
submap.resources :samples
# other controllers
end

At this point, you should be able to get pages, with a URL something like this:
http://localhost:3000/sample_log/samples/


Links
So far so good. The tricky part (especially if you already have a project that you want to do this to) is handling links in and out of the subdirectory. The standard link_to method invocation looks like this:
link_to 'Cylinders', :controller => 'cylinders'

This will generate a link within the sub-directory. How do you link to other subdirectories, or to the top level? Append a slash to your controller name, like this:
link_to 'Cylinders', :controller => '/cylinders'
link_to 'Samples', :controller => '/sample_log/samples'

The various helper methods like new_samples_path and edit_samples_path seem to work fine, but require the directory name to be appended to the method name (run rake routes to see the helper methods listed):
link_to 'List', sample_log_generic_samples_path
link_to 'Show', sample_log_generic_sample_path(@sample)
link_to 'Edit', edit_sample_log_generic_sample_path(@sample)
redirect_to sample_log_samples_path

However, Rails does not seem to be able to cope with links like this (use the helper methods just mentioned instead):
link_to 'Show', @sample
redirect_to @sample

For some reason, Rails does not provide a helper method for destroy, so you will need to given that link through the action:
link_to 'Destroy', { :action => :destroy, :id => sample.id },
:confirm => 'Are you sure?',
:method => :delete

If you use the form_for functionality, or polymophic methods (presumably with STI), you need to send the directory as a parameter (as a symbol or string), wrapped up in an array:
form_for([:sample_log, @sample]) do |f|
link_to 'Edit', edit_polymorphic_path([:sample_log, @samples[1]])


Pointing to templates and partials
Any time you explicitly invoke a template in a controller, you will obviously need to change that so it points to the correct directory, and in your views, your partials will need to be adjusted similarly if you are using the full directory path (which you might do if the partial is in the directory of another controller).
# This works (as does the implicit version, i.e., no render statement at all)
render :action => 'show'

# This
render :template => 'samples/show'
# ...becomes this
render :template => 'sample_log/samples/show'

# This
render :partial => 'samples/list_table_row',
:collection => @samples
# ... becomes this
render :partial => 'sample_log/samples/list_table_row',
:collection => @samples

# But this stays the same
render :partial => 'list_table_row',
:collection => @samples


Functional Testing
You will need to modify your functional tests. Just as with the controllers, they need to go into an identically name directory, and put into the correct name space. Rake will find the tests in subdirectories without any prompting.
class SampleLog::SamplesControllerTest < ActionController::TestCase

Besides that, the other major change is to make sure all your paths are defined using helper methods, as in the controller.

Integration Testing
These tests will need to be modified so your HTTP requests point to the right URL, and the templates you expect are in the right folder
# This
get "/samples/home"
# ... becomes
get "/sample_log/samples/home"

# This
assert_template "samples/home"
# ... becomes
assert_template "sample_log/samples/home"


Models
Controllers and views are closely coupled, so if you want your controllers in a subdirectory, your views must be in an identically named subdirectory. On the other hand, your models can be handled independantly (when I was trying this out, I moved the controllers and views of one section first, and had the project working fine with the corresponding models still in the top app/model directory, then I moved all the models for all the sections, and again had it working fine, then moved the remaining controllers and views). That said, it would seem to me that best practice has to be to have your directory structure identical for controllers, views and models.

You have two choices with the models. The first is to use the same namespace concept as the controllers. In this case, the database name also needs to have the nampespace prepended.
sample_log/sample.rb    # The file name
SampleLog::Sample # The class name
sample_log_samples # The database

That is probably the best way to go if you are starting from scratch, but if you are modifying an existing project, you could find that there are a lot of changes required (and so a lot of potential for errors).

The alternative is to forget the name spaces, and just make sure Rails can find your files. Two (nearly identical) approaches can be seen here:

http://toolmantim.com/articles/keeping_models_in_subdirectories
http://www.paperplanes.de/2007/5/2/namespacing_your_rails_model_an.html

The basic idea is that you tell Rails about the location of your models. Rails keeps an array of paths that it loads from, so you need to add your new paths to that in config/environment.rb. This code snippet adds three folders, sample_log, computer_log and user_log and would go inside the Rails::Initializer.run do |config| block.
ary = %w(sample computer user)
ary.each do |dir|
config.load_paths << "#{RAILS_ROOT}/app/models/#{dir}_log" end

That is all you need to do for your models. The unit tests can be shifted into their own subdirectory if you want (and I think you should), but none of the unit test or model files need to be changed at all.

ActionMailer
One last point. If you put your ActionMailer templates in a subdirectory, you need to tell ActionMailer where to find them. You do that in config/environment.rb, inside the big Rails::Initializer.run block, like this:
config.action_mailer.template_root = "#{RAILS_ROOT}/app/views/#{my_sub_dir}"



Struggling with Ruby: Contents Page

Saturday, 7 November 2009

Ruby Arrays

An array is a group of values in a certain order. You can mix-and-match what you put in the array (it is all objects with Ruby), including hashes and other arrays.
a = ['one', 2, 3.0]

A quick way to create an array of strings (if each string is a single word) is like this (you can use any matching brackets, or indeed more punctuation):
a = %w(one two three)
# => ["One", "Two", "Three"]

To add an element to an existing array do this:
array << "new element"
You can join two arrays using the addition operator.
b = [4, 16]
c = a + b
# => ["One", "Two", "Three", 4, 16]

Use include? to determine if the given object is in the array. To access an array member use [], or at or fetch. The [] and at methods return nil if the index is out of range, while fetch throws an exception, or a default value of given. A negative index counts back from the end, while a range returns a subset of the array.
ary = %w(zero one two three four five six)
p ary[2]
# => "two"
p ary.at(3)
# => "three"
p ary[-1]
# => "six"
p ary[2..4]
# => ["two, "three", "four"]


delete_if
Ruby has some very neat tricks with arrays. Want to delete all the elements of an array of hashes that have a body that is nil?
array.delete_if { |x| x.body.nil? }

The delete_if removes from the array any elements that evaulate to true in the block. Note that the delete_if method seems to ignore the convention of naming methods that affect the object itself with an exclamation mark.

join
The join method concatenates each member of an array into a long string. The supplied parameter is used to separate each item. The * operator does the same.
a = %w(one two three four)
# => ["one", "two", "three", "four"]
a * ', '
# => "one, two, three, four"


map, select and reject
The map method (aka collect) constructs a new array by processing each element in the array as per the block, while select returns a new array containing only those elements where the block evaluates to true. The reject method gives an array for the elements where the block is not true.
people = [
{:name => 'Fred', :age => 19},
{:name => 'Boris', :age => 23},
{:name => 'Mary', :age => 27},
]

p people.map {|e| e[:name]}
# => ["Fred", "Boris", "Mary"]

p people.select {|e| e[:age] <> [{:name=>"Fred", :age=>19}]

p people.reject {|e| e[:age] <> [{:name=>"Boris", :age=>23},
{:name=>"Mary", :age=>27}]


sort
The sort method will, as the name suggests, sort the array, using the <=> relationship. Alternatively you can supply block to have it sorted by a custom comparison.
people.sort { |a, b| a[:age] <=> b[:age] }
# youngest to oldest

Note that your comparison must return -1, 0, or +1. In this example, I therefore could not use <.

If you use the Enumerable mixin discussed later you also have a sort_by method. This is not as fast to run, but can be quicker to code. In this method, the block determines a value for the element, for ranking purposes. Here is the previous example re-written.
people.sort_by { |a| a[:age] }


pop and push, shift and unshift
You can use push and pop to add or remove the last element, and unshift and shift to add and remove from the start.
a = %w(one two three)
# => ["one", "two", "three"]
a.push 'four'
# => ["one", "two", "three", "four"]
a.pop
# => "four"
a
# => ["one", "two", "three"]
a.unshift 'zero'
# => ["zero", "one", "two", "three"]
a.shift
# => "zero"
a
# => ["one", "two", "three"]


Extending Array, part 1
You can, of course, add your own methods to Array. Here are some examples.
class Array

# Shuffle an array
# from http://snippets.dzone.com/posts/show/2994
def shuffle
sort_by { rand }
end
def shuffle!
self.replace shuffle
end


# Randomly pick one element of the array.
def pick
fetch rand(length)
end


# Returns a total over each element in the array
# where the value for an element is determined
# by the given block. This example will look
# through an array of hashes
# and return the total of the square of values
# with the key :value
# ary.total { |e| e[:value] * e[:value] }
def total &prc
val = 0
each do |e|
val += prc.call(e)
end
val
end


# Returns an element that best fits the criteria
# given by the block. This example will look
# through an array of hashes and return the element
# with the highest value with the key :value
# ary.find_best { |x, y| x[:value] < y[:value] }
def find_best &prc
best = fetch(0)
each { |e| best = e if yield(best, e) }
best
end


# Returns the elements of the given array as
# string with each element listed
# in the form "one, two and three".
def list
join(', ').reverse.sub(' ,', ' dna ').reverse
end
end


Extending Array, part 2
Another way to extend Array is to "mixin" the Enumerable module.
class Array
include Enumerable
end

This has several interesting methods (and also some that are already in Array). Use any? and all? to determine if at least one element evalauates to true on the block, or all of them do.
a = %w(one two three)
a.any? {|e| e.length == 5}
# => true
p a.all? {|e| e.length == 5}
# => false

You can find an element in an array using find (aka detect). This method returns the first element for which the block evaulates to true. The method takes an optional parameter, this is returned if no element is found that fits.
people.find { |e| e[:age] == 23 }

The inject method combines each element of the array.
total_age = people.inject(0) { |memo, e| memo += e[:age] }
# => 69

The supplied parameter is the initial value (in this example, zero). This is technically optional, but going to be necessary in most cases (including this example) for the calculation to work in the first iteration. The memo is the ongoing value, so the increment is added to this.



Ref:
http://www.ruby-doc.org/core/classes/Array.html
http://ruby-doc.org/core/classes/Enumerable.html

Struggling with Ruby: Contents Page

Thursday, 22 October 2009

Duck-typing

Let us say I want to be able to output a variety of objects (say strings, floats and dates) on a web page. With Java I would create a new class with over-loaded methods, like this (warning: my Java is getting rusty, and this is untested):
class Formatter {
SimpleDateFormat sdf = new SimpleDateFormat("MM/dd/yy");

public static String show(String s) { return s; }
public static String show(Date d) { return sdf.format(d); }
public static String show(double x) { return x < 0.05 ? '<0.1' : "" + (Math.round(x * 10) / 10.0); }
public static String show(Object o) { return o.toString(); }
}

To invoke, I would use this:
Formatter.show(myObject);

Java will select the method based on the class I send. Note that there is a method for object to catch anything unexpected.

In Ruby, I would approach this quite differently. There is no need for a new class, just modify the existing classes. This is not possible in Java; I could extend Date, but I would have to ensure that every date I sent was of my date class. String and float cannot be extended at all.
class String
def show
to_s
end
end

# Remember the require 'date.rb'
class DateTime
DATE_FORMAT = '%d/%b/%y'

def show
strftime(DATE_FORMAT)
end
end

class Float
def show
self < 0.05 ? '<0.1' : (self * 10).round / 10.0
end
end

class Object
def show
to_s
end
end

That is more verbose, but the result is much neater, much more object-orientated, as now the method can be invoked like this:
my_object.show

I have a library of useful Java methods. It is a collection of static methods that do various operations on arrays and strings. In Ruby, I am building up a library that changes how arrays and string behave. Just occasionally, that is not the best way - again because of duck-typing, as it happens. Say I have a method to format dates consistently. All it does is invoke strftime with a certain format string. I could define my method as part of the DateTime class, however I would then be unable to use it with Time objects. In this case, I am better adding the method to the Object class, then DateTime, Date and Time objects would all be able to use it, as they all have strftime methods (and duck-typing allows this to work).

Wednesday, 21 October 2009

The Case Statement and Relationship Operator

Ruby supports a case statement, in which the value of something is matched against a set of options
case value
when 1, 2, 5
do_this
when 3
do_that
else
do_the_other
end

In this example the first when will catch three different values. Note that unlike the C family of languages, there is no break statement used. Options cannot fall though to the next one.

You do not need to give a parameter to the case statement, as seen here.
case
when @t == 7
p 't is 7'
when @s == :this
p 's is :this'
else
p 'none of the above'
end

In this form, the case is like an if/elsif chain.
if @t == 7
p 't is 7'
elsif @s == :this
p 's is :this'
else
p 'none of the above'
end

So why use case? Well, case returns a value, so instead we could do this:
s = case
when @t == 7
't is 7'
when @s == :this
's is :this'
else
'none of the above'
end


The Relationship Operator

The case statement uses the relationship operator, === (aka triple equals or trequals operator) when comparing the value to each when. The relationship operator is really a method, and in Object the relationship operator is defined to do the same as the equals operator. However, the important point here is that it can be overridden as required. Patterns override it to check for a match, and the Range class overrides it to check if the value is within the range. That allows you to do things like this:
mark = 56

grade = case mark
when 1..25
'Fail'
when 26..50
'C'
when 51..75
'B'
when 76..100
'A'
else
'Out of range!'
end

Here grade is set to 'B' because (51..75) === 56 evaulates to true. Note that this is calling the === method on 51..75. Write it the other way around, 56 === (51..75), and the === method of 56 is invoked, and the expression evaluates to false.

See more here:
http://www.pmamediagroup.com/2009/07/dont-call-it-case-equality/


Struggling with Ruby: Contents Page