Ruby File Access

Reading Files
Note that while Ruby accepts backslashes, it seems to prefer forward slashes in file paths, even on Windows.

You can access the contents of a file like this:

file = File.open(filename, 'r')
do_stuff_with file
file.close

To test if the end of the file has been reached, use file.eof?. To test if a file has been closed, use file.closed?.

You can read the entire contents of file in a block:

contents = File.open(filename, 'r') { |file| file.read }

Or just:

contents = File.open('test.txt', 'r').read

If you prefer, you can put each line of the file into an array:

lines = []
File.open(filename, 'rb') { |file|
file.each_line { |line|
 lines << line
}
}

Note that file here is exactly the same object as in the very first example. The big advantage of using it in a block is that Ruby will ensure the file is closed for you. You can use each_byte to iterate through the bytes (or, of course, your custom do_stuff_with file method). You can break lines at any character, for example, at the full stops:

lines = []
File.open(filename, 'rb') { |file|
file.each_line('.') { |line|
 lines << line
}
}

Alternatively, use IO.foreach:

lines = []
IO.foreach(filename, '.') { |line|
lines << line
}

Or simply use IO.readlines (why was it not named read_lines?):

lines = IO.readlines(filename, '.')

Writing Files
To write data to a file, you can do something like this:

file = File.new("test.txt", "w")
file.syswrite("First line of text")
file.syswrite("Second line of text")
file.close

However, the prefered way, as with reading a file, is to do it in a block, and leave it to Ruby to close the file:

File.open("test.txt", "w") { |file|
file.syswrite("First line of text")
file.syswrite("Second line of text")
}

As well as syswrite, you can also use write, print, puts (adds a return at the end of the line), p (alias for puts) and printf (formated print, just like the C function of the same name). Or you can use the append function:

File.open("test.txt", "w") { |file|
file << "First line of text"
file << "Second line of text"
}

Binary Files
Binaries files are just the same; you just add a b to the second parameter of the open method. This example reads the contents of a binary file to contents (which is a string, by the way), and then writes that to a new binary file:

contents = File.open(in_filename, 'rb').read

File.open(out_filename, 'wb') { |file|
file << contents
}

Manipulating Files
These are all pretty self-explanatory:

File.rename(old_filename, new_filename)
File.exists?(filename)
File.file?(filename)
File.delete(filename)
File.directory?(filename)
File.executable?(filename)  # Returns true for a text file!
File.readable?(filename)
File.writable?(filename)
File.zero?(filename)
File.size(filename)   # Return 0 if the size is zero
File.size?(filename)   # Returns nil if the size is zero
File.ftype(filename)

Using Directories
You can list the contents of a directory at least three way:

Dir.entries(dir)  # Surprisingly, no default for current directory
Dir["#{dir}/*.txt"] #
`dir`

The first and second will produce arrays of the files in the given folders. The first has just the filenames, and starts with entries for "." and "..". The second has the full path for each file, but does have the facility to filter (as in the example, only .txt files will be listed). The third uses a backquoted string to invoke a system command, and will produce a flat string with the same contents that you would see if you typed "dir" at the command prompt (presumably an error on some operating systems).

Other methods:

Dir.pwd                  # Gets the current directory
Dir.chdir('c:')          # Changes the current directory (to c:)
Dir.mkdir('newfolder')   # Create a new directory
Dir.delete('newfolder')  # Or unlink or rmdir

Putting It Together
Here is an actual program. It uses the directory functionality to create an array of file names conforming to the filter (in this case, files end .nif; NetImmersion format), then iterates through the array. For each filename, the file is opened in binary format, the contents read to a string, and then each occurance of one set of strings replaced by another (this was subst, an array of hashes, giving replacement names for texture files). Finally, the file is saved, again in binary format, with a new filename.

files = Dir["#{mesh_dir}/in_r*.nif"]
print "Found #{files.length} files.\n"
Dir.chdir(mesh_dir)
files.each { |f|
contents = File.open(f, 'rb').read
subst.each { |h|
 contents.gsub!(h[:old], h[:new])
}
File.open(f.gsub(/In_R/i, 'And'), 'wb') { |file|
 file << contents
}
print '.'
}
print "\nDone.\n"

File data in with code?
If you put __END__ in your code, Ruby execution will terminate at that point. Everything beyond that will be considered data, and can be accessed as a file, DATA. Here is a complete program to illustrate that:

p DATA.read

__END__

Your data goes here

Reading a CSV file
This is trivial, as Ruby provides a library to do just this.

require "csv"
values = CSV.read "C:/my-data.csv"

This gives you your data in a two dimensional array.


Struggling with Ruby: Contents Page

Contents Page

This is not a blog in the true sense, where I create a new post each time I learn something. Instead, I am collecting what I learn into a collection of larger posts, and updating those posts as I learn more. Below is a list (a contents page if you like) of posts to date (22/Dec/09) grouped by topic, rather than chronologically, for easy reference.

Ruby Basics
Constants
Variables
Conditional statements
Case statements and relationship operator
Loops
Regular Expressions
Exception Handling

Methods, classes, etc.
Ruby Classes
Duck-typing
Methods - Basics
Methods - Not Overloading
Methods - method_missing
Methods - calling
Modules
Singletons
Proc objects
Blocks
Accessors
Operator Overloading

The Ruby Library (excluding Rails)
Hashes
Arrays
Strings
File Access
Time, Date and DateTime
Unit Testing
YAML
REXML for XML
Sockets
GUI with Shoes
GUI with Monkeybars (and Swing)

Miscellaneous
JRuby and Java

Links to APIs
Because I am always refering to these three...
Array
String
Hash

Rails
Building a Project
Basic Project
Deploying on Tomcat

The Model
Creation
Validation and Association
Interactive Rails Environment
Testing
Find and other methods
Date and Time
Gotchas

The Controller
Routing
Basic Methods
Render and Filter
Functional Testing
Testing for valid HTML

The View
ERB and Links
Scope, Helpers and Partials
Using Forms
Using Select/Options
Partials as methods

Handling Images On Rails
Dynamic Images with RMagick
Dynamic Images with Java
Generating Line Graphs with Java
Uploadable Images with File Column

More On Rails
Integration Testing
Rails Mailer
Usage Statistics on a Bar Graph with HTML
Single Table Inheritance
Moving to Rails 2.2.2
Moving to Rails 2.3.8
Using JavaScript
Using Java applets
Organising in Subdirectories
named_scope
Iterating through records
Singularize and Pluralize
Capturing file uploads

Useful Links
General Ruby and Rails
Good Rails tutorial that go beyond the very basic:
http://www.akitaonrails.com/2008/5/25/rolling-with-rails-2-1-the-first-full-tutorial-part-1

Book on Ruby (not Rails):
http://ruby.activeventure.com/programmingruby/book/index.html

FAQ
http://www.ruby-doc.org/docs/ruby-doc-bundle/FAQ/FAQ.html

Cheatsheet:
http://www.ilovejackdaniels.com/cheat-sheets/ruby-on-rails-cheat-sheet/

Quick ref:
http://www.zenspider.com/Languages/Ruby/QuickRef.html

Others
PDF generation:
http://prawn.majesticseacreature.com/

XML handling with REXML
http://www.germane-software.com/software/rexml/docs/tutorial.html
http://www.developer.com/lang/article.php/3672621

The Controller Part 4 - Functional Testing

Functional testing targets individual methods in your controllers. It tests the user is redirected, gets the right HTTP response and so on, without actually generating any web pages (though it does generate the HTML so will find errors there).

Setting it up
Rails creates functional tests, but there are a few things to be done first... The test database must be set up (though this was probably done when you did your unit testing):

rake db:test:prepare

As with unit testing the require needs altering, if you are using NetBeans 6.1: require 'test_helper' becomes require 'test/test_helper'. You should now be able to run the default tests put there by Rails (depending on the changes you have made to the controller and model). Outside of an IDE, the easiest way is using rake.

rake test (run all tests)
rake test:functionals (run all functional tests)
rake test:recent (run all test files modified in last 10 minutes)

There are typically three parts to a test method. The first part is setting up, and might include, for instance, logging a user on. The second, and only required, part calls the method and checks changes to the database. The third part checks the HTTP result.

def test_should_get_new
 login
 get :new
 assert_response :success
end

Method Calls
By default, controller method calls have all the parameters you would expect (eg :controller), but often you want to add more. In this example, an extra parameter is given. This is equivalent to ?run=199 at the end of the URL.

get :new, :run => '199'

In his example, parameters for a model are given. This is equivalent to data in a form, as usually accessed via params. I found that this was essential for create, otherwise there is no data in your model, and so nothing gets saved and the test fails.

post :create, :sample => { :sample_reference => 'P12345', :sample_type => 'PP1'}

Checking the results
While your view can, of course, access any instance variable in the controller, you do not have direct access to those variables from within your tests. Instead, they can be accessed though the assigns method. This allows you to test against values.

assert_not_nil assigns(:samples)
assert_equal 1, assigns(:samples).length
assert_equal 'glc', assigns(:sample).analysis
 # Test value of a specific attribute for one record

To check the returned HTTP code, just use the standard method:

assert_response :redirect

Other options are :success, :missing and :error. Not sure why you would test for the last two.

You can also test the template used. This is for those occasions when you specified a template (like this; render :template => 'folder/template'), otherwise the template is nil.

assert_template 'samples/new'

The last check is where the page has been redirected to. You can use the same parameters for assert_redirected_to that you can for redirect_to.

assert_redirected_to :controller => 'samples', :action => 'home'
assert_redirected_to samples_path
assert_redirected_to sample_path(assigns(:sample))

See also:
http://api.rubyonrails.com/classes/ActionController/Assertions/ResponseAssertions.html

You can also check what has happened to your database table with the assert_difference method. Wrap this method around your method call, and it will check that the number of records has changed by the appropriate amount. This example checks the number of records for a model called Post has increased by one (the default value) when a new record is created:

def test_should_create_post
 assert_difference('Post.count') do
   post :create, :post => { }
 end
 assert_redirected_to post_path(assigns(:post))
end

The destroy method needs to check that the number of records has decreased by one, so in that case the method call looks like this:

    assert_difference('Post.count', -1) do

What assert_difference does is to determine the value of the first parameter (Post.count in the example), then run the block, and then determine the value of the first parameter again. It then compares the value before and after, and checks that difference against the second parameter (which defaults to +1).

Fixtures
Fixtures are a way to easily generate example data. You can point your tests to a fixtures file with:

fixture :my_data

By default, rails includes the statement fixture :all in tester_helper.rb, so all the fixtures get loaded for all your tests.

Rails will look for test/fixtures/my_data.yml (or test/fixtures/my_data.csv), and use the data in there to create a data structure, my_data (actually, I suspect my_data is a method that returns the data_structure, but the effect is the same), as well as in your database table. Here is an example of a fixtures file, employees.yml.

:one
name: Fred Smith

:two
name: Mary Jones

You can access specific records from your fixtures using their label. For the employees fixture, to access Fred Smith, use:

employees(:one)

Functional testing gets a little more complicated once you have a model that belongs_to another (see here for how to set up the model). Let us suppose the companies.xml fixture file look like this:

:one
name: The Excellent Software Company

:two
name: MegaSoft Ltd

In the Employees fixture file, employees.yml, you can set up the association for the company very easily. Just add a key named after your associated model, with the value equal to the label you used for your entry (without the colon):

:one
name: Fred Smith
company: one

:two
name: Mary Jones
company: one

To assign a company to an employee in a test, use something like this:

post :create, :employee => {:name => 'Tom Johnson', :company => companies(:one) }

Preventing something when testing
It may be desirable to prevent some actions happening while testing, for example, confirmation e-mails being sent whenever a new user account is set up. This is easily done by checking ENV['RAILS_ENV'].

def my_method
return if ENV['RAILS_ENV'] == 'test'   # Do not during testing
do_stuff_only_if_not_in_test_environment
end

Obviously this could cause potential problems, as you are specifically not testing some of your code, so use with care!

Testing functions that require a logged in user
This is what works for me...

At the start of each relevant method, put in a line like this:

login_as_admin(@request)

In test_helper.rb, define the login method, something like this:

def login_as_admin(request)
 user = User.new
 user.login = "tester"
 user.email = "tester@domain.com"
 user.username = "Test Administrator"
 #...
 # Set up other details, permissions, etc.
 #...
 request.session[:user_id] = user.id
end

You should also test that a user will get redirected if not logged in (and that a user logged in but with permissions gets redirected too). Here is how:

def test_should_be_refused_without_login
 get :new   # Example method to test
 assert_response :redirect
 assert_redirected_to :controller => 'session', :action => 'new'
end

Similar methods can be set up for other user roles, to test how the system behaves.

Testing helper methods
You can test your helpers (in application_help.rb) in your functional tests.

class HelperTest < Test::Unit::TestCase
  include ActionView::Helpers::TextHelper
  include ActionView::Helpers::TagHelper
  include ApplicationHelper
  # include whatever helpers you want to test here, sometimes you'll need
  # to include some of the Rails helpers, as I've done above.

  def test_some_helper
  end
end

However, you do not have access to the usual Rails helpers, and so any of your methods that rely on them are going to generate errors. There may well be a way around that that I have yet to discover, though as some Rails helpers use the context of the web user to determine their behavior (eg, the output of link_to depends on the current web page) there may not.

Credit: http://blog.lathi.net/articles/2006/03/31/testing-rails-helpers

Testing filter methods
You can test filter methods via the usual fuctional tests, for example having one test in which a user is signed in, and another in which the user is not. However, I feel it is better to also test the method itself. I would describe that as unit testing, but as your filter may well do a redirect, you will have to test it in your functional tests. This will be a protected, instance method, so can be invoked like this:

MyModelsController.new.send(:check)

However, as far as I can find, there is no way to set the params variable, so you cannot test a filter that, for example checks an id exists (see forum thread here for more).

What to test
My opinion is that you need one functional test for each outcome of each action in your controller. If an action can result in either of two pages being rendered, then that action needs two tests (that said, I rarely bother to test for when a save fails). This rule of thumb does rely on you having pretty much nothing in your controller besides setting up the instance variable, and deciding what page to display; all the complicated stuff is in the models, and that is adequately covered in your unit tests.

Things to watch out for
If you get a error complaining about a database table or column missing check that the table or column really is there via SQL - have you updated your test tables after modifying them in the development environment? Alternatively, if there is not supposed to be a table or column of that name (eg a subclass in STI or a model you later removed), check the test/fixtures directory; Rails may be trying to put data from here into your old table - just delete the .yml file for an erroneous table, or edit the data to remove references to the erroneous column.

If you are getting 300 responses when you expect successes, it might be because you need to set up a session with an appropriate user.

I write tests by copy-and-pasting existing tests. This leaves me liable to having two tests with the same name. Ruby will give no warning about this, but will only perform one of the tests, which can lead to a false sense of security.

You cannot rely on the values of the id field. In the development database they number sequentially from 1, but in the test database they start at some huge number.

I had a method that found the most recent entry of a certain type in the database and used that as a template for a new one. This is difficult to test, because all the entries in your fixtures file get created at the same time, so you have no idea which one will be selected as the most recent. Fortunately, in my case I had a reference number for each record and I could use that instead.

The data in fixture files gets loaded without validation, which is fair enough. However, validation will apply to the method calls in your tests. So if your update test uses update_attributes, and is working with a record that will not validate (perhaps the record has a required field missing, or the same supposedly unique field as another), the result will be that update_attributes will return false, with no clue as to why. It took me about a long time to realise this.

Similarly, if your create method is failing to increase the count in your database table, it may be because the validation fails.

Be aware that functional testing will not check any of your links actually go anywhere, or that your web pages make sense.

I have to say that in setting up functional tests for my project, most of the errors it turned up were down to the testing, rather than real bugs...

Struggling with Ruby: Contents Page

Loops

Ruby has several options for loops; I will start with for and each loops. In practical terms, there is not a lot between for and each besides style, though each is actually a method call that takes a block, while for seems to be a language feature. They both iterate over a collection, which could be a hash, a string, an array or a range. Let us look at ranges first.

For and each with ranges
Use of a range takes the place of the normal for/next loop. In these examples, 1..10 and 1...11 are Range objects; the three dots indicates that the end stops before the terminator, two dots indicates it includes the terminator. These three examples will all print out the numbers 1 to 10, illustrating the difference between each and for, and the two and three dots.

for i in 1..10
 do_stuff
end

for i in 1...11
 do_stuff
end

(1..11).each do i
 do_stuff
end

Ruby is perfectly happy with characters and variables in a Range object, and the next example shows how a range can be assigned to a variable.

r = 'a'..'z'
for i in r
 puts i
end

n = 5
(1..n).each do i
 puts i
end

These loops output the Range object, by the way, and you can test if something is within a range using the member? method (eg r.member? "c").

A quick note about arrays and ranges. I got really confused by putting square brackets around a range. What is [1..5]? This is an array with a single member, and that member is a Range object.

For and each with arrays, hashes and strings
The each method allows you to iterate over each member of a string, an array or a hash.

array.each { member do_stuff_with(member) }
array.each do member
 do_stuff_with(member)
end

For a hash, you pass two values to the block

hash.each { key, value do_stuff_with(key, value) }
hash.each do key, value
 do_stuff_with(key, value)
end

You can use the for feature with hashes and arrays too, for example:

h = {:a => 19, :b => 35, :c => 3456}
for k, v in h
 puts "key=#{k} value=#{v}"
end

The times method
In Ruby everything is an object, and integer objects (of the class Fixnum) have a method called times, which, like each, accepts a block, and iterates from zero up to one less that the number. In these examples, the numbers zero to nine are printed.

10.times { i puts(i) }

10.times do i
 puts i
end

n = 10
n.times { i puts i }

Invoking times on a negative number will create a loop with zero iterations!

The yield feature
You can easily define your own methods that work like each and times, using the keyword yield. Say this method is defined in a class, Car:

def cars
 yield "Red car"
 yield "Green car"
 yield "Blue car"
end

The following will then produce a list of cars; red, green and blue.

m = MyClass.new
m.cars { c puts c }

The cars method runs until it hits the yield, and at that point passes the value back to the calling code, which then processes it as required. Then the loop goes again, and the cars method continues to the next yield stament. The loop iterates until it runs out of yields (the cars method will run to the end regardless of any further yields in the code).

The while and until loops
Ruby also supports while and until. The until loop is just a while not loop. There appears to be no option to check the conditional after the iteration that I have found.

Infinite loops
You can also create infinite loops with the loop method. Hmm, beter make sure there is some way to break out of the loop, which brings us to:

The break, redo, retry, next and return keywords
Ruby also has some interesting keywords to modify loop behaviour. As with C and derivatives, the break statement drops you out of the current loop. The next statement replaces continue; it stops the current iteration, moves to the next one, and passes control to the condition at the end of the loop. The retry statement is similar, but does not move to the next iteration; it repeats the current one. The redo statement starts the whole loop from the first iteration. There is also a return statement, which drops you straight out of the entire method (and optionally takes a parameter; a value returned by the method).

Struggling with Ruby: Contents Page

Conditional statements

Before going any further, try this in the Ruby interactive console:

puts false or true        # prints false
puts true and false       # prints true

Surprisingly, what gets printed is false and true respectively. What is going on? The problem is that and and or do not bind as strongly as the method call. In effect, you are doing this:

(puts false) or true
(puts true) and false

Contrast this with && and . These bind more tightly than the method call, and so behave as you would expect.

puts false  true        # prints true
puts true && false        # prints false

More on operator precedence here:
http://www.techotopia.com/index.php/Ruby_Operator_Precedence

if and unless
While C and its derivatives have only if (condition) statement; Ruby supports two conditionals, if and unless, each in two formats:

if condition
 statement
end

unless !condition
 statement
end

statement if conditional

statement unless !conditional

You can optionally surround your conditional in brackets.

Ruby uses lazy conditionals. In the following code, if s is nil, Ruby knows that the expression will be false, and so does not test s.length > 10 - which is good, because that would throw an exception. This could, however, be a problem if you are changing the state of something in your condition; it might not change as often as you might imagine.

if !s.nil? and s.length > 10
 puts 'Long string'
end

Conditional assignments can be done with the ||= operator.

s ||= "default"

The way this works is that nil counts as false, and in effect it say s = s || "default". If s is nil or false, then s becomes "default", otherwise it keeps its orignal value.

Ruby also supports the usual tertiary operator:

x = conditional ? value1 : value2

Struggling with Ruby: Contents Page