Multiplay Labs

If you mix relative and fully qualified conditions in ActiveRecord, your in for random results with the potential for invalid SQL.

The following is an example of code which may or may not work.

User.find( :all, { :name => 'wibble', 'email.valid' => true }, :join => [ :email ] )

The problem lies in the base method sanitize_sql_hash_for_conditions that generates the fully qualified SQL statements. It takes a “default” table name that’s used for unqualified conditions, however it overwrites this variable if it comes across an fully qualified condition. This means that the generation of the SQL depends on the internal order of the hash resulting in unpredictable output.

In our case code which had been working for months suddenly stopped generating valid SQL without any changes, we even had a dev instance and a live instance running identical code but generating different SQL its that unpredictable. It seems something totally unrelated had effected the internal hash order and hence the validity of the generated SQL.

The fix for this is simply to ensure that fully qualified conditions don’t overwrite the default for relative conditions, the following patch achieves this.

Index: vendor/rails/activerecord/lib/active_record/base.rb
===================================================================
--- vendor/rails/activerecord/lib/active_record/base.rb (revision 1332)
+++ vendor/rails/activerecord/lib/active_record/base.rb (working copy)
@@ -2308,11 +2308,13 @@
 
               # Extract table name from qualified attribute names.
               if attr.include?('.')
-                table_name, attr = attr.split('.', 2)
-                table_name = connection.quote_table_name(table_name)
+                tn, attr = attr.split('.', 2)
+                tn = connection.quote_table_name(tn)
+                attribute_condition("#{tn}.#{connection.quote_column_name(attr)}", value)
+              else
+                attribute_condition("#{table_name}.#{connection.quote_column_name(attr)}", value)
               end
 
-              attribute_condition("#{table_name}.#{connection.quote_column_name(attr)}", value)
             else
               sanitize_sql_hash_for_conditions(value, connection.quote_table_name(attr.to_s))
             end

If for whatever reason the foreign key is invalid for a belongs_to relationship in ActiveRecord then the relationship is silently ignored. This results in object.relationship.method calls failing with nil errors and is quite frustrating to debug as no attempt to load the related object is logged.
Given this if a related object which is listed as a belongs_to in the model yet doesn’t work check that either the manual :foreign_key or the automatically determined version i.e. name + ‘_id’ exists in the dependencies table.

When using ruby’s URI.parse yesterday I was getting URI::InvalidURIError on what seemed to be a perfectly valid URI. After much digging in the source and reading the URI RFC 2396 and subsequent 3986, I discovered that the reason for this is that URI.parse applies this RFC strictly, so where a number of applications are quite happy to provide and use URI’s containing the “unwise” characters, URI.parse doesn’t accept them and throws a URI::InvalidError.

Fear not though there’s a really easy fix, just replace the direct call with:

myuri = URI.parse( URI.encode( uri_string ) )

This does however raise the question: Should parse implicitly call encode?

I’ve been doing some work to www.fileplay.net which required the dowload of files using ruby and during tests I was getting very slow download times coupled with very high cpu load from the ruby process.

The code in use was very simple:

require 'open-uri'
open( [http uri] )

I did some benchmarking and the download of a 157MB file from the local machine was taking over 20 seconds and using 100% where as wget for the same file only took 0.7 seconds.

Digging some more and profiling the code with RubyProf revealed that during the execution of the download over 11,000 threads where being created. This I tracked down to the net/protocol module and the Net::BufferIO::rbuf_fill method which is using a timeout block to wrap the @io.sysread(1024) call. This is clearly an extremely bad way to do this at it creates a new thread for every read call to monitor for timeout and was totally crippling the performance.

After playing with several changes to the core net/protocol.rb including:

• Replacing timeout( @read_timeout ) { .. } with IO.select
• Increasing the read requests to 1Mb
• Garding against the use of a str.split!

I managed to get ruby to perform very similar to wget and download my test file in 0.8 seconds.

For those using ruby to do http requests of any significant size I would hence strongly suggest applying the patch I’ve uploaded to ruby bug tracker here: Ruby very slow http downloads bug

Ever had a problem with query strings causing strange behaviour in apache when using the CGI module?
If you have the patch below might just save you some time and effort hunting down the problem.

--- CGI.pm.orig	Sat Mar  1 16:58:19 2008
+++ CGI.pm	Sat Mar  1 18:39:21 2008
@@ -2779,5 +2779,10 @@
    my $raw_script_name = $ENV{SCRIPT_NAME} || '';
    my $raw_path_info   = $ENV{PATH_INFO}   || '';
-   my $uri             = unescape($self->request_uri) || '';
+   my $uri             = $self->request_uri || '';
+
+   # ensure we dont get any query string as that can include escaped //
+   # e.g. a url parameter, which will break the apache bug fix
+   $uri =~ s/\?(.*)$//; 
+   $uri = unescape($uri);
 
    my $protected    = quotemeta($raw_path_info);

I’ve recently had trouble with a fresh Ruby install on a new FreeBSD box. Passenger wouldn’t start the app, and script/console gave the following warnings:

Loading development environment (Rails 2.2.2)
/usr/local/lib/ruby/gems/1.8/gems/activerecord-2.2.2/lib/active_record/base.rb:394:NoMethodError: undefined method `cattr_accessor' for ActiveRecord::Base:Class
/usr/local/lib/ruby/gems/1.8/gems/activesupport-2.2.2/lib/active_support/testing/setup_and_teardown.rb:14:in `included':NameError: uninitialized constant ActiveSupport::Callbacks
/usr/local/lib/ruby/gems/1.8/gems/rails-2.2.2/lib/console_with_helpers.rb:19:NoMethodError: undefined method `require_dependency' for main:Object


Doing rake -T threw up an iconv error. Simply installing the converters/ruby-iconv port fixed all the problems

Lots of hair tearing over the last few hours due to a totally unexpected gotcha. Take the following code from a shoulda functional test:

context "Logging in as an inactive user" do
	setup do
		@user = Factory(:inactive_user)
		post :create, :login => "user@example.com", :password => "monkee"
	end
 
	should_redirect_to "users_activate_url(#{@user.id})"
end

On first glance, you’d think this would work right? Wrong. The setup isn’t called until after our should macro is evaluated, and so #{@user.id} fails with a nice WhinyNil error.

This would ideally be worked around by a block deferring the execution a-la factory_girl deferred blocks. I’ll have a think about how best to accomplish this and submit it to the Thoughtbot guys. In the meantime, the unfortunate workaround is to use an instance variable from your controller, like so:

should_redirect_to "users_activate_url(@user.id)"

As part of our new Fileplay system, we’re implementing automated uploads of Match Demos at i35 direct to Fileplay. Our tournament admins at the events use ClanForge to manage the tournament, so we wanted to allow them to do it direct from there. Enter XML-RPC. ClanForge already speaks this, so it seemed the best way forward was to teach Rails to speak XML-RPC. Should be easy right?

Oh how wrong we were.

Googling around for a plugin to make this Just Work™ yielded results for Rails pre-2.0 using ActionWebService, which is now unused and “ousted”. Luckily, the solution is simple using XMLRPC4R

As XML-RPC generally only has one endpoint, we simply create a new controller with an index action which will do all the request handling.

Within the action, we start out by instantiating a basic class that knows how to speak XML-RPC.

xmlrpc = XMLRPC::BasicServer.new

The docs for XMLRPC4R say that we should never need to instantiate this. The key word here is “should”, normally it’s subclassed by something that does the handling of the HTTP stuff. Not in this case as Rails is handling that.

Now we want to hand off the request to the XML-RPC library.

response = xmlrpc.process(request.body.read)

If an exception gets raised within our handler, it’ll get automagically converted into an XML-RPC fault structure. But we still want the error to appear in our log:

# Log the error if there is one
parser = XMLRPC::XMLParser::XMLStreamParser.new
ret = parser.parseMethodResponse(response)
logger.error("XMLRPC fault raised. Code: #{ret[1].faultCode}: Message: #{ret[1].faultString}") unless ret[0]

Finally, render the XML-RPC response.

render :text => response, :layout => false

XML-RPC Methods

So how do we define methods that the consumers of our service can… consume?

Easy.

# Handler definitions
xmlrpc.add_handler("file.get") do |fileid|
	file = Multiplay::Fileplay::File.find_by_fileid(fileid)
	raise XMLRPC::FaultException.new 404, "File Not Found" if file.nil?
	file
end

Sure it’ll capture any raised Exceptions, but if you want to pass something meaningful to consumers, raise an XMLRPC::FaultException with a faultCode and a faultString.

Remember, as this is a block, you can’t return from it. The value to return over XML-RPC will be the value yielded, in this case the file.