Multiplay Labs

tech hits and tips from Multiplay

Archive for the ‘Rails’ Category

ruby bundle gems to vendor

without comments

It should be easy but the commands to use bundle to setup gems into a rails vendor directory for production are a little longer than expected so here goes for reference:

bundle install --without development:test --path vendor/bundle -j4 --deployment

And to test everything is working as expected:-

bundle exec rails console production

If things don’t work make sure you’ve previously not run bundle install without a --path, if you have clean up the gems that where installed into the system path and try again.

Written by Dilbert

April 11th, 2014 at 4:14 pm

Posted in Rails

Rails 2 to Rails 4 Basic Upgrade Tips

without comments

We’ve just had the “pleasure” of upgrading a very basic legacy Ruby on Rails 2 app to Rails 4 and the following where the basic steps needed.

1. Create a new app
2. Import the old code /app and /lib into the new app
3. Update the models
3.1. Update custom table name declarations

set_table_name 'table' -> self.table_name = 'table'

3.2. Update custom primary key declarations

set_primary_key 'primary_key' -> self.primary_key = 'primary_key'

4. Updated configuration removing any config.gem as this is maintained in Gemfile now
5. Migrate old app/controllers/application.rb -> app/controllers/application_controller.rb

In our app we use a custom path to log to for Rails 4 changing this is simply not documented, we found howere the following worked for us:-
Add to config/application.rb

# Use logs directory not log directory for logs
config.paths['log'] = File.join('logs', "#{Rails.env}.log")

Finally one of the plugins we used was validates_as_email which errored on startup due to a UTF8 error this was easily fixed by switching from // syntax to Regexp.new specifically:

--- validates_as_email.rb.orig       2013-07-22 12:53:11.360404644 +0000
+++ validates_as_email.rb      2013-07-19 08:36:52.969805090 +0000
@@ -25,7 +25,7 @@ module RFC822
     domain = "#{sub_domain}(?:\\x2e#{sub_domain})*"
     local_part = "#{word}(?:\\x2e#{word})*"
     addr_spec = "#{local_part}\\x40#{domain}"
-       pattern = /\A#{addr_spec}\z/
+       pattern = Regexp.new "\A#{addr_spec}\z", nil, 'n'
   end
 end

Written by Dilbert

July 22nd, 2013 at 9:47 am

Posted in FreeBSD,Rails

Rails 4 Javascript Dependency Failure on Boot

without comments

Rails includes a package manager which does on demand javascript packaging, this depends by default on node under FreeBSD which installs its binaries into /usr/local/bin which shouldn’t be a problem however the this isn’t in the path for services started at boot.

Because node can’t be found in the PATH passenger will fail to start the app.

We fixed this by adding the following to config/boot.rb

# Ensure /usr/local/bin is in the path
ENV['PATH'] = (ENV['PATH'] || '').split(/:/).push('/usr/local/bin') * ':'

Written by Dilbert

July 22nd, 2013 at 9:33 am

Posted in FreeBSD,Rails

Elusive error in rack randomly triggered by uploads

without comments

For a while now I’ve been trying to deal with a bizarre issue with rails where uploads would, apparently at random, cause a server error. The error would report a ‘bad content’ error and often the upload would nearly complete as far as the user was concerned, only to fall at the last hurdle, bailing out at 99%. This happened pretty infrequently and I could never seem to replicate it but I eventually found an article that explained the problem we seemed to be having.

Heisenbug: Ruby on Rails Corrupted/Broken POST data on a big form, it’s all Rack’s fault

In the above article, it’s explained that there is an odd bug in a specific version of rack that only gets triggered in very specific cases where multipart form boundaries happen to align with multipart content parser’s buffer size.

A fix can be found in later versions, as show in this commit: Fixed multipart parameter parsing for when a field’s body ends at the same time as a chunk (i.e. we’ve reached EOL and buffer is empty)

Once you read the code and understand the context, it becomes clear from the commit why the error was occurring. Essentially the parser was thinking it had reached the end of the document if it just so happened that the data so far was perfectly aligned in the buffer instead of also assuring that it wasn’t just at the end of a field, rather than the entire document.

It’s an incredibly annoying and hard to diagnose error, especially since, from a high level view all the way at the frontend of a rails app, it’s extremely unclear as to what is triggering it. It was causing nginx to spout out 422 errors which lead me down the path of suspecting it was the user’s clients prematurely terminating the upload connection, which wasted some time, but no, eventually it turned out to be this peculiar boundary alignment bug. Keeping any eye out to see whether or not any more issues crop up, but hopefully that has solved it.

On a side note, it seems odd that rack is using such a small buffer size of only 16k. For processing large uploads that seems to put an unnecessarily high amount of I/O load on the server considering the amount of times it has to do reads from files potentially upwards of several gigabytes. Once we’ve seen more evidence that this bad content issue has gone away it might be worth adjusting this value to make it more appropriate for processing large multipart data payloads.

Written by Andrew Montgomery-Hurrell

December 20th, 2010 at 5:15 pm

Posted in Rails

Fun with Cookie Domains and Rails

without comments

If you are working on a site in rails that supports multiple subdomains and want to share a cookie between them, it makes sense to set your cookie domain in your environment.rb to something like:

	config.action_controller.session = {
		:key => 'my_app_key',
		:secret => 'my_super_secret',
		:domain => '.domain.tld'
	}

However, you may come across a weird problem when developing locally, where the CSRF protection no longer works, always throwing up an invalid authenticity token error.

The answer to this is simple – by setting the cookie domain you’ve essentially cut your local machine (or any machine not in that domain for that matter) out of the loop, as per the usual cookie security policies. The easy way to be able to continue to do local development is set up a local DNS alias with a matching domain scheme. For me, I did this to my /etc/hosts file:

127.0.0.1 local.domain.tld

And now doing local development via http://local.domain.tld works a charm.

Written by Andrew Montgomery-Hurrell

June 16th, 2010 at 8:55 am

Posted in Code,Hackery,Rails

Invalid SQL Generated by Mixed Relative / Qualified Conditions in ActiveRecord

without comments

If you mix relative and fully qualified conditions in ActiveRecord, your in for random results with the potential for invalid SQL.

The following is an example of code which may or may not work.

User.find( :all, { :name => 'wibble', 'email.valid' => true }, :join => [ :email ] )

The problem lies in the base method sanitize_sql_hash_for_conditions that generates the fully qualified SQL statements. It takes a “default” table name that’s used for unqualified conditions, however it overwrites this variable if it comes across an fully qualified condition. This means that the generation of the SQL depends on the internal order of the hash resulting in unpredictable output.

In our case code which had been working for months suddenly stopped generating valid SQL without any changes, we even had a dev instance and a live instance running identical code but generating different SQL its that unpredictable. It seems something totally unrelated had effected the internal hash order and hence the validity of the generated SQL.

The fix for this is simply to ensure that fully qualified conditions don’t overwrite the default for relative conditions, the following patch achieves this.

Index: vendor/rails/activerecord/lib/active_record/base.rb
===================================================================
--- vendor/rails/activerecord/lib/active_record/base.rb (revision 1332)
+++ vendor/rails/activerecord/lib/active_record/base.rb (working copy)
@@ -2308,11 +2308,13 @@
 
               # Extract table name from qualified attribute names.
               if attr.include?('.')
-                table_name, attr = attr.split('.', 2)
-                table_name = connection.quote_table_name(table_name)
+                tn, attr = attr.split('.', 2)
+                tn = connection.quote_table_name(tn)
+                attribute_condition("#{tn}.#{connection.quote_column_name(attr)}", value)
+              else
+                attribute_condition("#{table_name}.#{connection.quote_column_name(attr)}", value)
               end
 
-              attribute_condition("#{table_name}.#{connection.quote_column_name(attr)}", value)
             else
               sanitize_sql_hash_for_conditions(value, connection.quote_table_name(attr.to_s))
             end

Written by Dilbert

October 9th, 2009 at 10:33 am

Posted in Code,Hackery,Rails

Invalid Foreign Key Silently Breaks Rails ActiveRecord belongs_to

without comments

If for whatever reason the foreign key is invalid for a belongs_to relationship in ActiveRecord then the relationship is silently ignored. This results in object.relationship.method calls failing with nil errors and is quite frustrating to debug as no attempt to load the related object is logged.
Given this if a related object which is listed as a belongs_to in the model yet doesn’t work check that either the manual :foreign_key or the automatically determined version i.e. name + ‘_id’ exists in the dependencies table.

Written by Dilbert

September 21st, 2009 at 12:18 am

Posted in Code,Rails

Rails belongs_to breaks attribute assignments

without comments

So belongs_to and other auto associations in rails are a great time saver, until they break and prevent attribute assignments that is 🙁

So here’s an example of the problem:-

# created_state_id = 1
# published_state_id = 2
post = Post.new( "my post", "stuff..." )
post.state = created
post.save
logger.info "Post saved: #{post.state_id}"
....
logger.info "Before update: #{post.state_id}"
post.state_id = published_state_id
logger.info "After update: #{post.state_id}"
post.save
logger.info "After save: #{post.state_id}"

You would expect that after the second save that post.state_id would be 2 yes? Well its not; the output from the above would be:
> Post saved: 1
> Before update: 1
> After update: 2
> After save: 1

So what’s the deal? The problem is that the assignment to post.state before the first post.save call flags the association, in BelongsToAssociation, as updated but this updated flag is never cleared even after save, so after that point its impossible to change the value of state_id via direct assignment.

Quite a serious issue really as information is lost. The only way to avoid this is to only ever assign using either the association or the attribute i.e. don’t mix and match the two.

I’ve raised this as part of the bigger update issue when changing attributes that are the pk of an association so keep your eye on: Rails issue #142: belongs_to association not updated when assigning to foreign key

Written by Dilbert

July 24th, 2009 at 6:47 pm

Posted in Hackery,Rails

Rails Spawn in production mode = missing log entries

with 2 comments

When running under passenger in production mode ( which uses buffered logging ) when an error occurs in a task / process which is created using Spawn the log messages are lost. This is due to the call to exit! in spawns fork_it method.

The following patch fixes this and a few more minor issues

--- vendor/plugins/spawn/lib/spawn.rb.orig      2009-07-22 20:27:51.000000000 +0100
+++ vendor/plugins/spawn/lib/spawn.rb   2009-07-22 20:37:02.000000000 +0100
@@ -17,7 +17,7 @@
     if !env || env == RAILS_ENV
       @@method = method
     end
-    @@logger.debug "spawn> method = #{@@method}" if defined? RAILS_DEFAULT_LOGGER
+    @@logger.debug "spawn> method = #{@@method}"
   end
 
   # set the resources to disconnect from in the child process (when forking)
@@ -103,7 +103,7 @@
         # run the block of code that takes so long
         yield
 
-      rescue => ex
+      rescue Exception => ex
         @@logger.error "spawn> Exception in child[#{Process.pid}] - #{ex.class}: #{ex.message}"
       ensure
         begin
@@ -116,6 +116,8 @@
           end
         ensure
           @@logger.info "spawn> child[#{Process.pid}] took #{Time.now - start} sec"
+          # Because we use exit! we must try to ensure the log is flushed else output including errors could be lost
+          @@logger.flush if @@logger.respond_to?(:flush)
           # this form of exit doesn't call at_exit handlers
           exit!(0)
         end

Written by Dilbert

July 22nd, 2009 at 8:19 pm

Posted in Hackery,Rails

Debugging Rails: Rails application failed to start properly

without comments

Having spent the entire of today chasing a problem with rails, running under passenger, where it always erroring for specific URL’s with: “Application Error” “Rails application failed to start properly” I thought I’d share one very useful tip which identified the problem me, so hopefully it will save other hours of head banging.

./scripts/console
>> app.get '<Problem URL>'

Now if this is a problem with most parts of your app the error will be plain to see, instead of being caught up in the hand off between passenger and apache resulting in the less than useful Application Error.

N.B. Another good indication that this will help is the existence of “Premature end of script headers” errors in your apache log file.

Written by Dilbert

July 6th, 2009 at 5:19 pm

Posted in Hackery,Rails