Multiplay Labs

tech hits and tips from Multiplay

Archive for the ‘Hackery’ Category

Fun with Cookie Domains and Rails

without comments

If you are working on a site in rails that supports multiple subdomains and want to share a cookie between them, it makes sense to set your cookie domain in your environment.rb to something like:

	config.action_controller.session = {
		:key => 'my_app_key',
		:secret => 'my_super_secret',
		:domain => '.domain.tld'
	}

However, you may come across a weird problem when developing locally, where the CSRF protection no longer works, always throwing up an invalid authenticity token error.

The answer to this is simple – by setting the cookie domain you’ve essentially cut your local machine (or any machine not in that domain for that matter) out of the loop, as per the usual cookie security policies. The easy way to be able to continue to do local development is set up a local DNS alias with a matching domain scheme. For me, I did this to my /etc/hosts file:

127.0.0.1 local.domain.tld

And now doing local development via http://local.domain.tld works a charm.

Written by Andrew Montgomery-Hurrell

June 16th, 2010 at 8:55 am

Posted in Code,Hackery,Rails

Adding Google Chrome Frame support to nginx hosted sites

without comments

I’m trying to add support for google chrome frame to our nginx config the simple way would be to include a fragment in each server block which looks something like:

    if ( $http_user_agent ~ chromeframe ) {
        add_header X-UA-Compatible "chrome=1";
    }

The problem is for some reason add_header isn’t allowed in server-if blocks on location-if blocks, so this errors with: add_header directive is not allowed here
I can’t see or think of a reason why this shouldn’t be allowed, so I tried adding it and it seems to work without a problem.

The following patch, made against 0.8.40, adds server if block support for both expires and add_header.

--- src/http/modules/ngx_http_headers_filter_module.c.orig  2010-06-14 23:48:58.000000000 +0100
+++ src/http/modules/ngx_http_headers_filter_module.c   2010-06-14 23:49:27.000000000 +0100
@@ -79,7 +79,7 @@
 
     { ngx_string("expires"),
       NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_HTTP_LIF_CONF
-                        |NGX_CONF_TAKE12,
+                        |NGX_HTTP_SIF_CONF|NGX_CONF_TAKE12,
       ngx_http_headers_expires,
       NGX_HTTP_LOC_CONF_OFFSET,
       0,
@@ -87,7 +87,7 @@
 
     { ngx_string("add_header"),
       NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_HTTP_LIF_CONF
-                        |NGX_CONF_TAKE2,
+                        |NGX_HTTP_SIF_CONF|NGX_CONF_TAKE2,
       ngx_http_headers_add,
       NGX_HTTP_LOC_CONF_OFFSET,
       0,

Cross posted to the nginx mailing list here

Written by Dilbert

June 14th, 2010 at 11:06 pm

Posted in Hackery,Nginx

Invalid SQL Generated by Mixed Relative / Qualified Conditions in ActiveRecord

without comments

If you mix relative and fully qualified conditions in ActiveRecord, your in for random results with the potential for invalid SQL.

The following is an example of code which may or may not work.

User.find( :all, { :name => 'wibble', 'email.valid' => true }, :join => [ :email ] )

The problem lies in the base method sanitize_sql_hash_for_conditions that generates the fully qualified SQL statements. It takes a “default” table name that’s used for unqualified conditions, however it overwrites this variable if it comes across an fully qualified condition. This means that the generation of the SQL depends on the internal order of the hash resulting in unpredictable output.

In our case code which had been working for months suddenly stopped generating valid SQL without any changes, we even had a dev instance and a live instance running identical code but generating different SQL its that unpredictable. It seems something totally unrelated had effected the internal hash order and hence the validity of the generated SQL.

The fix for this is simply to ensure that fully qualified conditions don’t overwrite the default for relative conditions, the following patch achieves this.

Index: vendor/rails/activerecord/lib/active_record/base.rb
===================================================================
--- vendor/rails/activerecord/lib/active_record/base.rb (revision 1332)
+++ vendor/rails/activerecord/lib/active_record/base.rb (working copy)
@@ -2308,11 +2308,13 @@
 
               # Extract table name from qualified attribute names.
               if attr.include?('.')
-                table_name, attr = attr.split('.', 2)
-                table_name = connection.quote_table_name(table_name)
+                tn, attr = attr.split('.', 2)
+                tn = connection.quote_table_name(tn)
+                attribute_condition("#{tn}.#{connection.quote_column_name(attr)}", value)
+              else
+                attribute_condition("#{table_name}.#{connection.quote_column_name(attr)}", value)
               end
 
-              attribute_condition("#{table_name}.#{connection.quote_column_name(attr)}", value)
             else
               sanitize_sql_hash_for_conditions(value, connection.quote_table_name(attr.to_s))
             end

Written by Dilbert

October 9th, 2009 at 10:33 am

Posted in Code,Hackery,Rails

Logging php errors when using php-fpm

without comments

When moving to using php-fpm to serve pages via nginx from apache, its not very obvious how to see errors logged via the php function error_log, so I thought I’d post how to do it here so others don’t have to do quite a much searching as I did.

Its really simple just ensure both the following php variables are set:

  • error_log
  • log_errors

This can be done either in the php.ini but the method I prefer is setting it up in php-fpm.conf e.g.

<?xml version="1.0" ?>
<configuration>
  <workers>
    <section name="pool">
      <value name="php_defines">
        <value name="error_log">/var/log/php-error.log</value>
        <value name="log_errors">true</value>
      </value>
    </section>
  </workers>
</configuration>

Written by Dilbert

September 13th, 2009 at 8:22 pm

Posted in Hackery,Performance

Rails belongs_to breaks attribute assignments

without comments

So belongs_to and other auto associations in rails are a great time saver, until they break and prevent attribute assignments that is 🙁

So here’s an example of the problem:-

# created_state_id = 1
# published_state_id = 2
post = Post.new( "my post", "stuff..." )
post.state = created
post.save
logger.info "Post saved: #{post.state_id}"
....
logger.info "Before update: #{post.state_id}"
post.state_id = published_state_id
logger.info "After update: #{post.state_id}"
post.save
logger.info "After save: #{post.state_id}"

You would expect that after the second save that post.state_id would be 2 yes? Well its not; the output from the above would be:
> Post saved: 1
> Before update: 1
> After update: 2
> After save: 1

So what’s the deal? The problem is that the assignment to post.state before the first post.save call flags the association, in BelongsToAssociation, as updated but this updated flag is never cleared even after save, so after that point its impossible to change the value of state_id via direct assignment.

Quite a serious issue really as information is lost. The only way to avoid this is to only ever assign using either the association or the attribute i.e. don’t mix and match the two.

I’ve raised this as part of the bigger update issue when changing attributes that are the pk of an association so keep your eye on: Rails issue #142: belongs_to association not updated when assigning to foreign key

Written by Dilbert

July 24th, 2009 at 6:47 pm

Posted in Hackery,Rails

Rails Spawn in production mode = missing log entries

with 2 comments

When running under passenger in production mode ( which uses buffered logging ) when an error occurs in a task / process which is created using Spawn the log messages are lost. This is due to the call to exit! in spawns fork_it method.

The following patch fixes this and a few more minor issues

--- vendor/plugins/spawn/lib/spawn.rb.orig      2009-07-22 20:27:51.000000000 +0100
+++ vendor/plugins/spawn/lib/spawn.rb   2009-07-22 20:37:02.000000000 +0100
@@ -17,7 +17,7 @@
     if !env || env == RAILS_ENV
       @@method = method
     end
-    @@logger.debug "spawn> method = #{@@method}" if defined? RAILS_DEFAULT_LOGGER
+    @@logger.debug "spawn> method = #{@@method}"
   end
 
   # set the resources to disconnect from in the child process (when forking)
@@ -103,7 +103,7 @@
         # run the block of code that takes so long
         yield
 
-      rescue => ex
+      rescue Exception => ex
         @@logger.error "spawn> Exception in child[#{Process.pid}] - #{ex.class}: #{ex.message}"
       ensure
         begin
@@ -116,6 +116,8 @@
           end
         ensure
           @@logger.info "spawn> child[#{Process.pid}] took #{Time.now - start} sec"
+          # Because we use exit! we must try to ensure the log is flushed else output including errors could be lost
+          @@logger.flush if @@logger.respond_to?(:flush)
           # this form of exit doesn't call at_exit handlers
           exit!(0)
         end

Written by Dilbert

July 22nd, 2009 at 8:19 pm

Posted in Hackery,Rails

Debugging Rails: Rails application failed to start properly

without comments

Having spent the entire of today chasing a problem with rails, running under passenger, where it always erroring for specific URL’s with: “Application Error” “Rails application failed to start properly” I thought I’d share one very useful tip which identified the problem me, so hopefully it will save other hours of head banging.

./scripts/console
>> app.get '<Problem URL>'

Now if this is a problem with most parts of your app the error will be plain to see, instead of being caught up in the hand off between passenger and apache resulting in the less than useful Application Error.

N.B. Another good indication that this will help is the existence of “Premature end of script headers” errors in your apache log file.

Written by Dilbert

July 6th, 2009 at 5:19 pm

Posted in Hackery,Rails

perl Makefile.PL errors with “Unable to find a perl 5” on cygwin

without comments

When trying to build a new module on one of our machines which runs cygwin the basic command perl Makefile.pl failed with the error:

Unable to find a perl 5 (by these names: perl.exe perl.exe perl5.exe perl5.8.0.exe miniperl.exe, in these dirs: /usr/local/bin /usr/bin /bin /usr/X11R6/bin /usr/local/libexec/apache /usr/local/bin /usr/bin /bin /usr/X11R6/bin /cygdrive/c/WINDOWS/system32 /cygdrive/c/WINDOWS /cygdrive/c/WINDOWS/System32/Wbem /cygdrive/c/Program /bin /usr/bin)

After doing some digging this turned out to be a simple permissions issue on the perl.exe although perl scripts would happily run in perl -x /usr/bin/perl.exe was returning false.

The fix was a simple chmod a+x /usr/bin/perl.exe for anyone who comes across this issue.

Written by Dilbert

July 4th, 2009 at 1:29 pm

Posted in Hackery,Perl

ruby very slow http downloads and high cpu usage

without comments

I’ve been doing some work to www.fileplay.net which required the dowload of files using ruby and during tests I was getting very slow download times coupled with very high cpu load from the ruby process.

The code in use was very simple:

require 'open-uri'
open( [http uri] )

I did some benchmarking and the download of a 157MB file from the local machine was taking over 20 seconds and using 100% where as wget for the same file only took 0.7 seconds.

Digging some more and profiling the code with RubyProf revealed that during the execution of the download over 11,000 threads where being created. This I tracked down to the net/protocol module and the Net::BufferIO::rbuf_fill method which is using a timeout block to wrap the @io.sysread(1024) call. This is clearly an extremely bad way to do this at it creates a new thread for every read call to monitor for timeout and was totally crippling the performance.

After playing with several changes to the core net/protocol.rb including:

  • Replacing timeout( @read_timeout ) { .. } with IO.select
  • Increasing the read requests to 1Mb
  • Garding against the use of a str.split!

I managed to get ruby to perform very similar to wget and download my test file in 0.8 seconds.

For those using ruby to do http requests of any significant size I would hence strongly suggest applying the patch I’ve uploaded to ruby bug tracker here: Ruby very slow http downloads bug

Written by Dilbert

June 27th, 2009 at 4:24 pm

Posted in Code,Hackery

Safari Exports Element Ids to Javascript Namespace

without comments

Well seems Safari is totally retarded and exports all html elements to the javascript namespace so if you have the something like the following it breaks:-

<div id="myDialog">my content</div>
<script type="text/javascript>
  var myDialog;
  if ( ! myDialog )
  {
     // Safari never gets here as myDialog = the HTMLElement defined above
     myDialog = new MyDialog( .... );
     ...
  }
  // Safari now fails as the object was never created
  // Instead myDialog is the div element
  myDialog.myFunction();
</script>

This was experienced here in Safari 4 build 528.16 and reported by users in Safari 3 as well.

Written by Dilbert

May 28th, 2009 at 5:54 pm

Posted in Hackery