Multiplay Labs

tech hits and tips from Multiplay

Archive for the ‘Code’ Category

ruby bundle gems to vendor

without comments

It should be easy but the commands to use bundle to setup gems into a rails vendor directory for production are a little longer than expected so here goes for reference:

bundle install --without development:test --path vendor/bundle -j4 --deployment

And to test everything is working as expected:-

bundle exec rails console production

If things don’t work make sure you’ve previously not run bundle install without a --path, if you have clean up the gems that where installed into the system path and try again.

Written by Dilbert

April 11th, 2014 at 4:14 pm

Posted in Rails

sftp-server umask not working under older versions

without comments

Older versions of openssh’s sftp-server, such as the version shipped in 8.3-RELEASE, includes a bug which means the command line option for umask is not processed correctly.

This can be used to support chroot’ed sftp only as done via the following block in

/etc/ssh/sshd_config
Subsystem   sftp    internal-sftp
Match group chroot
    ChrootDirectory %h
    X11Forwarding no
    AllowTcpForwarding no
    ForceCommand internal-sftp -u 0477

The following patch fixes this issue:

--- crypto/openssh/sftp-server.c.orig	2013-09-27 15:10:32.089496594 +0000
+++ crypto/openssh/sftp-server.c	2013-09-27 15:12:06.128649706 +0000
@@ -1378,7 +1378,7 @@ sftp_server_main(int argc, char **argv, 
 	SyslogFacility log_facility = SYSLOG_FACILITY_AUTH;
 	char *cp, buf[4*4096];
 	const char *errmsg;
-	mode_t mask;
+	long mask;
 
 	extern char *optarg;
 	extern char *__progname;
@@ -1412,11 +1412,11 @@ sftp_server_main(int argc, char **argv, 
 				error("Invalid log facility \"%s\"", optarg);
 			break;
 		case 'u':
-			mask = (mode_t)strtonum(optarg, 0, 0777, &errmsg);
-			if (errmsg != NULL)
-				fatal("Invalid umask \"%s\": %s",
-				    optarg, errmsg);
-			(void)umask(mask);
+			mask = strtol(optarg, &cp, 8);
+			if (mask < 0 || mask > 0777 || *cp != '\0' ||
+			    cp == optarg || (mask == 0 && errno != 0))
+				fatal("Invalid umask \"%s\"", optarg);
+			(void)umask((mode_t)mask);
 			break;
 		case 'h':
 		default:

Written by Dilbert

September 27th, 2013 at 3:41 pm

Posted in Code,FreeBSD

Rails 2 to Rails 4 Basic Upgrade Tips

without comments

We’ve just had the “pleasure” of upgrading a very basic legacy Ruby on Rails 2 app to Rails 4 and the following where the basic steps needed.

1. Create a new app
2. Import the old code /app and /lib into the new app
3. Update the models
3.1. Update custom table name declarations

set_table_name 'table' -> self.table_name = 'table'

3.2. Update custom primary key declarations

set_primary_key 'primary_key' -> self.primary_key = 'primary_key'

4. Updated configuration removing any config.gem as this is maintained in Gemfile now
5. Migrate old app/controllers/application.rb -> app/controllers/application_controller.rb

In our app we use a custom path to log to for Rails 4 changing this is simply not documented, we found howere the following worked for us:-
Add to config/application.rb

# Use logs directory not log directory for logs
config.paths['log'] = File.join('logs', "#{Rails.env}.log")

Finally one of the plugins we used was validates_as_email which errored on startup due to a UTF8 error this was easily fixed by switching from // syntax to Regexp.new specifically:

--- validates_as_email.rb.orig       2013-07-22 12:53:11.360404644 +0000
+++ validates_as_email.rb      2013-07-19 08:36:52.969805090 +0000
@@ -25,7 +25,7 @@ module RFC822
     domain = "#{sub_domain}(?:\\x2e#{sub_domain})*"
     local_part = "#{word}(?:\\x2e#{word})*"
     addr_spec = "#{local_part}\\x40#{domain}"
-       pattern = /\A#{addr_spec}\z/
+       pattern = Regexp.new "\A#{addr_spec}\z", nil, 'n'
   end
 end

Written by Dilbert

July 22nd, 2013 at 9:47 am

Posted in FreeBSD,Rails

Rails 4 Javascript Dependency Failure on Boot

without comments

Rails includes a package manager which does on demand javascript packaging, this depends by default on node under FreeBSD which installs its binaries into /usr/local/bin which shouldn’t be a problem however the this isn’t in the path for services started at boot.

Because node can’t be found in the PATH passenger will fail to start the app.

We fixed this by adding the following to config/boot.rb

# Ensure /usr/local/bin is in the path
ENV['PATH'] = (ENV['PATH'] || '').split(/:/).push('/usr/local/bin') * ':'

Written by Dilbert

July 22nd, 2013 at 9:33 am

Posted in FreeBSD,Rails

Integrating Legacy PHP applications with WordPress

with 2 comments

Recently we wanted to integrate some existing legacy code into a new wordpress site we were developing. Obviously, we didn’t want to have to reimplement all the code for the wordpress theme separately in a standalone way just to make the two codebases visually consistent, so we looked into how to integrate the two, allowing us to make use of the wordpress theme within the new code.

This turned out to be surprisingly simple, albeit with a few caveats.

First of all, we made sure the legacy codebase had access to the wordpress installation. Next, in our main include in the legacy code (a setup/config script known to always be included), we added the following code:

require( "{$_SERVER['DOCUMENT_ROOT']}/wordpress/wp-load.php" );

This causes all pages in the legacy app to load the wordpress framework. Contrary to instructions on the WordPress Codex (which direct you to require wp-blog-header.php) this serves the page correctly, rather than as a wordpress 404 page. Obviously, you should adjust the require path to match your own environment.

Once this was done, in the legacy applications header and footer includes, we could use wordpress functions such as get_header() and get_footer() to include the parts of the theme we needed.

However, there are some caveats to this approach when integrating with legacy codebases. We found that the database connection of wordpress was overriding the database connection established by our legacy application. Since they had different permissions, the legacy application failed to perform any of it’s queries. A quick hack of resetting the connection back to the app after rendering the header, then restoring the wordpress connection prior to rendering the future fixes this. Obviously the correct solution is to update the legacy app to use a specific database connection object rather than the default of the last connection established, but this isn’t always feasible for some projects due to the time/resources involved.

Written by Andrew Montgomery-Hurrell

July 11th, 2013 at 8:19 am

Posted in PHP,Wordpress

Tagged with ,

GeoIP C interface isn’t all thread safe

without comments

It’s documented that GeoIP C interface after v1.3.6 is thread safe, with the exception of GEOIP_CHECK_CACHE, however this is not really the case.

A large number of the C API functions such as: GeoIP_new, GeoIP_open_type, GeoIP_db_avail call the internal _GeoIP_setup_dbfilename function which statically initialises the GeoIPDBFileName global pointer without any locking.

This means that if several threads call any of these methods to gain a geoip handle then its possible to trash the memory structure while its being read and hence result in a crash.

So if your using the GeoIP C interface in a threaded application ensure that you use locking around the use of these methods.

Written by Dilbert

July 4th, 2013 at 2:11 pm

Posted in Code

SSL errors using PayPal Sandbox in the PHP-SDK

without comments

Recently we’ve been working on some payment related code and found that communication with PayPal’s Sandbox was constantly failing due to SSL related errors, such as these:

curl_exec error 35 error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure
curl_exec error 60 SSL certificate problem, verify that the CA cert is OK. Details: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed

As a result, we couldn’t test any code properly which led to a lot of frustration. As it turns out, we weren’t alone in this.

After trying to figure things out, testing our certificates and verifying everything was setup correctly we eventually realised that the PayPal PHP SDK actually has a mistake in it, and specifies the incorrect endpoints for their sandbox when using the API Signature communications method.

On page 25 of their API reference document (PDF) actually specifies the the API Signature and the API Certificate methods on the sandbox should be the same:

But this is not the case, an assertion I make supported by a different set of documentation on the PayPal site.

Fixing the Issue

The entries in the SDK you download are actually wrong so we edited our PayPal SDK files (PayPal/wsdl/paypal-endpoints.php) to point as the correct endpoints specified in the latter document and everything worked fine.

Written by Andrew Montgomery-Hurrell

October 29th, 2012 at 10:43 am

Posted in PHP

Tagged with , , ,

LWP::Protocol::https::Socket: socket: Address family not supported by protocol

without comments

When trying to build LWP::Protocol::https to add https support to libwww the tests fail with:-

LWP::Protocol::https::Socket: socket: Address family not supported by protocol at /usr/lib/perl5/site_perl/5.10/LWP/Protocol/http.pm line 51

After some investigation it turns out there’s nothing wrong with LWP::Protocol::https but IO::Socket::INET6 was broken and trying to interpret : as an IPv6 address instead of IPv4 address + port. Updating to IO::Socket::INET6 v2.69 from the previous v2.57 fixed the issue.

This was encountered under cygwin on Windows in case this is a key factor.

Written by Dilbert

May 24th, 2012 at 7:56 pm

Posted in Code,Cygwin,Perl,Windows

Installing new modules for use with perlapp

without comments

Another really simple little thing when you find it but the ActiveState doc’s really hide how to install new perl modules that are required for compilation with their PDK’s perlapp when targeting a foreign platform.

perlapp --target <target> --target-install <module>

e.g.

perlapp --target windows --target-install JSON::XS

Written by Dilbert

March 21st, 2012 at 9:19 pm

Posted in Perl,Windows

Using the jQuery deferred object pipe method to validate success data

with one comment

Often I’ll make an AJAX call with jQuery to retrieve some data and while that data might be not be erroneous, it might not be exactly what I want. A typical solution to this is to validate the data in the success callback and call out to an external error handler if it fails. However, jQuery provides the capability of setting a failure callback and it would be nice to use that without needing to have to resort to some horrible code like:

var errorHandler = function() {
	alert("ERROR!");
};
 
$.ajax({ 
	url: '/my_ajax_call', 
})
.done(function(data) {
	if(data.success !== 'true') {
		errorHandler();
	} else {
		// do the good stuff
	}
})
.fail(errorHandler);

The problem with the above is that you lose all the benefits of jQuery’s deferred objects[1] and as such if you wanted to attach additional error handlers, you’d have to update the done callback to call them each time which just isn’t feasible when you might be passing around your AJAX promise object all over the place.

However, there is a better way which allows you to leverage the full power of the deferred/promise object system by using the pipe[2] method.

$.ajax({ 
	url: '/my_ajax_call', 
})
.pipe(function(data, textStatus, jqXHR) { 
	if(data.success !== 'true') {
		var deferred = $.Deferred(); 
		deferred.rejectWith( this, [ jqXHR, textStatus, 'invalid' ] ); 
		return deferred; 
	}
	return jqXHR; // return the original deferred object
})
.done(function() { 
	alert('Hurray, a success!');
})
.fail(function() { 
	alert('Boo, a failure!');
})

In the above code, pipe accepts as it’s first argument a function that filters the done callback parameters before the callbacks functions are actually called. Returning null from pipe allows the unfiltered values to pass through, but returning a deferred object will apply the callbacks against that new one, rather than the original. As a result, you can return a deferred object in a failure state to force the failure callbacks to trigger. As you’ll note, using the rejectWith method of the deferred object allows by to fully set what the callbacks evaluation context will be and what arguments it will get. For compatibility, I follow the exact same setup as the defaults, but you’ll see I have set a custom error string.

Because pipe is chainable, you can add multiple of these further down the line. Pipe also accepts additional parameters for filtering other callbacks too. This way it’s much easier and cleaner to validate data and handle errors with the additional flexibility of the deferred object system.

Reference:
[1]: http://api.jquery.com/category/deferred-object/
[2]: http://api.jquery.com/deferred.pipe/

Written by Andrew Montgomery-Hurrell

March 9th, 2012 at 11:18 am

Posted in Hackery,Javascript