Multiplay Labs

tech hits and tips from Multiplay

Archive for the ‘FreeBSD’ Category

Rails 2 to Rails 4 Basic Upgrade Tips

without comments

We’ve just had the “pleasure” of upgrading a very basic legacy Ruby on Rails 2 app to Rails 4 and the following where the basic steps needed.

1. Create a new app
2. Import the old code /app and /lib into the new app
3. Update the models
3.1. Update custom table name declarations

set_table_name 'table' -> self.table_name = 'table'

3.2. Update custom primary key declarations

set_primary_key 'primary_key' -> self.primary_key = 'primary_key'

4. Updated configuration removing any config.gem as this is maintained in Gemfile now
5. Migrate old app/controllers/application.rb -> app/controllers/application_controller.rb

In our app we use a custom path to log to for Rails 4 changing this is simply not documented, we found howere the following worked for us:-
Add to config/application.rb

# Use logs directory not log directory for logs
config.paths['log'] = File.join('logs', "#{Rails.env}.log")

Finally one of the plugins we used was validates_as_email which errored on startup due to a UTF8 error this was easily fixed by switching from // syntax to specifically:

--- validates_as_email.rb.orig       2013-07-22 12:53:11.360404644 +0000
+++ validates_as_email.rb      2013-07-19 08:36:52.969805090 +0000
@@ -25,7 +25,7 @@ module RFC822
     domain = "#{sub_domain}(?:\\x2e#{sub_domain})*"
     local_part = "#{word}(?:\\x2e#{word})*"
     addr_spec = "#{local_part}\\x40#{domain}"
-       pattern = /\A#{addr_spec}\z/
+       pattern = "\A#{addr_spec}\z", nil, 'n'

Written by Dilbert

July 22nd, 2013 at 9:47 am

Posted in FreeBSD,Rails

Rails 4 Javascript Dependency Failure on Boot

without comments

Rails includes a package manager which does on demand javascript packaging, this depends by default on node under FreeBSD which installs its binaries into /usr/local/bin which shouldn’t be a problem however the this isn’t in the path for services started at boot.

Because node can’t be found in the PATH passenger will fail to start the app.

We fixed this by adding the following to config/boot.rb

# Ensure /usr/local/bin is in the path
ENV['PATH'] = (ENV['PATH'] || '').split(/:/).push('/usr/local/bin') * ':'

Written by Dilbert

July 22nd, 2013 at 9:33 am

Posted in FreeBSD,Rails

Configuring Supermicro IPMI interface NIC using ipmitool

without comments

Newer Supermicro IPMI interfaces come configured by default in “failover” mode which means that the IPMI will bind to either the dedicated IPMI NIC port or share with one the the machine NIC ports.

This can cause IPMI to come up on wrong NIC and hence be inaccessible if the dedicated NIC doesn’t detect a link.

You can use ipmitool to change this behavour

First query the current setting:
ipmitool raw 0x30 0x70 0x0c 0

The result will be one of the following
0x00 = Dedicated
0x01 = Onboard / Shared
0x02 = Failover

Next to configure it you can use one of the following.

For older models:
ipmitool raw 0x30 0x70 0x0c 1 1 0

For X9 motherboards:
ipmitool raw 0x30 0x70 0x0c 1 0

References for this can be found here:

Written by Dilbert

June 27th, 2013 at 5:03 pm

Posted in FreeBSD,Networking

Fixing bad FreeBSD boot.config on ZFS

without comments

Today we had a machine rebooted with a broken /boot.config file, preventing it from booting.

It took us some time to find the solution to for a full ZFS machine so worth a mention.

From boot prompt simply enter:


Written by Dilbert

May 14th, 2013 at 1:46 pm

Posted in FreeBSD,ZFS

Caching Steam Downloads @ LAN’s

with 32 comments

This article has been superseded by LANcache – Dynamically Caching Game Installs at LAN’s using Nginx

Gone are the days when users install most of their games from physical media such as CD’s / DVD’s instead they rely on high bandwidth internet connections and services such as EA’s Origin and Valve’s Steam.

This causes issues for events which bring like minded gamers together at LAN events such as Multiplay’s Insomnia Gaming Festival, due to the massive amount of bandwidth required to support these services with users patching games and downloading new ones to play with their friends.

With the release of Steam’s new Steampipe creating a local cache of steam download’s, so that game installs at these types of events is significantly quicker and requires dramatically less internet bandwidth, has become much easier.

Steampipe uses standard web requests from Valve Software’s new content servers so standard proxy technology can be used to cache requests.

This article describes how to use FreeBSD + nginx to create a high performance caching proxy for all steam content served by Steampipe

Hardware Specifications
The following is our recommended specification for a machine. Obviously the higher the spec the better, in particular more RAM.

  • Quad Core CPU
  • 32GB RAM (the more the better)
  • 6 x 1TB HD’s
  • 2 x 120GB SSD’s
  • 1Gbps NIC

Machine Install
Our setup starts by performing a FreeBSD ZFS install using mfsBSD we use a RAIDZ on the HD’s e.g.

zfsinstall -d da0 -d da1 -d da2 -d da3 -d da4 -d da5 \
    -r raidz -t /cdrom/8.3-RELEASE-amd64.tar.xz -s 4G

Once the OS is installed setup the L2ARC on the SSD’s

zpool add cache da6 da7

Next install the required packages:

pkg_add -r nginx

Enable FreeBSD http accept filter which improves performance of processing http connections as they are handled in kernel mode:

echo 'accf_http_load="YES"' >> /boot/loader.conf
kldload accf_http

Enable nginx to start on boot by adding nginx_enable="YES" to /etc/rc.conf:

echo 'nginx_enable="YES"' >> /etc/rc.conf

Configuring Nginx
First create some directories:

mkdir -p /data/www/steamproxy
mkdir -p /data/www/logs

The trick to mirroring the steam content so that duplicate requests are served locally is two fold.

  • First spoof *, content[1-8] and send all traffic to the proxy box
  • Configure nginx to on demand mirror the requests under /depot/ using none spoofing resolvers.

Here’s our example /usr/local/etc/nginx.conf. It uses Google’s open DNS servers for the resolver but any non-spoofed resolvers will do.

user www www;
worker_processes  4;
error_log  /data/www/logs/nginx-error.log;
events {
	worker_connections 8192;
	multi_accept on;
	use kqueue;
http {
	include mime.types;
	access_log  /data/www/logs/nginx-access.log;
	keepalive_timeout 65;
	# steam mirror
	server {
		listen 80 accept_filter=httpready default;
		server_name _;
		access_log /data/www/logs/steam-access.log;
		error_log /data/www/logs/steam-error.log;
		root /data/www/steamproxy;
		index index.html;
		location /depot/ {
			try_files $uri @mirror;
			access_log /data/www/logs/depot-local.log;
		location / {
			proxy_next_upstream error timeout http_404;
			proxy_pass http://$host$request_uri;
			proxy_redirect off;
			proxy_set_header Host $host;
			proxy_set_header X-Real-IP $remote_addr;
			proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
			add_header X-Mirror-Upstream-Status $upstream_status;
			add_header X-Mirror-Upstream-Response-Time $upstream_response_time;
			add_header X-Mirror-Status $upstream_cache_status;
		location @mirror {
			proxy_store on;
			proxy_store_access user:rw group:rw all:r;
			proxy_next_upstream error timeout http_404;
			proxy_pass http://$host$request_uri;
			proxy_redirect off;
			proxy_set_header Host $host;
			proxy_set_header X-Real-IP $remote_addr;
			proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
			add_header X-Mirror-Upstream-Status $upstream_status;
			add_header X-Mirror-Upstream-Response-Time $upstream_response_time;
			add_header X-Mirror-Status $upstream_cache_status;
			access_log /data/www/logs/depot-remote.log;

We use on demand mirroring instead of proxy “caching” as the files in steam /depot/ never change so storing them permanently eliminates the overhead of cache cleaners and other calculations.

Don’t forget this can also be used with smaller hardware to reduce steam bandwidth requirements in an office environment too 🙂

Update – 2013/08/19 – Added content[1-8] as additional content server hosts which need proxying
Update – 2013/09/10 – Changed from $uri to $request_uri (which includes query string args) required for authentication now.
Update – 2014/04/30 – Superseded by LANcache

Written by Dilbert

April 17th, 2013 at 1:01 pm

Posted in FreeBSD,Gaming,Nginx

FreeBSD LACP balancing

without comments

When using FreeBSD’s lagg interface one of the protocol options is LACP which is useful when bonding with switches to provide redundancy as well as additional capacity.

When we tried this with 8.3-RELEASE the results weren’t what was expected with only one NIC in the LACP seeing real amounts traffic in either direction.

After digging around in the LAGG code it became apparent that the hashing algorithm being used was the most likely cause of the issue as in our case we had FLOWTABLE removed from our kernel.

Looking in the FreeBSD source tree we found r232629 which adds the ability to configure the hashing method to include layer 2 (mac), layer 3 (IP) and layer 4 (port) details. Applying this to 8.3 and then configuring for l3 hashing with lagghash l3 significantly improved the balancing across the NIC’s for our application, so if your using LACP and aren’t seeing good balancing across your physical interfaces this is well worth a shot.

Written by Dilbert

March 21st, 2013 at 3:59 pm

Posted in FreeBSD

Listing package names only in FreeBSD

without comments

It’s sometimes useful to be able to get a list package names only or packages which match a specific pattern. The little used -E flag to pkg_info achieves this quickly and easily.

pkg_info -E '*'

Written by Dilbert

March 8th, 2013 at 11:06 am

Posted in FreeBSD

Renaming root zfs pool on FreeBSD

without comments

While not often required its sometimes nice to be able to rename the root zfs pool.

Armed with a mfsbsd cdrom this is a relatively painless process under FreeBSD (8.3-RELEASE in out case).

In this example we’re renaming the default root zpool from tank to zroot
1. Boot from mfsbsd cdrom
2. Import the root pool into an alternative location specifying a cachefile, this is in the important bit, as using -R sets cachefile=none which won’t generate the changes required.

zpool import -o altroot=/mnt -o cachefile=/boot/zfs/zpool.cache tank zroot

3. Copy the new cachefile to zpool updating the existing one.

cp /boot/zfs/zpool.cache /mnt/boot/zfs/zpool.cache

4. Update /mnt/boot/loader.conf with the new details e.g. vfs.root.mountfrom="zfs:zroot/root"
5. Reboot

If you have more thank one pool then always best to import them all even if not renaming them to ensure the new zpool.cache contains all your pools

Written by Dilbert

January 14th, 2013 at 5:01 pm

Posted in FreeBSD,ZFS

Making gstat work in FreeBSD jail’s

without comments

gstat requires /dev/devstat so for it to work under a jail you need enable devfs an ensure that /dev/devstat is available.

To enable devfs for all jails set the following in /etc/rc.conf


To add devstat to the default jail ruleset you can add the following to /etc/devfs.rules

add include $devfsrules_hide_all
add include $devfsrules_unhide_basic
add include $devfsrules_unhide_login
add path devstat unhide

Written by Dilbert

December 12th, 2012 at 2:46 pm

Posted in FreeBSD

FreeBSD Serial Over LAN

without comments

In a past post I showed how easy it was to setup IPMI support on FreeBSD this post goes one step further and shows how to configure SOL.

First you need to ensure that kernel has the uart device compiled in which is simply the following in your kernel config
device uart

Next you need to configure your console to support the settings of your IPMI in /boot/loader.conf. In our case that’s a built in Supermicro IPMI which is configured on serial port two with the default settings

# SOL console

Next edit /etc/ttys to enable console over serial port two

ttyu1   "/usr/libexec/getty std.115200" vt100   on secure

Finally to connect to your configured SOL use ipmitool

ipmitool -I lanplus -H <hostname> -U <username> -P <password> sol activate

Hardware Quirks

  • On some older Supermicro hardware where serial ports can be disabled while still keeping SOL active, it was necessary to disable the serial port to ensure the port speed didn’t drop to 19.2 during the kernel phase.
  • On some older Supermicro hardware the IPMI module advertised USB 2.0 speeds but only worked reliably at 1.0 speeds so it was necessary to add the disable high speed mode in loader.conf with hw.usb.ehci.no_hs=”1″
  • For X10SLD-F/HF (Intel 1150-based Microcloud), the same settings work (com3 despite MB has no com2) but also loader_color=”NO” is needed; otherwise, loader resets to black-on-black before invoking the kernel. (Thanks to Dmitry Morozovsk)
  • On some newer Supermicro Hardware we also needed to enable Serial Port Out-Of-Band Management for SOL to work after the boot loader (once the kernel had taken over).
  • If your comm port differs you can use loader.conf to set it e.g. com 3: comconsole_port=”0x3E8″
  • Limiting the console uart via loader.conf can also help e.g. hw.uart.console=1234
  • Some have reported configuring the console board rate via loader.conf e.g. hw.uart.console=”br:9600″

More information on what the settings mean and how to configure serial support for other configs can be found in the FreeBSD Handbook – Serial Console Setup

Hints and tips from other users experiences can be found here: SuperMicro IPMI/SOL and ipmitool troubles

Update: 23-05-2016 – Added comconsole_port and loader_color to the default config set which fixes incorrect choice of console port and black on black text.

Written by Dilbert

December 9th, 2012 at 9:02 pm

Posted in FreeBSD