How to simulate an OpenStack Infra Slave

Situation: You’ve committed your code, you’ve submitted a patch, and yet for some reason, and regardless of the number of rechecks, your tests simply won’t pass the gate? How can you test the gate, locally, to triage what’s happening? By creating a local slave VM.

Prerequisites

To complete this tutorial, you will need the following:

  • Vagrant
  • VirtualBox
  • A local clone of OpenStack’s system-config repository: git clone git://git.openstack.org/openstack-infra/system-config

Create a local.pp manifest.

A quick look at the .gitignore file at the root of the system-config project reveals that both ./manifests/local.pp and Vagrantfile are ignored. With that in mind, let us start by creating a simple local puppet manifest which describes our node:

# path: ./manifests/local.pp
# Any node with hostname "slave-.*" will match.
node /slave-.*/ {
  class { 'openstack_project::single_use_slave':
    sudo => true,
    thin => false,
  }
}

The openstack_project::single_use_slave manifest is used by nodepool – or rather, by disk-image-builder on behalf of nodepool- to build the virtual machine image used in OpenStack’s gate. This happens once a day, so any changes made in system_config will require at least 24 hours to propagate to the build queue.

Create a Vagrantfile

Next, we create a Vagrantfile that invokes the above manifest. Note that I am explicitly setting hostname on each node – this allows us to choose specifically which manifest will be applied to our guest.

# path: ./Vagrantfile
VAGRANTFILE_API_VERSION = "2"

Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|

  # Create a new trusty slave: `vagrant up slave-trusty`
  config.vm.define "slave-trusty" do |trusty|
    trusty.vm.box = "ubuntu/trusty64"
    trusty.vm.network 'private_network', ip: '192.168.99.10'
    trusty.vm.hostname = 'slave-trusty' # Use this to control local.pp
  end

  # Create a new xenial slave: `vagrant up slave-xenial`
  # Will only work in vagrant > 1.8.1
  config.vm.define "slave-xenial" do |xenial|
    xenial.vm.box = "ubuntu/xenial64"
    xenial.vm.network 'private_network', ip: '192.168.99.11'
    xenial.vm.hostname = 'slave-xenial' # Use this to control local.pp
  end

  # Increase the memory for the VM. If you need to run devstack, this needs
  # to be at least 8192
  config.vm.provider "virtualbox" do |v|
    v.memory = 2048
  end

  # Install infra's supported version of puppet.
  config.vm.provision "shell",
      inline: "if [ ! -f '/etc/apt/preferences.d/00-puppet.pref' ]; then /vagrant/install_puppet.sh; fi"

  # Install all puppet modules required by openstack_project
  config.vm.provision "shell",
      inline: "if [ ! -d '/etc/puppet/modules/stdlib' ]; then /vagrant/install_modules.sh; fi"

  # Symlink the module in system_config into /etc/puppet/modules
  config.vm.provision "shell",
      inline: "if [ ! -d '/etc/puppet/modules/openstack_project' ]; then ln -s /vagrant/modules/openstack_project /etc/puppet/modules/openstack_project; fi"

  config.vm.provision :puppet do |puppet|
    puppet.manifest_file  = "local.pp"
  end
end

IMPORTANT NOTE: As of Vagrant 1.8.3, the above declared slave-xenial will fail to boot properly. This is because at this time, the published ubuntu/xenial64 image does not contain the guest additions, which must be installed manually. For specifics on how to do this, please examine this launchpad issue.

Vagrant up!

Last step: Execute vagrant up slave-trusty. With luck, and a little patience, this will create a brand new, clean, running jenkins-slave for you to test your build in.

Where next?

From this point, you should take a look at the project-config repository and determine which additional VM configuration steps are being executed by your job, so you can create an environment specific to the problem you’re trying to triage. Alternatively, you can explore some of the other nodes in ./manifests/site.pp, and perhaps extend the Vagrantfile above to instantiate a VM for one of infra’s services, such as StoryBoard or Grafana. Using the above template, you should be able to construct test instances of any infra component.

Update (June 27th, 2016)

The above method may also be used to simulate a regular OpenStack Infra server, with a few modifications. For this example, we’ll try to simulate an OpenStack Mirror. Add the following to your local puppet manifest:

# path: ./manifests/local.pp
node mirror {
  # This module is included on all infra servers. It sets up accounts, public keys, and the like.
  class { 'openstack_project::server':
    iptables_public_tcp_ports => [22, 80],
    sysadmins                 => hiera('sysadmins', [])
  }
  
  # This module includes functionality specific to this server.
  class { 'openstack_project::mirror':
    vhost_name => $::ipaddress,
    require    => Class['Openstack_project::Server'],
  }
}

After doing so, add this node to your Vagrantfile:

# path: ./Vagrantfile
VAGRANTFILE_API_VERSION = "2"

Vagrant.configure(VAGRANTFILE_API_VERSION) do |config|

  # Create a new mirror slave: `vagrant up mirror`
  config.vm.define "mirror" do |mirror|
    trusty.vm.box = "ubuntu/trusty64"
    trusty.vm.network 'private_network', ip: '192.168.99.22'
    trusty.vm.hostname = 'mirror' # Use this to control local.pp
  end

... # Continue from example above.

And done! Now you can invoke vagrant up mirror and watch as your openstack-infra mirror server is provisioned. There are a few caveats:

  1. If you want to add a new puppet module, you’ll want to add it to modules.env. Doing so will only trigger an automatic install if you’re starting from a fresh guest host, so you’ll either also have to install it manually, or recreate your guest.
  2. Some manifests require a hostname. In this case, I usually reference the hosts’ IP Address, as managing DNS is too much effort for most test scenarios. vhost_name => $::ipaddress

Database Upgrades and Migrations with Adobe AIR

This post will show you how to update your SQLite application when you release a new version of your AIR application (Android or otherwise). For those of you who are impatient, just copy the two classes provided and use them as demonstrated at the end of the post (or roll your own, your choice).

When releasing a new version of an AIR application that makes use of SQLite, it’s often necessary to upgrade the schema for that database as well. While taking a Hail Mary approach to the problem and just deleting the old database may work for some, it’s not an elegant option and will ensure that you can never persist data between versions. Instead, creating a way for your application to gracefully migrate its own database to the most recent version will ensure that your local data is not compromised, providing a smooth and seamless experience for the end user.

The Concept

We know that a database is the accumulation of all the CREATE, UPDATE and DELETE statements that have come before it. This means it may be easily represented and reconstructed by an ordered sequence of SQL files. It then becomes our task to manage which SQL files have already been executed in a given application version, and to execute (in order) those files which have not.

To accomplish this, we do the following:

  1. We create a migration table within your database in which we store which scripts have been executed.
  2. We check each script against this table before we run it.

There are other optimizations we can perform, however for this case we’re going to keep the idea simple.

Part 1: The Migration Class

The Migration class is a simple value container that contains your SQL Script as well as a Unique ID. Both of these are derivatives, since we want to give options on how to include the SQL file, and we don’t want to make our users have to think too much about how to set up a migration.

package com.fancybrandname.core.db
{
        import com.adobe.crypto.MD5;
        import com.fancybrandname.core.utils.LogUtil;

        import flash.data.SQLConnection;
        import flash.data.SQLStatement;
        import flash.errors.IllegalOperationError;

        import mx.logging.ILogger;

        /**
         * A specific database migration instance. It should contain both a version
         * and an embedded SQL source file that will be executed against the database.
         *
         * @author Michael Krotscheck
         */
        public class Migration
        {
                /**
                 * Logger
                 */
                private static const LOG : ILogger = LogUtil.getLogger( Migration );

                /**
                 * The SQL Source to use for this migration.
                 */
                private var _source : *;

                /**
                 * The SQL Statement extracted from the SQL Source object.
                 */
                private var _sqlStatement : String;

                /**
                 * This is the unique version key we use to identify this particular migration.
                 */
                private var _versionKey : String;

                /**
                 * The SQL Source to use for this migration.
                 */
                public function get source () : *
                {
                        return _source;
                }

                /**
                 * @private
                 */
                public function set source ( value : * ) : void
                {
                        _source = value;

                        // Here we attempt to extract the SQL statement from the passed object. If 
                        // the object is a Class (as with the @Embed directive), we instantiate and convert it.
                        // If it's anything else we try to invoke the toString method, and throw an error if we
                        // can't find it.
                        if ( _source is Class )
                        {
                                _sqlStatement = ( new _source() ).toString();
                        }
                        else
                        {
                                try
                                {
                                        _sqlStatement = _source.toString();
                                }
                                catch ( e : Error )
                                {
                                        throw new IllegalOperationError( 'Unrecognized type for SQL Migration. Please use Embeds or Strings' );
                                }
                        }

                        // Now we generate a unique ID from this string.
                        // You could request this key manually so that you don't have a dependency
                        // on the adobe crypto library, but I prefer doing it this way.
                        _versionKey = MD5.hash( sqlStatement );
                }

                /**
                 * This is the unique version key we use to identify this particular migration.
                 *
                 * @read-only
                 */
                public function get versionKey () : String
                {
                        return _versionKey;
                }

                /**
                 * This is the unique version key we use to identify this particular migration.
                 *
                 * @read-only
                 */
                public function get sqlStatement () : String
                {
                        return _sqlStatement;
                }

                /**
                 * Constructor.
                 */
                public function Migration ()
                {
                }
        }
}

Step 2: The Migration Manager

Our migration manager does the heavy lifting for this class, by checking the database for the last applied migration key and only applying those keys that come after it. It assumes several things. Firstly, that the migrations are in order of execution. Secondly, that the SqlConnection instance passed to it is a synchronous, not asynchronous connection. This method would definitely work for asynchronous connections, however you’d have to adjust the class to iterate using eventListeners. I leave this as an exercise to the reader.

package com.fancybrandname.core.db
{
	import flash.data.SQLConnection;
	import flash.data.SQLResult;
	import flash.data.SQLStatement;
	import flash.errors.IllegalOperationError;
	import flash.errors.SQLError;
	
	/**
	 * This metadata sets the default property for child nodes used in MXML. This
	 * makes MXML markup a lot easier to understand.
	 */
	[DefaultProperty( "migrations" )]
	/**
	 * A simple class that assists in managing multiple SQL files that may need
	 * to be run against a locally stored database. It makes the assumption
	 * that the migrations must be executed in the order in which they are provided,
	 * and that the database starts "clean" and untouched.
	 *
	 * @author Michael Krotscheck
	 */
	public class MigrationManager
	{
		
		/**
		 * The SQL Connection to use to connect to the database and check for migration.
		 */
		public var sqlConnection : SQLConnection;
		
		/**
		 * The migrations to run against the database. These must be in order of execution.
		 */
		public var migrations : Vector.<Migration> = new Vector.<Migration>();
		
		/**
		 * The name of the table within the database to store the migrations in.
		 */
		public var tableName : String = 'migration';
		
		/**
		 * Constructor
		 */
		public function MigrationManager ( sqlConnection : SQLConnection = null )
		{
			this.sqlConnection = sqlConnection;
		}
		
		/**
		 * This method runs the database migrations.
		 */
		public function migrate () : void
		{
			// Can we migrate?
			if ( !sqlConnection )
			{
				throw new IllegalOperationError( 'No SQLConnection provided' );
				return;
			}
			
			try
			{
				// Make sure the migration table exists.
				createMigrationTable();
				
				// Get the last version key applied to the database.
				var lastVersionKey : String = getLastVersionKey();
				
				// Iterate through our collection and execute every script AFTER the version key we found.
				var len : int = migrations.length;
				
				// Switch to let us know whether the last version key was found.
				// If the last version key is empty, apply all of them.
				var lastKeyFound : Boolean = lastVersionKey == '' ? true : false;
				
				for ( var i : int = 0; i < len; i++ )
				{
					var currentMigration : Migration = migrations[ i ];
					
					if ( lastKeyFound )
					{
						apply( currentMigration );
					}
					else if ( currentMigration.versionKey == lastVersionKey )
					{
						lastKeyFound = true;
					}
				}
			}
			catch ( e : SQLError )
			{
				throw( e );
			}
		}
		
		/**
		 * This method creates the migration table if it doesn't exist yet.
		 */
		private function createMigrationTable () : void
		{
			var sql : String = 'CREATE TABLE IF NOT EXISTS ' + tableName + ' (version varchar(32) NOT NULL PRIMARY KEY, applied_on timestamp NOT NULL);';
			
			var statement : SQLStatement = new SQLStatement();
			statement.sqlConnection = sqlConnection;
			statement.text = sql;
			statement.execute();
		}
		
		/**
		 * This method gets the last version key recorded in the database.
		 */
		private function getLastVersionKey () : String
		{
			var statement : SQLStatement = new SQLStatement();
			statement.text = 'SELECT version FROM ' + tableName + ' ORDER BY applied_on DESC LIMIT 0,1';
			statement.sqlConnection = sqlConnection;
			statement.execute();
			
			var sqlResult : SQLResult = statement.getResult();
			
			if ( sqlResult == null || !sqlResult.data || sqlResult.data.length == 0 )
			{
				// No migrations found.
				return '';
			}
			else
			{
				// We found one, return it.
				return sqlResult.data[ 0 ].version;
			}
		}
		
		/**
		 * This method applies the database migration to the database passed to this function.
		 * Please note that this version of the class only works with Synchronous SQLConnections.
		 */
		private function apply ( migration : Migration ) : void
		{
			// Apply the migration
			var updateStatement : SQLStatement = new SQLStatement();
			updateStatement.text = migration.sqlStatement;
			updateStatement.sqlConnection = sqlConnection;
			updateStatement.execute();
			
			// Update the version key
			var versionStatement : SQLStatement = new SQLStatement();
			versionStatement.text = 'INSERT INTO ' + tableName + ' (version, applied_on) VALUES ( :version , :timestamp )';
			versionStatement.parameters[ ':version' ] = migration.versionKey;
			versionStatement.parameters[ ':timestamp' ] = ( new Date() ).getTime();
			versionStatement.sqlConnection = sqlConnection;
			versionStatement.execute();
		}
	}
}

Step 3: Usage

Finally it’s time to use our class. As you can see the MXML markup is pretty straightforwar, and usage in ActionScript is similarly easy. Note that I’m using both @Embed directives and raw text, and pay particular attention to the embed mime-type. Without it, this doesn’t really work.

<s:Application 
	xmlns:fx="http://ns.adobe.com/mxml/2009"
	xmlns:parsley="http://www.spicefactory.org/parsley"
	xmlns:s="library://ns.adobe.com/flex/spark"
	xmlns:db="com.fancybrandname.core.db.*"
	>
	<fx:Script>
		<![CDATA[
			override protected function initializationComplete():void
			{
				super.initializationComplete();
				
				var dbFile :File = File.applicationStorageDirectory.resolvePath('myDb.db');
				var connection :SQLConnection = new SQLConnection();
				connection.open( dbFile );
				
				migrationManager.sqlConnection = connection;
				migrationManager.migrate();
			}
		]]>
	</fx:Script>

	<fx:Declarations>
		<db:MigrationManager id="migrationManager" tableName="migrations">
			<db:Migration source="@Embed(source='/sql/CreateSomeTable.sql',mimeType='application/octet-stream')"/>
			<db:Migration source="DROP TABLE IF EXISTS someTable;"/>
			<db:Migration source="@Embed(source='/sql/CreateSomeOtherTable.sql',mimeType='application/octet-stream')"/>
		</db:MigrationManager>
	</fx:Declarations>
</s:Application>

Serializing and Deserializing ValueObjects in Flex and AIR

This post originally written for Resource Interactive’s Technology Blog.

One of the holy grails of the "Build Once, Deploy Anywhere" promise of the Adobe AIR isn’t simply that your application will cross all major operating systems, but also that the same codebase can be deployed via the web as an RIA. Admittedly, there remain some differences in functionality; For instance, a desktop application has to worry about windows or contextual menus. Another challenge altogether is transferring data between a desktop and a web application, which can be a daunting task if you’re trying to share multiple files, yet is perhaps not as difficult as you might think.

AIR, via the ByteAray and FileStream classes, allows us to take an arbitrary data object and serialize it to our own custom file type. Since the ByteArray class is available within the standard Flex environment as well, the only trick remaining is to make this file accessible to an application deployed via the web. When running in the browser, Flash’s security sandbox prevents us from directly accessing it on the desktop (At least until the release of Flash Player 10, which relaxes that restriction), however we can still send a file to the server and then open it from there. In short, files generated from AIR can be easily deserialized and used within the Flash Player runtime.

Potential uses for this are many. Consider, for instance, that your application has a fairly complex customized display object that draws its information from a value object. This value object contains strings, arrays, and perhaps even an image. By serializing this information into a file, a user can easily share all this information without the need for a data transfer process that has to support multiple file objects. Simply write your file from binary and send it.

Step 1: Create your Value Object

The first step is of course to create your Value Object. The below code should be no surprise to anyone – the only difference is  the [RemoteClass] metadata tag. This tag is a catchall identifier used by Actionscript to map class-to-binary identification, and is used anywhere  serialization occurs: The AMF protocol, SharedObject, ByteArray Serialization, etc.

package com.resource.vo
{
	/**
	 * A simple data Value Object to show that it is possible
	 * to deserialize a binary object created in AIR within the
	 * Flex environment.
	 */
	[RemoteClass(alias="com.resource.vo.SimpleDataVO")]
	public class SimpleDataVO
	{
		[Bindable]
		public var firstName : String = "";
		[Bindable]
		public var lastName : String = "";
		[Bindable]
		public var email : String = "";
	}
}

Step 2: Save your Value Object to a File

The code shown in the next two steps is almost exactly the same, except that the button handlers are a little different. As you will note, this is an AIR application in which we create a simple form that writes a constructed data object to a specified location on the file system when the Save button is clicked.

<?xml version="1.0" encoding="utf-8"?>
<mx:WindowedApplication xmlns:mx="http://www.adobe.com/2006/mxml" layout="absolute">
	<mx:Script>
		<![CDATA[
			import com.resource.vo.SimpleDataVO;
			import flash.filesystem.*;

			[Bindable]
			private var dataObject : SimpleDataVO;

			override protected function initializationComplete():void
			{
				super.initializationComplete();

				dataObject = new SimpleDataVO();
			}

			private function saveFileButtonHandler ( event : MouseEvent ) : void
			{
				// First, generate your ByteArray from the VO.
				var byteArray : ByteArray = new ByteArray();


				byteArray.writeObject( dataObject );

				// Resolve your file location.
				var file : File = File.applicationStorageDirectory.resolvePath( "testFile.ri" );
				var fileStream:FileStream = new FileStream();

				// Save the file to the given location.
				fileStream.open(file, FileMode.WRITE);
				fileStream.writeBytes( byteArray );
				fileStream.close();
			}

		]]>
	</mx:Script>
	<mx:Form x="10" y="10" width="363" height="196">
		<mx:FormItem label="First Name">
			<mx:TextInput id="firstNameInput" text="{dataObject.firstName}" change="dataObject.firstName = firstNameInput.text"/>
		</mx:FormItem>
		<mx:FormItem label="Last Name">
			<mx:TextInput id="lastNameInput" text="{dataObject.lastName}" change="dataObject.lastName = lastNameInput.text"/>
		</mx:FormItem>
		<mx:FormItem label="Email">
			<mx:TextInput id="emailInput" text="{dataObject.email}" change="dataObject.email = emailInput.text"/>
		</mx:FormItem>
		<mx:FormItem>
			<mx:Button label="Save" click="saveFileButtonHandler(event)" />
		</mx:FormItem>
	</mx:Form>
</mx:WindowedApplication>

Step 3: Read your Value Object

This next application written for web deployment, and matches the above in all but the button handler. Here we use the URLLoader and URLRequest classes to retrieve our file from an upload directory. Once it’s been received we access the returned data and let the ByteArray class reconstruct our logical object.

<?xml version="1.0" encoding="utf-8"?>
<mx:Application xmlns:mx="http://www.adobe.com/2006/mxml" layout="absolute">
	<mx:Script>
		<![CDATA[
			import com.resource.vo.SimpleDataVO;

			[Bindable]
			private var dataObject : SimpleDataVO;

			private function loadFileButtonHandler ( event : MouseEvent ) : void
			{
				// Execute the file load.
				var request : URLRequest = new URLRequest ( "upload/testFile.ri" );
				var receptor : URLLoader = new URLLoader( request );

				// Make sure our content is interpreted as a ByteArray.
				receptor.dataFormat = URLLoaderDataFormat.BINARY;
				receptor.addEventListener( Event.COMPLETE, fileLoadedHandler );
			}

			private function fileLoadedHandler ( event : Event ) : void
			{
				// Retrieve the event target, cast as the URLLoader we just created
				var loader : URLLoader = event.target as URLLoader;

				// Retrieve the loaded data. We know it's a ByteArray, so let's cast it as well.
				var data : ByteArray = loader.data as ByteArray;

				// Use the ByteArray.readObject method to reconstruct the object.
				var obj : Object = data.readObject();

				// Cast the object and assign it to our data container.
				dataObject = obj as SimpleDataVO;
			}
		]]>
	</mx:Script>
	<mx:Form x="10" y="10" width="363" height="196">
		<mx:FormItem label="First Name">
			<mx:TextInput id="firstNameInput" text="{dataObject.firstName}" change="dataObject.firstName = firstNameInput.text"/>
		</mx:FormItem>
		<mx:FormItem label="Last Name">
			<mx:TextInput id="lastNameInput" text="{dataObject.lastName}" change="dataObject.lastName = lastNameInput.text"/>
		</mx:FormItem>
		<mx:FormItem label="Email">
			<mx:TextInput id="emailInput" text="{dataObject.email}" change="dataObject.email = emailInput.text"/>
		</mx:FormItem>
		<mx:FormItem>
			<mx:Button label="Load" click="loadFileButtonHandler(event)" />
		</mx:FormItem>
	</mx:Form>
</mx:Application>

A few caveats

For ByteArray to properly serialize and deserialize your object, two things must be the case. First, you must use the [RemoteClass(alias='')] metadata tag. Secondly, you have to make sure your value object is compiled into your application. This can be done either implicitly (via the type declaration I used above), or explicitly by adding your class to the mxmlc compiler arguments.

Additionally, there are a few things  I explicitly omit in this tutorial. Foremost of these is the send/receive architecture that would ensure the proper distribution of a file created from AIR. This will vary by implementation, so I’m not going to tell you how this should or should not be done. The other is that I don’t discuss the creation of ByteArrays within Flex and returning them to AIR. Since the Flash Player doesn’t yet allow you to save to the desktop without bouncing through a server, the implementation of Save-From-Flash can be complex and involved, and will have to wait for a future article.

Launching Cruise Control during Startup on OSX

I spent a good amount of time today trying to figure out how to turn my “old” iMac into a development environment, which included installing Cruise Control. Since I didn’t want to run the launch script manually, I tried (and failed) to find someone who’s successfully gotten cruise to autostart on boot. A bit of investigation later and I came up with a solution, drawn from an excellent blog post on adding your own osx boot launch scripts, an archive script on launching cruise on unix init, and the tacit information in the MySQLCOM Bootstrapper contained in the OSX Binary Distribution.

Step 1: Download and Install Cruise Control

You can download Cruise Control from Sourceforge. For the purpose of this demo I’ve used the binary distribution, though you can compile your own. Once you’ve downloaded your distribution, unzip it into a convenient directory. I’ve used the base applications directory: /Applications/CruiseControl.

Step 2: Create a User

It’s best practice to not run cruise as root, so open up your user accounts and create a new user called Cruise Control.

Step 3: Create a folder in the StartupItems directory.

$ sudo mkdir /Library/StartupItems/CruiseControl

Step 4: Create the Parameters File

Create a file called /Library/StartupItems/CruiseControl/StartupParameters.plist. This file will contain your startup launch configuration information- in other words it’s a file describing your startup task.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
	<key>Description</key>
	<string>Cruise Control</string>
	<key>OrderPreference</key>
	<string>None</string>
	<key>Provides</key>
	<array>
		<string>CruiseControl</string>
	</array>
	<key>Uses</key>
	<array>
		<string>Network</string>
	</array>
	</dict>
</plist>

Step 5: Create/Configure the launch script

The launch script needs to be command line executable with the following three options: start, stop, reset. I’ve included my sample here, but I recommend you adjust it to match your own configuration and user name.

#!/bin/sh
#
# Error out when no parameter given
if [ -z $1 ] ; then
	echo "Usage: $0 [start|stop|restart] "
	exit 1
fi

# Source the common setup functions for startup scripts
test -r /etc/rc.common || exit 1
. /etc/rc.common

###################################################################################################
# USER CONFIGURATION

# The user under which CruiseControl should run.
CC_USER=cruisecontrol

# Cruise Install Directory
CC_INSTALL_DIR=/Applications/CruiseControl

# Directory for the config.xml file.
CC_WORK_DIR=$CC_INSTALL_DIR

# Logfile Directory
CC_LOGFILE_DIR=$CC_INSTALL_DIR

# Port under which Cruise should run
CC_WEBPORT=8080

# JMX Port. It's suggested you don't change this unless the port is already in use.
CC_JMXPORT=8082

# Port for the RMI gateway, leave blank to disable.
CC_RMIPORT=

###################################################################################################
# DO NOT MODIFY ENTRIES BELOW THIS LINE

NAME=cruisecontrol
DESC="CruiseControl - continuous integration build loop"

PATH=/sbin:/usr/sbin:/usr/bin:/bin
export PATH

CC_DAEMON=$CC_INSTALL_DIR/cruisecontrol.sh
CC_CONFIG_FILE=$CC_WORK_DIR/config.xml
CC_LOG_FILE=$CC_LOGFILE_DIR/cruisecontrol.log
CC_COMMAND="$CC_DAEMON -configfile $CC_CONFIG_FILE -webport $CC_WEBPORT -jmxport $CC_JMXPORT -rmiport $CC_RMIPORT -user youruser -password supersecret"

# overwrite settings from default file
if [ -f /etc/default/cruisecontrol ]; then
  . /etc/default/cruisecontrol
fi

# does the executable exist?
test -f $CC_DAEMON || (echo "The executable $CC_DAEMON does not exist!" && exit 0)

if [ `id -u` -ne 0 ]; then
        echo "Not starting/stopping $DESC, you are not root."
        exit 4
fi

# Get the PID output from the startup script
if [ -f $CC_INSTALL_DIR/cc.pid ]; then
	CC_PID=`cat $CC_INSTALL_DIR/cc.pid`
else
	echo "No cc.pid file found.  CC process may not be controllable from this script!"
fi

StartService ()
{
	cd $CC_INSTALL_DIR
	su $CC_USER -c "/bin/sh -c \"$CC_COMMAND >> $CC_LOG_FILE 2>&1\"" & RETVAL=$?
	echo "$NAME started with jmx on port ${CC_JMXPORT}"
}

StopService ()
{
	if [ -n "$CC_PID" ] && ps -p ${CC_PID} > /dev/null ; then
		kill -9 ${CC_PID}
		RETVAL=$?
	else
		echo "$NAME is not running"
		RETVAL=1
	fi
}

RestartService ()
{
	RunService "stop"
	RunService "start"
}

RunService "$1"

Step 6: Set permissions

Given that the startup script will ignore anything that isn’t owned by root, and right now cruise still doesn’t have write access to the cruise install dir, we have to make a few adjustments to our file structure to make this work.

$ sudo chown -R cruisecontrol /Applications/CruiseControl
$ sudo chown -R root /Library/StartupItems/CruiseControl
$ sudo chmod 755 /Library/StartupItems/CruiseControl/CruiseControl

That’s it. I hope this was helpful

Using the Metrics Package to record Flash Application Analytics

Web analytics is a way in which individual visitor action can be easily tracked within a site, and the aggregate statistical data derived from this can often lend remarkable insights into the effectiveness of your design, how ‘sticky’ your content is, and what your users are actually looking for. Unfortunately, extending this paradigm into flash has always been tricky, because it doesn’t adhere to the page-based paradigm on which most Analytics packages are built. Once a flash application or widget is loaded, the server loses most knowledge about what the user is actually doing within it.

Usually this isn’t really a problem- flash applications have not been too complex and not many people care where on the banner you clicked, just that you left the site as a result. Yet now with the strong growth of Flex and Ajax our web applications are becoming more and more complex, and marketers and usability experts are now demanding this tracking data in spite of the paradigm limitations.

If you really think about it, what we really are interested in tracking is a user action, rather than the page loads we are collecting right now, which means that the largest part of an analysts job is turning these page events into meaningful user actions, rather than interpreting those users. There’s been some attempt to set metrics to individual places within a page flow, yet nobody has yet thought to rethink the paradigm. But I digress…

Most Metrics providers have since opened their API’s enough to allow a developer to pretend like a new page refresh has occurred. While this is hardly optimal, it does allow us to track user events from inside of flash, but the fact that each provider has implemented their API a different way means that implementing metrics for each is still a string of unique problems to solve.

To that end I’ve written the Metrics package, which is intended to provide a common metrics proxy that any developer may use, which relies on a common library of connectors that can be swapped out as needed. In this article I go over the details of how to use it, and touch on how individual connectors might be written. You can access and download the actual library here.

Step 1: Initialize the Metrics Engine

The first step is to initialize the metrics controller with the connectors you’d like it to use.

import com.practicalflash.metrics.connectors.GoogleConnector;
import com.practicalflash.metrics.connectors.UrchinConnector;
import com.practicalflash.metrics.Metrics;

// Add the connectors
Metrics.addConnector( com.practicalflash.metrics.connectors.GoogleConnector);
Metrics.addConnector( com.practicalflash.metrics.connectors.UrchinConnector);

Note that the package can take an arbitrary number of connectors, yet will only instantiate those which successfully autodetect their environment. Should you want to deploy an application and don’t know in advance what environment you’ll be reporting to, you can hedge your bets and add several, as I have above. The only caveat is that any provided class must implement the IMetrics interface.

Step 2: Dispatch a Metrics Event

Once you’ve initialized your connector, you may dispatch metric events from any point in the application by creating a MetricsEvent object. The following example demonstrates a case where you might have an event listener waiting for a particular user action.

import com.practicalflash.metrics.MetricsEvent;

function someEventHandler( event : Event ) : void
{
        var metricEvent : MetricsEvent = new MetricsEvent("/this/is/a/test.html?foo=bar");
        metricEvent.dispatch();
}

Additional options include setting the event type, the dispatching target, or adding an arbitrary number of parameters to be parsed and passed along. Exact behavior for these parameters is described in the documentation, and may be dependent on their level of support within the various connectors.

metricEvent.target = event.target;
metricEvent.type = MetricsEvent.RECORD;
metricEvent.addParameter( "bar" , "foo" );

Further Customization

If any of the provided connectors don’t suit your purpose, the task of writing your own is actually fairly simple. The only requirement for the connector is that it implements the IMetrics interface, which prescribes two functions: detect(), which runs any autodetection script you’d care for, and send(), which should process and deliver any received information appropriately.

Additionally, the AbstractConnector class is provided with a few useful utility features, including a singleton instance of the JavascriptConnector (accessible via the jsBridge parameter) as well as getDOMBounds(), which returns an objects’ rectangular bounds with respect to the HTML DOM (just in case your metrics package supports the reporting of a page’s click area).

A sample connector might be as simple as the following:

package com.practicalflash.metrics.connectors
{
        import com.practicalflash.metrics.MetricsEvent;
        import com.practicalflash.metrics.interfaces.IMetrics;

        public class UrchinConnector extends AbstractConnector implements IMetrics
        {
                public function detect() : Boolean
                {
                        if( available )
                                return jsBridge.hasParameter( "urchinTracker" );
                        else
                                return false;
                }

                public function send( event : MetricsEvent ) : void
                {
                        jsBridge.urchinTracker( event.page + "?" + event.getURLString() );
                }
        }
}

Using ExternalInterface to write Javascript to the HTML DOM

Every Actionscript Developer will sooner or later have to interface with the webpage in which his .swf is presented, and Adobe has provided the ExternalInterface Class to simplify this process. Even so, communicating with an external javascript API almost inevitably requires some number of bridging functions to properly couple the two runtime environments. In cases where the developer has control over both environments this poses no further problems, as for example in an application where the .swf takes animation and event cues from an interface written largely in HTML. Some instances nevertheless remain where we don’t have that luxury, and are writing to an external API that we can neither guarantee nor contribute to; In short, we cannot add a javascript file to the HTML wrapper that will act as our translation bridge.

This situation is particularly important for individuals writing packages to simplify interaction with common public interfaces such as Omniture, WebTrends, or other Web 2.0 API’s. When developing for these, we’re left with the chore of packaging a javascript file with our .swc as well as extensive documentation on how to properly integrate the pieces. Yet it turns out that this rather messy solution is not necessary at all, because of a peculiar behavior of ExternalInterface.call(): Rather than execute the method of the name that you pass, it attempts to resolve whatever you pass to it as a function. The following example demonstrates this:

import flash.external.ExternalInterface;

// This line passes a string to the alert function within the javascript environment.
ExternalInterface.call("alert","foo");


// This line creates a function that sets a variable and then calls alert on it.
ExternalInterface.call("function() { alertString = 'foo'; alert(alertString); }");

// Note that they both do exactly the same thing.

Since javascript is poorly scoped, this gives us access to set or retrieve anything within the HTML DOM. In short, we can write our own custom javascript directly to the page without having to rely on an external file. Security concerns aside (imagine covertly embedding Facebook’s beacon into a banner ad), the usefulness of this technique should be obvious. Even so, manually writing every javascript function into a method call is no way to keep your code legible, and since actionscript does not allow us to include text via embed directives, so writing external .js files isn’t an option. Instead, the only real option is to define a series of strings, or to make use of E4X to include our javascript in CDATA tags as follows:

<mx:xml id="scriptCache">
	<scripts>
		<script methodcall="demoFunction" methodname="jsDemoFunction">
			<![CDATA[
				function(var1, var2, var3) {
					alert(var1);
				}
			]]>
		</script>
	</scripts>
</mx:xml>

At this point we have a solution to our problem: We can embed our API into our actionscript class, easily edit it within the XML, and write it to the HTML DOM when it is necessary. The last piece of the puzzle then is to ensure any library we write is entirely transparent to other actionscript developers. In short, no method call should visibly use ExternalInterface. This is where mx.flash.Proxy comes in handy, because it allows us to pass arbitrary function calls to a handler of our choice. The implementation thereof is beyond the scope of this article, though you should have enough pointers to write your own. Even so I’ve taken the time to write a class that doesn’t just allow the registration of methods, but also gives you easy access to all the javascript variables. You can download it and the documentation here.