Publishing builds from Hudson to S3 & on to EC2 instances.
UPDATE: The source for this work is now part of Hudson at Github.
The plan: have Hudson publish successful builds into a bucket on S3, and then be able to pull those builds to EC2 instances. The result: success, but it took a bit of head scratching.
Step 1: I wasn't able to get Doug MacEachern's S3 Hudson plugin to work with Hudson 1.367, but on reflection I may have just not been using it properly (this is ok: I can look an idiot so you don't have to). So that'll probably work for you, but I ended up forking the source, updating the code to match the latest Hudson API and adding some error reporting (although similar error reporting can be found in Steve Gilardi's fork).
I know pretty close to nothing about developing Hudson plugins, so I followed what appears to be a tradition of looking at the source of the scp Hudson plugin and figuring things out from there.
$ mvn package
...you'll get errors about Base64 Codec, but you'll end up with target/s3.hpi which you can install in Hudson and then...
Step 2: configure
The screen shots show you what to do, but in summary: define a profile (your AWS credentials) globally; then for each project, set it to publish particular files to particular buckets using the profile.
Run your build, and files appear on S3.
Step 3: the final step is to be able to pull those build artefacts to your EC2 instances. There are lots of ways to do this, but the option I wanted to try was just using cURL. And you can do that, because S3 supports query string authentication. The idea is you define what you want to access (GET /bucket/file), how long the access should be available for (in seconds since the disco era), and then you sign those details. Stuff all of that into a request, and you can download the file. The details are fiddly, so I've put the script on Assembla for anyone to hack around with. The output is a URL that looks like this:
curl -o foo.war 'https://s3-eu-west-1.amazonaws.com/bucket/foo.war?AWSAccessKeyId=ABCD123456789&Expires=1601719200&Signature=Pkd742kjJKHfku8%3D'
- By adding the extra error reporting to the plugin I discovered that Hudson needed more -Xmx to push the build artefacts to S3 (384m rather than the 256m default we'd given it).
- If things don't appear to be working, it may be helpful to create a custom Java logging file and start Hudson with it: java -Djava.util.logging.config.file=log.properties -jar hudson.war
