Hudson + EC2 + Windmill = Cloud Testing

This should give you a pretty good idea how to use the available tools to run your functional tests out there on the cloud.

I do plan to keep updating this post, so consider this a rough draft for the time being! I hope this provides some value.

Setting up Hudson

It would be pretty silly for me to try and re-document the Hudson installation process, so I will refer you to Meet Hudson.

Installing Hudson Plug-ins

Manage Plugins

Click the “Manage Hudson” link, and then “Manage Plugins.

Tabs

Select the “Available” plugins tab.

EC2

Windmill Plugin

Select both the Amazon EC2 plugin, and the windmill plugin, then click Install:Install

You should see the following:

Installed

Click “Restart Now”

Restarting

Success!

Create your slaves

Click “Manage Hudson”

Manage Nodes

Click “Manage Nodes”, then “New Node” in the upper left.

Configure the node

Configure the node, we do 1 executor so that Windmill test runs don’t step on each other, c:\hudson is the standard place to store hudson job workspaces, and to keep other jobs from using our slave we specify “Leave this machine for tired jobs only”, then save.

Setup some jobs

In the upper left link menu, click “New Job”

Configuration Matrix Job

We need to make this a “multi-configuration project”, so that we can run the same job against multiple browsers on multiple slaves. Click save.

Get the test

In this example, I am using wget (which I have installed via cygwin) to pull down the test file into the workspace that we are going to run.

Specify the machines and browsers

In this example I have selected to run this test on three different machines, each against firefox, ie and safari. This would be useful if each of these machines had different versions of the browsers. You can run each of the combination’s from one job.

Windmill Plugin

Test run

Configure your Windmill test run, setting Browser to %browser% allows the job to get the browser from the configuration matrix. We also specify the url, and to run the test file that we download in the previous step. Save.

Setting up EC2

Get an account

Sign Up

Head to http://aws.amazon.com/ and sign up!

Setup Amazon EC2 API Tools

I found this to be relatively straight forward, however these docs look terrifying and poorly organized: http://developer.amazonwebservices.com/connect/entry.jspa?externalID=351

These were much easier on the eyes, and to the point.. plus they are OSX specific, which was nice: http://www.robertsosinski.com/2008/01/26/starting-amazon-ec2-with-mac-os-x/

Once you are all setup, allow connections to all images you launch with remote desktop by issuing the following command:

ec2-authorize default -p 3389

Launch Windmill AMI images

AWS Management Console

Sign In
Navigate to the AWS Management Console

Launch Instances

Click to Launch some Instances

Find AMI's

Click to the “Community AMI’s” tab

Search for Windmill

My pre-maid Windmill AMI’s should appear, ‘windmillxp’ specifies that it’s the windows xp Windmill image. Click ‘select’.

Configure Launching

Your security groups will reflect what you configured for your account, but specify the number of slaves you would like to use. Then click “Launch”.

Confirm

You should get the following confirmation that your slaves have been launched.

Dashboard

Start the slave agents

For each of the slaves in the following list:

Slaves

Connect to instance

Check the image, and click connect.

Download Dialog

Click “Download shortcut file”.

Auth

If you have Remote Desktop Installed, you should connect and see the following dialog. I have also found CoRD to be a great alternative on MacOSX.

You will be prompted for a username and password, use ‘Administrator’ and ‘w1ndmill’ respectively. Note: the second character of the password is the number one.

In the Window you can now access, launch a browser and navigate to the URL of your Hudson Instance and find your slave node in the list.
Slave

Click that the node to get it’s configuration screen.

Launch Slave

Click the “Launch” button, your EC2 image is now an available Hudson slave.

Run that Test

Build

Click the clock image on the right to start the job running. In the VM you should see the Windmill test zooming along. You can also navigate around the job to find access to the console output to see line by line what is happening on the standard out.

Other

The manual launching process can be automated using the hudson EC2 plugin for Linux, but is not yet compatible with Windows. Resolving this step would make the whole process of launching cross platform cross browser tests in the cloud a fully automated process, which we are looking forward to!

Other possible ways to handle this are by using the hudson VMWare plugin, and launch images on a VMWare server.

Posted in Uncategorized | 3 Comments

Windmill Plugin for Hudson

Over the last 6+ months, I have been using Hudson in conjunction with Windmill very heavily for continuous integration. For the most part using the build step specific to whatever the slave OS requires has worked sufficiently well until recently when my needs changed.

I use the ‘configuration matrix’ option to build a matrix of browsers to run the tests, this way I can have one job that represents a test run on multiple boxes and multiple browsers on each box. Drilling down allows me to see the results for each of these test runs within the job. (Configuration Matrix is awesome btw, except for one really annoying bug for, which there is a reasonable workaround)

Having many build resources, sometimes I want to move a job from one machine to another in order to equalize load on the different machines or to have a job running in ff2 and ie6 instead of ff3 and ie7. Thus the need for a uniform build step that will run the same way regardless of the machine or installed browsers, became necessary. Additionally the commands for running tests got bigger and less manageable all the time, so the time for a clean user interface finally came. This way I can automatically append arguments like ‘exit’ to keep the build step interface simple and clean.

The other simplification is that in the Hudson configuration page for the Windmill Test plug-in you can tell it to automatically call the contrib ‘clean_run.py’ script with the correct arguments (assuming clean_run.py can be found in your path).

As I am relatively new to the Java world, I struggled through the development process but thanks to some reasonable plug-in documentation, responses from the hudson dev mailing list and some IRC conversations with Kohsuke (thanks so much) I finally have something that sufficiently addresses my needs.

Screenshots



Project

The code is all available on GitHub and I welcome any improvements or input from the community as I know that continuous integration is a very important piece of the utility provided by Windmill and this pieces should be as easy as possible to setup.

The combination of this plug-in, and others like the amazon s3 plug-in or the hudson VMware plugin bring us that much closer to seamlessly creating test resources, running our functional web tests across all of our supported platforms and then throwing it away until it’s again needed (or using the CLOUD).I have also done some integration with virtual box that I have found to be very successful.

Download

Please log bugs and let me know what you think! If you are an avid Java and or Hudson plug-in developer and you are interested in contributing, please jump right in!

Posted in Automation, Continuous Integration, java, Open Source, Slide, Technology, Web, Windmill, Windmill-dev | Tagged , , , , | Leave a comment

PyCon 2009 Recap

Getting back in the swing of things after conferencing for weeks can be pretty painful, thus the lateness of the post. However I think it’s important to go over some thoughts still lingering in my brain as a result.

First off, I have to say that for those of you who don’t know, PyCon is a community organized event, and amazingly well done. I was impressed by the design of the conference, the way they had four talks going on at once and they tried to keep them in a similar interest track. Every talk I attended was at least “good”, and many were “great”. You could really feel a community vibe, and for a conference that had 800+ attendees in the middle of a major recession they had every right to be excited.

There were two major themes content wise that really impressed me, the first was an amazing amount of web framework focus. Django obviously being the twinkle in the eye of the community, but there were smaller communities for each of the other projects, Pylons-Turbo Gears, web2py and lots of tools built on top of them. One that struck me with some major promise is the Pinax Project. Their goal is to make it so that I don’t ever have to deal with building user registration and in site messaging… and all the other features expected for any site that has social network functionality.

The other major theme was a mini testing conference going on within PyCon, that I was very comfortable hanging around with. We had a hugely successful Birds Of a Feather, as well as a surprisingly active Open Space talk for Windmill.

The “Using Windmill” talk turns out to be pretty successful in every aspect that I really care about. I do wish that I had been able to get a little more sleep the night before, and I have to admit the size of the venue was a bit overwhelming. I now realize watching the footage that I used the word “UM” way too much, and the demo videos must have been hard to watch from the very back of the room. But barring those two things, I am quite happy (I shall learn and practice for the next round of shameless PR at OSCON 2009).

The “Functional Testing Tools in Python” panel was very successful, and a lot of fun. I always enjoy the friendly banter between the different project owners. Everyone has a different opinion on what they care about, focus on and feel they do the best. Obviously since the only two projects represented that focused on Web Testing were Windmill and Selenium, we got a lot of attention.

Watching that panel footage I definitely think that the introductions were too long, but I still think our Journey themed – mind blowing – Windmill demo video was a great intro. At the very least, the audience had a little entertainment before the geek droning began :)

Slide had an awesome presence this year, a fun booth, huge banners everywhere and 6 attendees. It was fun to see all of the great responses I received about Slide from people out there in Python land.

Here are links to the videos:

And some pictures:

Me, with the great Slide backdrop

Mikeal answering Questions More me

If you are interested in seeing the new and improved version of the “Using Windmill” talk, please make it out to OSCON 2009, “Scheduled for 16:30 on 22 Jul 2009.” in San Jose, CA.

OSCON 2009

We are waiting to hear back from both Open Source Bridge and the AJAX Experience as to whether we will be participating in those conferences (fingers crossed)!

Posted in Automation, Career, Continuous Integration, JavaScript, Open Source, Python, Slide, Technology, Web, Windmill, Windmill-dev | Tagged , , , , , , , , , , | 2 Comments

Skinning Windmill with JQuery UI Themes

I have been doing a lot of UI work on the Windmill trunk, and over the past few months I have had multiple requests for the ability to apply skins. Of course my reaction up until this point has been… alter the CSS! Which is not exactly the answer people were looking for.

Here is your answer, and it is now easier than ever.

1. Go to: http://jqueryui.com/themeroller/ and create your theme (or pick a pre-defined one)
2. Download and unzip the file. (it doesn’t matter what you select for jQuery components)
- On a mac you will find something like this: jquery-ui-1.7.1.custom.zip Folder
3. Open the contained CSS folder there will be another folder, currently windmill uses “smoothness”
4. Find windmill/html/css folder in the windmill source directory
5. Copy the specified folder in step 3 into this folder
6. Edit windmill/html/remote.html, line 10 to read like the following:

1
<link type="text/css" href="css/*your folder name*/jquery-ui-1.7.1.custom.css" rel="stylesheet" />

7. Load windmill!!

Here are a couple examples of pre-defined themes I tested out.

Humanity
Windmill Theme
UI Darkness
Windmill Theme

Happy skinning.

Posted in Automation, JavaScript, Windmill, Windmill-dev | Tagged , , , , , , | 1 Comment

MozMill 1.1 UI Overview

During the Open Design session at Mozilla with Aza we were informed that we could load a HTML file with a Chrome URL, allowing me to rebuild the MozMill UI a bit more like a web page instead of using the XUL constructs that I had been struggling with. Granted it feels a lot more like a web page than it does a desktop application, but the speed that I can build new UI features by using libraries like JQuery UI have made it worth it.

The combination of writing content style HTML, and the good advice we received have come together into what I feel is a pretty usable user interface. Granted this is the first revision and will probably continually be refined to become even more user friendly, but from 1.0 to 1.1 it is a vast improvement.

MozMill Editor

Improved Editor

The first major improvement is the implementation of a full featured code editor named EditArea.

We have been keeping our eye on Bespin, which we will look more into integrating when it is a bit more modular.

For the meantime EditArea does a great job, and was *relatively* painless to integrate.

Some of the features include:

  • Multiple file editing
  • Syntax hilighting
  • Full Screen mode
  • Adjustable font properties
  • Jump to line numbers
  • Search and replace
  • Automatic tabulation
  • Toggle hi-lighting
  • Toggle edit modes

EditArea implements execCommand similar to the implementation in Midas.

Reorganized Menu’s

Part of the move from XUL involved no longer relying on the toolbox, so we have reorganized the menu’s into dialog’s (don’t worry most functions have a keyboard shortcut if you are one of those people that doesn’t want to deal with that extra click).

This cleanly displays all the available options, and doesn’t clutter up the main UI. This also provides space to easily add new features that fall into these logical spaces in the future.

Test Dialog

File Dialog

Options Dialog

Improved Inspector
Inspector

There was a lot of frustration when it came to getting the results from the inspector into the editor, some of this had to do with the non editable default nature of the elements we were using to draw them out in XUL, but the integration of the feature into the IDE became very messy and confusing.

This creates an obvious separation between the rest of the UI and the inspector feature and puts helper features out in front of you to simplify the process by dumping to the clipboard and then moving you to a focused editor window.

Improved Output
Output

The output UI has been completely revamped to give you the most important information quickly, but allow you to navigate down an expandable tree to explore the output of the exception.

All of the information in the output divs can be easily selected and copied to stick in bugs etc, but also saves you a trip to the error console as it should encompass all of the information being thrown in the error object, serialized and organized into a more readable format.

Thanks to JQuery UI’s information and error boxes I was able to tweak the CSS to make some relatively attractive, but more importantly, informative UI that should quickly give you the status of your test run.

More Information

Thanks everyone who logged bugs!

Posted in Automation, Firefox, JavaScript, Mozilla, Open Source | Tagged , , , , , , | 3 Comments

Micro-Bookmarking with MyTabs

MyTabsA few weeks ago I realized that every time I boot my laptop, one of the first things I do is to open Firefox, and immediately load about 7 web sites as tabs. Some of them requiring user interaction to navigate to the desired state. I didn’t realize that this was actually a phenomenon called “micro-bookmarking”, and that I may not be the only person who has this routine.

I don’t necessarily want to bookmark these sites, and this process of opening them is somehow part of my routine, however spending this kind of time every time I want to get to this state is just silly. Part of the ritual is to go through each of the open tabs, read the most recent updates and then close the tab, repeating this whole process many times a day.

At this point you should be starting to get the jist, to automate this whole process I created a Firefox Extension called MyTabs. The UI and use case is simple, but some of the tools I built in make it potentially very powerful.

MyTabs has the MozMill Controller built in, and is automatically setup in the tabs content window scope, allowing you to write and attach scripts to each tab to be executed ‘onload’. Thus automating the need to ever manually login to your web applications again.


Example script for logging into Facebook:

1
2
3
4
5
6
7
8
9
10
11
12
13
//Lookup the three elements I want to interact with
var email = new elementslib.ID(doc, 'email');
var pass = new elementslib.ID(doc, 'pass');
var login = doc.getElementsByClassName('UILinkButton_A')[0];
 
//Controller expects elementslib objects
var elibLogin = new elementslib.Elem(login);
 
//Tell the controller to type the credentials
tab.type(email, 'my_facebook_account@address.com');
tab.type(pass, 'my_password');
//Click the login button
tab.click(elibLogin);

This is all possible by a really slick snipped of code I found on MDC:

1
2
var newTabBrowser = gBrowser.getBrowserForTab(gBrowser.addTab("http://www.google.com/"));
newTabBrowser.addEventListener("load", function() { newTabBrowser.contentDocument.body.innerHTML = "Stuff"; }, true);


Security/Preferences

For the sake of security I made it so that you can only attach scripts, not edit them so that you friends can’t go and view your script to get those plain text passwords. All the tab data is stored in the prefManager so that it is available across sessions. However if someone knew enough to pick apart the source, install a trusted JS Shell and manually access the prefManager you may be in trouble… so be careful about super sensitive information especially on a shared machine.

Using jQuery UI I was able to make the UI relatively attractive, intuitive and possibly skinnable in the future (if you want to do this yourself, grab the source swap out the jQuery UI theme CSS directory and boom you have yourself  new look.)


Some Screenshots

Welcome
Tab OverviewAdding new TabsTab Scripting

The source is fully available on GitHub, and the XPI is available on Mozilla Add-ons.

Remember that I am currently on bug fix release 0.3, but please let me know if you run into any problems!

Posted in Firefox, JavaScript, Mozilla, Open Source, Web | Tagged , , , , , | 2 Comments

XUSH (.01), you know it’s fun to say.

Over the past 8 months I have been neck deep in XUL and XPCOM and with Venkman being as unintuitive as it is I have badly needed a shell with access to the trusted space in the browser. I also had a couple more requirements, which were that it was super easy to get at (keyboard shortcut), and that it looks awesome (transparency required).

Since making things looks really nice in XUL is really hard, I decided that I would build the UI in the current Firefox content window (sorry tbird etc, but I built a less shiny version of this into Mozmill trunk for all of you).

Some of the functionality that I decided was a top priority was some slightly abstracted access to the window mediator which I am calling ‘windows’, and is simply an array of all the different windows available in the whole browser. The other big one is dir(), I am constantly looking for a property in one of the many, many objects I am trying to access and now that is really easy.

All of that goodness combined with enough space to stick real code blocks and a full history (which you can access using the shift key and up/down arrows), I find myself pretty comfortable.

After installing the extension, you can simply press ‘meta (cmd on mac) + shift + s’ and you should be greeted with the following (toggling will show/hide the shell):

UPDATE: It was pointed out to me that on windows meta may not be bound, so I have updated both the extension and git, please try ‘alt+shift+ctrl+s’ or ‘ctrl+shift+s’ on windows/linux, (I am testing in a VM so it’s a bit strange. Sorry about the confusion!) 

 

Welcome to XUSH

Welcome to XUSH

To get started, you can type ‘help’ to see what is available to you.

Help Menu

Help Menu

 

 

Other available functionality includes a big chunk of the mozmill controller for driving user actions in the browser, as well as elementslib — which gives you all kinds of powerful features for looking up nodes in the browser.

ElementsLib

ElementsLib

As I mentioned above, the windows helper can quickly give you an idea of what is open in the browser and let you directly access the objects in the windows[x] array.

Windows Helper

Windows Helper

 

The nice big console gives you enough space (and using shift) you can do multiple line commands to actually execute code snippets:

Code Blocks

Code Blocks

My next round of features will include some more helpers, including tab completion and any bug fixes I find, please let me know of any that you find!

If you would like to play with the source, I have it on github and have a python script that uses mozrunner to launch it in dev mode (with venkman installed, although now I don’t really need it), but saves you from installing and uninstalling the extension every-time you want to try out your latest code changes:

python xush.py

That should launch Firefox with XUSH and Venkman both installed and allow you to hit the shortcut and wham, test away.

After another round of bug fixes I will be submitting this to Mozilla AMO.

Other projects you might be into if this interests you:

Thanks for your interest!

Posted in Firefox, JavaScript, Mozilla, Open Source, Python, Web | Tagged , , , , , , , , | 10 Comments

Meet lookupNode

A few months ago I did a re-write of the DOM access functionality used by Windmill and various other projects of mine and the result was a wrapper around ElementsLib called lookupNode. I use this all over internally but until today didn’t realize that it could be hugely helpful for test developers and people trying to debug their applications.

The following list are the different options you have for looking up your node:

  • link
  • xpath
  • id
  • name
  • value
  • classname
  • tagname
  • label

Keep in mind that we iterate all iframes we can access in the window if the node isn’t found in the page document. lookupNode takes an object, like so:


var myNode = lookupNode({id:'myNodeId'});

The very interesting use case we had today was the need to access and parse data from a link on a page in a Python test, I believe the final result looked something like the following:

url = client.commands.execJS(code="lookupNode({link:'Link Text'}).href")["result"]
#url parsing in python goes here into needed_value
client.asserts.assertValue({id:'mycheckbox', validator=needed_value})


lookupNode in Action

lookupNode in Action



I hope someone finds this useful, the abstracted standalone ElementsLib for the content space can be found here!

Posted in Automation, Open Source, Web, Windmill, Windmill-dev | Tagged , , , | Leave a comment

This thing I’m calling Windmill-Lite

In preparation for the Windmill 2 client side re-architecting, and an article I am writing about simulating user sessions with JavaScript, I decided it was time to go through the Windmill JavaScript source and pull out the pieces necessary to drive a user session in JavaScript.

It turns out that it’s easily broken into a few pieces:

  • Events: Cross browser compatible event firing functionality
  • ElementsLib: DOM element lookup functionality via many methods called ‘locators’
  • Controller: The logic for firing the right events to simulate user actions such as ‘type’

In order to test all the functionality I wrote a small runner implementation, which was actually pretty fun and interested because of the need for waits and sleeps. As I didn’t want to have a continuous loop running, which has been the standard solution up to this point I decided to go about mine using a execution queue instead.

By ‘queue’ I mean that I’m using a JavaScript array with push and shift to keep track of what user actions need to be executed next. When a new action is pushed, if there is nothing in the stack it immediately executes, however if there are actions in the stack waiting those are then executed in order then it is executed. This works great, however there is one more level of complexity here involving sleeps and waits. If a sleep or wait is called a flag is set called ‘asleep’, that tells the executor to pop any new actions onto the stack instead of trying to execute them. When the sleep is done the asleep flag is turned off and all the actions in the stack that have been queued up are then executed in order.

Tests are very simple.

var test_run = function(){
wm.user('type',{'text':'testing', 'name':'q'});
wm.user('waits.sleep', {'ms':5000});
wm.user('type',{'text':'numbertwo', 'name':'q'});
wm.user('asserts.assertValue', {'name':'q', 'validator':'numbertwo'})
wm.user('waits.sleep', {'ms':3000});
wm.user('type',{'text':'finally', 'name':'q'});
wm.user('asserts.assertValue', {'name':'q', 'validator':'finally'})
}

This test can be executed against the Google Home page, and it simply types text in to the search box, sleeps a few times and asserts that the value typed in is actually in the form field when it should be. wm is the windmill code namespace, and user is the dispatch function that takes the method to executed and it’s parameters as an object.

Obviously as this was all taken out of the Windmill source all of the JUM functions are also available in wm.ctrl.asserts if you would like to mix them into your test.

So how would one use this?

There are actually a couple ways, one is to include the source in the page you want to test:

<script src="http://adamchristian.com/wmlite/wm-lite.js" type="text/javascript"></script>

or you can use the JavaScript Console to inject it into your page:

x = document.createElement('script')
x.src = "http://adamchristian.com/wmlite/wm-lite.js"
document.body.appendChild(x)

There is code that is executed when this file is included that figures out if the page has been loaded, if it has it calls in init function to enable the key shortcut listeners. If the page hasn’t been loaded, the window onload is pointed at the init so you are sure to have the shortcuts available to access the UI.

The short-cut is ALT-T. I was thinking ‘test’, meta-t and ctrl-t are almost always assigned by the browser but alt appears to work in all my test cases.

“Windmill-Lite UI Screenshot”

Windmill Lite UI

Why?

I don’t have a specific use case for this code/project but I know there are many uses out there for the functionality in the libraries, and for a low overhead cross browser litmus test tool it has some serious potential… also, it was fun.

Links


Additional Coolness

I’m sure there is something like this out there, but I got pretty annoyed trying to use various JavaScript compressor tools to make a single minified file, so I hacked one together in Python called ‘compressjsdir’. It requires jsmin.py, and you can find a zip of everything required here: http://adamchristian.com/code/compressjsdir.zip, or grab the source from github: http://github.com/admc/compressjsdir/tree/master

To run it you can simply do the following (after you put the compressjsdir directory in your path):

dhcp-10-10-9-230% cd directory_of_js_files
dhcp-10-10-9-230% compressjsdir.py
Reading and compressing...
Writing file...

There should now be a file called out.js, which is composed of all the js files in the directory minified and concatted together. This is all based on the JS compressor code Mikeal Rogers wrote for Windmill in our quest for performance.

I will be adding a couple easy features in the next few days, one of course is to take a parameter used to name the output file, and the other argument will be a comma delimited list for ordering of the files as they are minified and appear in the result (in case  you have dependent files).

More

If you are interested in more related or similar projects, you may want to check out FireUnit.

Posted in Automation, JavaScript, Open Source, Python, Technology, Web | Tagged , , , , , , | Leave a comment

Diving into GIT

Over the last year, I have known that the day would come when I could no longer avoid moving from SVN (my comfort zone) to this new beast called GIT that everyone is so excited about. My first hour, which was installing it on my Mac and pulling down a repo to play with was very pleasant. My ssh keys and ~/.ssh/config was already setup the way I wanted and everything just worked.

The pain began when I started in on moving our build slaves over to GIT. Of course two of the three are running Windows because we have to run the Windmill tests against IE. In an attempt to keep things simple, I wanted to avoid installing cygwin on the machines, so I tried msysGIT. Oh WAIT I have to get ssh to work before I can actually use GIT to pull down the repo. 

After trying and failing with openSSH, I finally realized that the Windows puTTY Package was the best way to go about this. There was much frustration involved with this process because it requires that you take the ssh key you generated on your mac (and had added on the server) and convert it to a puTTY ppk. Fortunately this turns out not to be that panful using puTTYgen. The next piece to the puzzle was to get pageant to load this key automatically when the machine boots. I went through some rigamarole trying to create a shortcut in the “Startup” items and appending a string to it’s path to get it to load the correct key. This didn’t work, I’m sure it was a combination of Windows being terribly un user friendly and my brain expecting things to “Just Work TM”. Finally I just created a short cut for the actual key and stuck it in the “Startup” folder, and it works! (You still have to enter your pass phrase on boot) but its better than the alternatives. I actually found a post on a puTTY forum where someone was asking to automate this piece too and the response was basically, “No, never, die. Doing that defeats the purpose of SSH”. 

One step that I forgot to mention is that to get GIT to use the right SSH key you have to set the environment variable: GIT_SSH=path/to/putty/plink.exe

Next I went and started my git clone, which kicked off perfectly. That wasn’t so bad! 46% later at 1.99GB of the retrieval process I received two fatal errors that looked similar to file system errors. I then did some Googling to find that msysGIT only supports repository’s up to 2GB. The answer on the forums for this problem was, “have a smaller repo”. This is a problem, so I head back to the GIT homepage, where I find that the only real lasting option is to use cygwin GIT. 

After I get cygwin GIT all installed I finally get the repo fully downloaded I check to see if I can use the msysGIT to pull changes (which should never be 2gb worth), and it absolutely freaks out. So that’s not an option, and neither is adding cygwin/bin to your path because that complains about ‘exec ssh’ not existing.

Your probably wondering, why don’t you just use cygwin to do your GIT stuff? Well the issue is that we are using hudson to queue up jobs on the machines using the cmd environment to pull changes and run the tests in the repo. And it became very clear that the best way to handle this was to run the slave agent from cygwin, that way the jobs are actually running in cygwin land and this turned out to work — awesome!

To finish describing my pain, the last problem was that since I am running the tests from c:\main, (you have to be inside a git repo to pull and I want to keep these jobs simple) I am generating the log file in the wrong place, because the job looks in it’s home directory to find the jUnit Compatible XML results file. Fortunately there are some environment variables that I can use to get the files back where I want; mv *.log %WORKSPACE%

Now that life is back to normal and I’m getting used to living in the new GIT world, I can get back to important things like making Windmill more awesome and increasing test coverage.

Cheers!

Posted in Automation, Continuous Integration, Hudson, Slide, Technology, Windmill, Work | Tagged , , , , , , , | Leave a comment