A large part of the utility in a testing framework like Windmill is the ability to interoperate with a continuous integration environment. Much of the work that has gone into Windmill recently has been the result of continuous integration needs. There are many ways to do this with existing software packages out there that include Tinderbox,Buildbot and Cruise Control however we picked Hudson as a result of the super small learning overhead and amazing simplicity required to setup slaves on the network.
One of the requirements of course for parsing results is the need for JUnit compatible XML output from the Windmill test runs. I don’t claim to be a Python wizard, or a XML/Java wizard for that matter but it wasn’t that painful to hammer out a function to generate some minimal output to get the process off the ground.
I would love to get a wiki page up on Get Windmill to start documenting the many ways to use Windmill in a continuous integration environment. So let me know if you have a working setup and would like to contribute.
Example Reporting Excerpt from __init__.py:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 | from functest import reports from datetime import datetime class JUnitReport(reports.FunctestReportInterface): def summary(self, test_list, totals_dict, stdout_capture): total_sec = 0 for entry in test_list: time_delta = entry.endtime - entry.starttime total_sec += time_delta.seconds out = '<?xml version="1.0" encoding="utf-8"?>\n' out += '<testsuite errors="'+str(totals_dict['fail'])+'" failures="'+str(totals_dict['fail'])+'" name="windmill.functional" tests="'+str(len(test_list))+'" time="'+str(total_sec)+'">\n' for entry in test_list: if entry.result is not True: entry_time = entry.endtime - entry.starttime out += '<testcase classname="'+entry.__name__+'" name="'+entry.__name__+'" time="'+str(entry_time.seconds)+'.'+str(entry_time.microseconds)+'">\n' out += '<failure type="exceptions.AssertionError">\n' #out += str(stdout_capture) #until I can figure out how to get the traceback out += 'There was an error in '+ entry.__name__ out += '\n</failure>\n' out += '</testcase\n>' else: entry_time = entry.endtime - entry.starttime out += '<testcase classname="'+entry.__name__+'" name="'+entry.__name__+'" time="'+str(entry_time.seconds)+'.'+str(entry_time.microseconds)+'"></testcase>\n' out += '<system-out><![CDATA[]]></system-out>\n<system-err><![CDATA[]]></system-err>\n' out += '</testsuite>' f=open('continuous_test.log','w') f.write(out); f.close() reports.register_reporter(JUnitReport()) |
Happy automating!
