Posts Tagged ‘Watin’

Web Integration Tests – with or without the browser

November 24, 2008

In this post, I would like to go in the topic of web tests – automatically verifying the functionality of a full deployed web site.
Automating the browser has been a popular approach, which gives the benefit of testing the site, as the user sees it, including images, css files and secondary functionality implemented with javascript as Ajax.
Internet Explorer can be automated through its COM API – this is a popular approach that is being used by several open source and commercial libiraries like watin and watir.
Other innovative way is taken from the selenium framework, which uses javascript to simulate user actions, which makes it portable across browsers and operating systems.

Automating the browser there is one big problem though – the tremendous amount of web requests issued for loading the complete page, including all images, css files, ad banners and so on, which can be a big performance hit if you have a lot of tests. While it is important to verify, that everything is properly deployed, all images are found and so on, the loading of all this secondary clutter does not provide any benefit for the functional tests and just slows the execution time.
When the tests run for several hours it is very difficult to use them as a feedback mechanism about the project status. Slow tests aren’t used by the developers and at the end, aren’t used at all and provide little or no value the project.

Confronted with this problem the test engineering team at AutoScout took another approach – testing without the browser, using direct http requests. This technique appears a little more challenging to use, but cuts the tests execution time by orders of magnitude. Our http tests run for a minute and half and provide very fast and reliable feedback, which makes them very useful.

If you want to use direct http tests there are several ways to write them – for .NET you can use the MS Test library that ships with Microsoft Visual Studio Testing Edition, or the HttpWebRequest/HttpWebResponse classes in the System.Net namespace. For MS Test library you have to install Visual Studio Tester Edition on your build servers.
For java there is the HttpClient package from apache. We ended up using our own small http implementation, optimized for testing purposes.
If there is interest about it, I would gladly go into more detail about it.

I would like to point out, that using direct http test requests does not replace browser tests when comes to testing ajax functionality and deployment issues of the secondary files. Our advice would be to use direct http tests when you can and browser tests when you have to. Combining the best of both worlds has given us satisfying test coverage and feedback speed.

Happy testing!