Writing clean WebDriver test suites for “duplicate” functionality by parameterizing on page objects

TL;DR: To avoid duplicating test code, it might be a good idea to parameterize tests on your page objects.

While duplication is mostly a bad thing when it comes to user interfaces as well as at is for code, there are cases where it might be desirable: For example, you might have several user interfaces for the same feature which will be used for A/B testing or expose the same functionality using a different interface for a specific user group, which was my use case here:

We developed a new screen for managing orders in our eCommerce system which is to be used by external users who have less knowledge about the system and fewer permissions. The new new interface reuses it’s functionality from the existing, full-featured one but leaves out some more advanced or internal features. While those interfaces of course reuse the code internally and thus should both work, I wanted to have some WebDriver acceptance tests to ensure the components were properly integrated.

We had a set of WebDriver tests for the existing interface using a page object. A trivial approach would be to copy/paste the code for all the tests I wanted to pass with the new interface to a new test class, but I apply the same rigour to my tests as to production code, so that was out of the question (I see people do it all the time and it makes me sick).

So what I did instead was to first use inheritance to create two page objects sharing a common base class with the shared content and functionality and modelling the differences in the extending classes, providing both a clean API to write automated test cases as well as executable documentation on why there are two so simliar mangement interfaces and how they differ.

Note: We are using Geb to define page objets and Spock to run our tests. Thus I will use their syntax for the example code, but this concept can probably be applied with any test framework and custom page objects as well.

Then I split the existing test into two test classes so that all the test methods I wanted to pass against both interfaces would be in a common class, making it obvious for someone reading that test class that this was shared functionality. Finally I parameterized those shared tests on the page object to have my test runner create the two very similar test cases for each feature instead of explicitly copying it. Suppose you had the following spec:

All I needed to do was to add a where block for the parameterization and change the line that goes to the page like this:

Done – the tests are running against two different implementations. One problem that remains is that in spock is seems not to be possible to parameterize a whole test class. Therefore I extracted the page object classes to a reusable list…

…and added the where block to each test method. So, unfortunately this approach is not 100% code-duplication-free after all, but it is a single line of code that is duplicated and I could imagine that a future spock version allows to parameterize whole classes.

Let me know what you think about parameterizing on page objects in the comments or ping @c089 on twitter.

Posted in Uncategorized | Tagged , , , | 2 Comments

Node.js: Custom error classes without the pain

Dustin Senos wrote a good article on using Custom Errors in Node.js, however I thought this was a lot of boilerplate code to write each time I needed to add a new Error class, so I opted to generate my error classes dynamically like this:

var util = require('util')
, errors;

// A list of error classes to generate.
errors = [ 'MyFirstError', 'MyOtherError' ];

// Abstract Error
function AbstractError(msg, constr) {
  Error.captureStackTrace(this, constr || this);
  this.message = msg || 'Error';
util.inherits(AbstractError, Error);
AbstractError.prototype.name = 'Abstract Error';

// Generate error classes based on AbstractError
errors.forEach(function (errorName) {
  var errorFn = exports[errorName] = function (msg) {
    errorFn.super_.call(this, msg, this.constructor);
  util.inherits(errorFn, AbstractError);
  errorFn.prototype.name = errorName;

You can then just require the errors module and use any error class like this:

var error = require('./lib/error')
callback(new error.MyOtherError('async error!'), null);

Of course this does not allow for a hierarchy (like AbstractError -> IOError -> FileNotFoundError) but could probably be extended if the application grows big enough to require this. I’d be happy to hear about any improvements over Dustins approach, especially as his post has been around for a while and there might be new idioms on error handling, but right now this feels like a good way to handle errors in node apps.

Posted in Uncategorized | Tagged , , | 2 Comments

Painless migration of WebDriver tests to Geb

Okay, so your Selenium test suite has grown over the years, you migrated to selenium 2.x/WebDriver and you constantly evolved your set of helper methods to work around the complicated Java Api. Your tests do what you want them to, but they are pretty ugly. They have CSS and XPath selectors scattered all over them and duplicated a dozen times. There is no separation between describing your page and your actual test. Then you stumble across Geb and fall in love, but you are afraid it won’t play well with your Tests and would take days to integrate into your complicated build process and CI system. Fear not, it will play along just fine.

We have a large set of JUnit4 Tests written in Groovy that all derive from a single base test class. All we really had to do to geb-enable our tests was to throw in two jar files, make the base class extend GebReportingTest (well, and remove/deprecate code that’s no longer needed because it’s provided by geb). It did not break anything and we can now start adding new Tests using the Geb DSL or rewrite existing ones. Next step will be to write new Tests as Spock specifications instead – this will probably only require us to modify some wildcards in our ant build.xml and/or Jenkins configuration to run *Spec as well.

Posted in Uncategorized | Tagged , , , , , , , , | Leave a comment

Test262 JavaScript Test Suite

This looks interesting: Nearly 11.000 test cases to see how well Browsers implement the JavaScript specification – and they are open so you can verify the results yourself. I actually wonder how Opera get’s anything to work with only 65% of the tests passing…

Posted in Uncategorized | Leave a comment

HTTP for Humans

Things shouldn’t be this way. Not in Python.

 Documentation of the “Requests” module
Posted in Uncategorized | Tagged , , | Leave a comment

Duck(umentation) typing

Simple JavaScript Duckumentation generator.

JSDuck Readme

Posted in Uncategorized | Tagged , , , | Leave a comment

Testing Ext JS 4 with JsTestDriver


For my thesis, I’m writing a little application using ExtJS. The app itself is nothing special, but the thesis’ focus is on testing web applications – and in the awesome, agile and extreme programming world of 2011, this really means “testing JavaScript using the same proven methods and the same rigour as with testing our server side application”. However, this is not as easy as one might hope.

A quick overview of the java script testing frameworks

For once, there’s a shitload of unit test runners for JavaScript to choose from: At least every major JS framework has its own unit testing tools, plus there’s quite a few generic ones. And the reason everyone is writing their own is because they all suck. It’s pretty much like writing your own web framework a couple of years ago. All of these frameworks can be categorized into two categories by the approach to the problem they’ve chosen:

You can implement your test runner in JavaScript and make it run in a browser. So the developer will setup some HTML page, include your test runner, the code under test and the unit test files and the test runner will execute the tests and manipulate the DOM to show the results. An example of this approach can be seen on the JsUnit demo page. This is can be used with every browser you are targeting and can display fancy results right in your browser. There’s one tiny little problem though: This can’t be (easily) integrated into a continuous integration system because the in-browser-test-runner just can’t write the results to an XML file. Also, running your tests from the shell just looks so much cooler ;)

The alternative is using a javascript engine outside of the browser, like RhinoUnit does by leveraging Rhino, which emulates Mozillas JS engine. The problem with this is that you just can’t emulate every browser’s JavaScript engine in a tool like this and emulation is never perfect, so you might not find bugs that are caused by different implementations of JavaScript. So this is not a good solution either. Also, Rhino is reeeeeheeeeaheeeeeeally slow.

Someone at Google was clever enough to come up with a really simple and obvious better approach and developed JsTestDriver: It’s combining the two failing approaches by using a simple client/server setup to distribute tests to browsers and gather the results: Browsers are “captured” by visiting a page on the mini http server started by JSTD. That server can then send javascript code and test cases to the browsers, where they are executed and the results sent back. This not only allows to write the results to an XML file to be picked up by Jenkins or another CI system, it’s also really fast (despite being written in Java). And you can use it with every browser you want to support. Writing a unit test in JSTD is really simple (I won’t go into how to actually run this here, it’s pretty easy and covered on the Getting Startedpage):

TestCase("RomanNumbersTest", {
    "test 1 in roman": function () {
        assertEquals("I", to_roman(1));
    "test 2 in roman": function () {
        assertEquals("II", to_roman(2));

Also, while it includes all your usual xUnit style assertions, it is also compatible with some of the other frameworks such as Jasmine or QUnit. So,JSTD is the way to go. Now, let’s test our fancy Ext application, should be equally easy, right?

ExtJS 4 and dynamic class loading

Now, because Version 4 of the Ext JS framework is already on Beta3 and the final version should be out in a few weeks, I decided to take the risk and start using it for my thesis. The architecture of Ext is really impressive. They have a very clean API and an awesome documentation which makes this a joy to work with. One of the key features of Ext 4 is “dynamic class loading”: You can just instantiate objects using their name and it will load the appropriate JavaScript file if it’s not available already. For example, when you have a file /myapp/controller/FooController.js, which defines a class FooController, then you can access this in ExtJs using “myapp.controller.FooController” just as you would in Python or Java and the framework will know where to load it from (it’s plain old convention over configuration really). They are also working on a command line tool which generates a minified, single-file version of all your files for deployment, but unfortunately that’s not available yet. So, all you need to do is include the ext-all.js and your main application.js files in your html page and you’re done.

Testing ExtJs 4 applications using JS-Test-Driver

Now, with JSTD this causes problems: I don’t know how exactly JSTD sends the javascript files to the test browsers and executes them, but I had to put in a lot of work to get it running without causing incomprehensible errors. There’s two key factors I identified: First, your JS files will be served under the “/test/” base directory of the JSTD web server which is probably not the same behaviour as your development and production environments. For this reason, you have to adjust the “appFolder” setting of the Application configuration object, which is as simple as using sed to modify the file (see below for my script).

The other thing is that it was not enough just to include ext-all.js and Application.js – the dynamic loading does something (I can see the GET requests), but it just won’t do the trick for some reason I could not figure out. May be a bug in Ext, may be something JSTD does that browsers don’t, may be me being stupid. So, this is a jstd.conf for a standard Ext JS 4 MVC directory layout looks something like this:

- js/extjs/ext-all-debug.js
- js/timetable/Application.js
- js/timetable/controller/*.js
- js/timetable/model/*.js
- js/timetable/store/*.js
- js/timetable/view/*.js

- unit/*.js
- unit/controller/*.js
- unit/model/*.js
- unit/store/*.js
- unit/view/*.js

So with all of this sorted out, you can now test any ExtJS class you want to, for example you could unit test a validator on a model like this:

"test name must be longer than 4 characters": function () {
    var e = Ext.ModelManager.create({
    }, 'myApp.model.myModel');

Wrapping up: CI and Conclusion

On the JSTD wiki, I found a nice script to run the JSTD server, open the capute page in a browser and start the tests to be used in a continuous integration system using a virtual framebuffer, which I used for my own project. I added some stuff to run all of this on my development computer, because to really get the integration of JSTD, ExtJS and my directory structure to work I had to navigate around a few other issues. It really is a dirty hack involving creating a temporary directory, copying all my files over and using sed to make the needed change to my Application.js, but for no, this satisfies my needs. You can get a copy of my script in case you’re having similar trouble. Also, feel free to contact me if I can help out or you just want to chat about test-driven web development, which will be all I’m doing for the next three months anyway.

Posted in Uncategorized | 2 Comments