magpiebrain

Selenium rocks - and you don’t need it

Selenium is a very good in-browser testing tool. It has bindings for many different languages (including Java, Python, Perl, C! and PHP). With it you can create a suite of tests which can run tests in multiple browsers (including IE, Opera, Firefox and Safari).

The ability to run tests inside a browser is a huge boon to those of us who have to worry about cross-browser compatible websites. By having an automated test suite (and having it run regularly, perhaps using Continuous Integration) you can automatically run a set of tests repeatedly on a number of platforms, on a number of browsers, whenever code changes. Whilst this doesn’t do away with the need for normal exploratory testing, nor is it always possible (or sensible) to automate all tests, this can dramatically reduce the QA time required prior to a go live.

Selenium is a very good tool, designed to be much simpler to use than similar tools (such as Mercury’s products). It is a very good tool that you probably don’t need all that much.

Selenium Slowdown

Selenium’s strength – that it tests web applications in the browser – is also it’s weakness. Testing in browsers is slow. Not only do you have the overhead of starting and marshaling an external process (the browser) but the fact that the tests have be rendered on screen means that a sizable Selenium test suite can take an awfully long time to run.

There are techniques which can be used to handle long running tests suites (more later perhaps) but I suspect for most of you, you don’t need to worry about them

Testing the DOM

Think about what it is you want to test in your web application. You need to simulate some user activity (clicking a link, entering text) and test that some result is displayed to the user. Selenium is as good as most things out there at doing that – but as we’ve already said, it’s slow. What is the alternative?

Figure 1: An overview of a browser driver

Well what is it we are really testing here? Let’s start with the user input. For the most part (I’m excluding AJAX interaction here – more later), when a user interacts with a web page, they end up creating a HTTP request to the server. Your server acts on that request, and returns some HTML, which the browser converts into a Document Object Model, and which in turn gets rendered to the user.

So when we want to check what is displayed to the user, what is it we are actually doing? Our testing tools don’t look at the screen rendering – all they need to do is carry out assertions on the DOM itself.

So to test most web applications, we need to create a HTTP request, and perform assertions on the DOM. And Selenium certainly isn’t the fastest way of doing that.

Faking the browser

The reason that Selenium is slow, is the browser. We are using Selenium to drive the browser, which in turn submits a request for us. The browser then handles the response, creates (or manipulates) the DOM, and renders the response. Why not simply remove the browser altogether?

Tools like HTTPUnit (for Java) or Twill (for Python) let you do just that. With them, you can create a request, submit it directly to the server, handle the response and interrogate the DOM. HTTPUnit and Twill are effectively emulating the browser’s ability to create a DOM from a server response.

Figure 2: An overview of a browser-emulation testing

Test suites using browser emulation tools like this will be an order of magnitude faster than similar Selenium test suites.

No place for Selenium?

There is certainly a place for in-browser testing. In our overview of browser testing above, we implied that the DOM for any given page is created entirely as a result of a response from the server, but the world isn’t that simple.

Using Javascript, web developers have for a long time been able to manipulate the DOM by executing on the client side with no interaction with the server. Selenium (and similar tools) are still very useful for testing these kinds of situations – however for most of us there will be much less need for these (slower) tests.

Conclusion

The tools available for browser testing have come on leaps and bounds in recent years. There is a place for browser drivers (like Selenium or Sahi) and for suites based on browser emulation techniques (such as HTTPUnit or Twill). Knowing which to use and when can result in significant time savings when running your test suites.

This entry was posted on Sunday, January 28th, 2007 at 7:38 pm and is filed under General. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site. It is tagged as testing, twill, python, java, selenium, slow, agile over at Technorati