January 29th, 2007
12:07 am
General
It’s common practise within a team to define a set of tasks which should be run by each developer prior to checking in. The purpose of the Check-in Gate is to attempt to ensure that all code satisfies some basic level of quality.
Like all development standards, a check-in gate helps a team stay on the same page. It eases integration, and can help give confidence as to the quality of the code checked in.
Check-in Gate with Continuous Integration
A check-in gate is typically used prior to checking in to a system which uses Continuous Integration – where the tasks in run during the check-in gate are the same tasks used as part of the continuous integration build. The main source of embarrassment associated with breaking a CI build tends to come down to the fact that the shamed individual completely forgot to run their check-in gate build.
The importance of speed
Check-in Gates need to be fast. The longer they take to run, the less developers will want to run them – this either results in less frequent check-ins or in developers not running them at all. Fewer check-ins result in more complex (and more error prone) integrations. Not running the check-in gate at all can result in a breakdown of code quality and can be a slippery slope to the gate being abandoned altogether.
Examples
The simplest example of a check-in gate would probably be ensuring that the code compiles prior to check-in. More often, the team will decide to run either some or all of a test suite. Again the constraining factor as to what you’ll want to run as part of a check-in gate is typically time – deciding how and what to run should always be defined by the team.
Selenium is a very good in-browser testing tool. It has bindings for many different languages (including Java, Python, Perl, C! and PHP). With it you can create a suite of tests which can run tests in multiple browsers (including IE, Opera, Firefox and Safari).
The ability to run tests inside a browser is a huge boon to those of us who have to worry about cross-browser compatible websites. By having an automated test suite (and having it run regularly, perhaps using Continuous Integration) you can automatically run a set of tests repeatedly on a number of platforms, on a number of browsers, whenever code changes. Whilst this doesn’t do away with the need for normal exploratory testing, nor is it always possible (or sensible) to automate all tests, this can dramatically reduce the QA time required prior to a go live.
Selenium is a very good tool, designed to be much simpler to use than similar tools (such as Mercury’s products). It is a very good tool that you probably don’t need all that much.
Selenium Slowdown
Selenium’s strength – that it tests web applications in the browser – is also it’s weakness. Testing in browsers is slow. Not only do you have the overhead of starting and marshaling an external process (the browser) but the fact that the tests have be rendered on screen means that a sizable Selenium test suite can take an awfully long time to run.
There are techniques which can be used to handle long running tests suites (more later perhaps) but I suspect for most of you, you don’t need to worry about them
Testing the DOM
Think about what it is you want to test in your web application. You need to simulate some user activity (clicking a link, entering text) and test that some result is displayed to the user. Selenium is as good as most things out there at doing that – but as we’ve already said, it’s slow. What is the alternative?
Figure 1: An overview of a browser driver
Well what is it we are really testing here? Let’s start with the user input. For the most part (I’m excluding AJAX interaction here – more later), when a user interacts with a web page, they end up creating a HTTP request to the server. Your server acts on that request, and returns some HTML, which the browser converts into a Document Object Model, and which in turn gets rendered to the user.
So when we want to check what is displayed to the user, what is it we are actually doing? Our testing tools don’t look at the screen rendering – all they need to do is carry out assertions on the DOM itself.
So to test most web applications, we need to create a HTTP request, and perform assertions on the DOM. And Selenium certainly isn’t the fastest way of doing that.
Faking the browser
The reason that Selenium is slow, is the browser. We are using Selenium to drive the browser, which in turn submits a request for us. The browser then handles the response, creates (or manipulates) the DOM, and renders the response. Why not simply remove the browser altogether?
Tools like HTTPUnit (for Java) or Twill (for Python) let you do just that. With them, you can create a request, submit it directly to the server, handle the response and interrogate the DOM. HTTPUnit and Twill are effectively emulating the browser’s ability to create a DOM from a server response.
Figure 2: An overview of a browser-emulation testing
Test suites using browser emulation tools like this will be an order of magnitude faster than similar Selenium test suites.
No place for Selenium?
There is certainly a place for in-browser testing. In our overview of browser testing above, we implied that the DOM for any given page is created entirely as a result of a response from the server, but the world isn’t that simple.
Using Javascript, web developers have for a long time been able to manipulate the DOM by executing on the client side with no interaction with the server. Selenium (and similar tools) are still very useful for testing these kinds of situations – however for most of us there will be much less need for these (slower) tests.
Conclusion
The tools available for browser testing have come on leaps and bounds in recent years. There is a place for browser drivers (like Selenium or Sahi) and for suites based on browser emulation techniques (such as HTTPUnit or Twill). Knowing which to use and when can result in significant time savings when running your test suites.
And no apologies:
- Rice Pudding & Red Wine
- The books of Terry Pratchett
- Cheesecake
- The Films of Adam Sandler
- The film Hackers
Yours?
December 18th, 2006
12:04 am
General
The MacBook Pro has arrived, and given the expense of purchasing it I’m starting to get worried about taking it out of the house. AppleCare for it will be arriving as soon as I’ve paid for christmas, but prior to taking it back home for the holidays, I’ve been considering getting cover for it.
Options right now are either to try and get extended all risks cover for it on the contents insurance, or go for a specialist. The helpful MoneySavingExpert forums turned up mentions for Compucover, and google turns up many more – but I wondered if readers had any advice? I’m resident in the UK but do plan to travel with it.
I’d lagged behind the latest Wordpress patches so finally bit the bullet and upgraded today. The “unzip the new files over the old” barfed for me, so I went for the slightly more complex “delete the old files first” approach, and all was well. I also ditched my (badly) tweaked Hemmingway theme for the current Web 2.0 one. Expect it to remain fully functional for as long as it takes me to start editing the theme… As always, feel free to report any glitches.
I had an enjoyable couple of days at XPDAY 2006 in London earlier this week. Stand-out presentations for me were Joe and Dan’s Awesome_Acceptance_Testing and Chris Matt’s Managing Uncertainty & Risk Using Real Options, more of which later.
Ivan’s Are We Nearly There Yet? has the makings of an interesting presentation, however I think he was slightly knocked off track by the larger than expected attendance. I personally found the discussion around using actual days for iteration-level estimation warranted the whole session.
Keynote Controversy
The second day’s keynote, Love in the Age of Software by James Noble and Robert Biddle was by degrees entertaining, annoying, embarrassing, enjoyable but not quite educational enough. It was a shame to see some people leave during it (which could be down to either the previous night’s free drinks or the unconventional presenting style) – all that did was remind me that many people in our industry are actually far more conservative than we think.
dbdeploy
Theoretically mine and Graham’s dpdeploy presentation was the official launch of the database refactoring tool. The talk went well enough I think, but I think some much of audience were looking for a silver bullet that just doesn’t exist. dbdeploy is nothing more than the latest in a long line of process change hiding behind a tool (CruiseControl being an excellent example).
Anyway, the dpdeploy website is up and the documentation is being improved all the time.
October 31st, 2006
11:42 pm
General
Once upon a time, the Internets was about two things:
Then, things changed, and the Internets became about these things:
Then blogging happened, and things changed again. The Internets was transformed into a media concerned with:
- Kittens
- Porn
- Gambling
- People talking about their macs
Yes, for those still listening, I’m awaiting the delivery of a shiny new MacBook Pro.
As Steven D Levitt and Stephen J Dubmner’s book Freakonomics explains, Economics can be considered as nothing more than the study of how incentives affect people. Broadly split into three types – Financial (What’s it worth to me?), Moral (Is it the right thing to do?) and Social (Will society act in a bad way if I do this thing?) by understanding the incentives available to someone, you can understand how they will act in a given circumstance.
Taken further, Econmics allows you to alter the available incentives to alter how people act. Want people to reduce their carbon output? Well, you could explain how they’d save money by reducing their energy use (financial incentive). You could also create a groundswell of opinion stating that society views excessive energy use badly (social incentive, which could ultimately become a moral incentive).
A good example of changing incentives to change behaviour could be seen on BBC Breakfast this morning. Since 1998, using a mobile phone whilst driving has been illegal in Jersey. The simple act of making something illegal attempts to act as a moral incentive – with the punishment of a £500 fine acting as a financial incentive. On it’s own, this wasn’t enough. There wasn’t enough stigma associated with the act of using a mobile phone whilst driving for it to stop altogether. In the end it took a campaign, started by Paul Newman (no relation) and backed by the local Jersey Evening Post and more surprisingly mobile phone company Jersey Telecom. The resulting Hands Off campaign is attempting to create the social incentive – using a mobile phone when driving is unacceptable to society.
I spotted some side by side photo’s of Apple’s latest 80GB iPod, and Microsoft’s Zune MP3 player.
For me, the decision about which one I’d rather have comes down to this – would I rather have a 5th generation product from Apple, or a 1st generation product from Microsoft?
September 21st, 2006
10:36 am
General
First came the worlds first most modernist computing conference, PoMoPro, then came this great bit of post modern spam:
Are you looking for bulk email software to promote your online
business and increase your profits?
Seeking ways to promote your web presence?
Do you need bulk emailing software to send the ads?
If so, our e-mail marketing utilities can help you reach your goals.
Let’s ignore the fact that I often blog about blogging, shall we?
