Comments on: Enforcing test coverage in the build http://www.magpiebrain.com/blog/2005/01/06/enforcing-test-coverage-in-the-build/ Sam Newman's blog Sat, 05 Jul 2008 00:35:49 +0000 http://wordpress.org/?v=2.5.1 By: Sam Newman http://www.magpiebrain.com/blog/2005/01/06/enforcing-test-coverage-in-the-build/#comment-613 Sam Newman Fri, 07 Jan 2005 09:45:58 +0000 http://www.magpiebrain.com/2005/01/06/enforcing-test-coverage-in-the-build/#comment-613 Agreed. Attempting to enforce an ideal in an automated fashion which no-one agrees with is never going to be easy, practical or sensible. If you take a look at Howard's presentation, his "pyramid of quality" shows different practises that can be used, each one being dependent on the one before. For example you can't sell test coverage as being important unless your team likes tests. I too would be dubious about 100% test coverage across anything other than a trivial code base. What I don't have is any real notion of what a reasonable expectation to set in terms of coverage - in many ways introducing this practise on an existing codebase would be much easier, as you already have a base level to aim for. Agreed. Attempting to enforce an ideal in an automated fashion which no-one agrees with is never going to be easy, practical or sensible. If you take a look at Howard’s presentation, his “pyramid of quality” shows different practises that can be used, each one being dependent on the one before. For example you can’t sell test coverage as being important unless your team likes tests.

I too would be dubious about 100% test coverage across anything other than a trivial code base. What I don’t have is any real notion of what a reasonable expectation to set in terms of coverage – in many ways introducing this practise on an existing codebase would be much easier, as you already have a base level to aim for.

]]> By: Robert Watkins http://www.magpiebrain.com/blog/2005/01/06/enforcing-test-coverage-in-the-build/#comment-612 Robert Watkins Thu, 06 Jan 2005 21:07:51 +0000 http://www.magpiebrain.com/2005/01/06/enforcing-test-coverage-in-the-build/#comment-612 You do have to be somewhat careful with this idea. In a team that doesn't really buy into the test coverage concept, you find (well, _I_ have found) that people will write silly little tests to give 100% coverage. Also, people seem to _stop_ writing tests when they get 100% coverage, even though a lot of the possible interactions haven't been explorered yet. I find it useful to set a threshold level, and then examine the fluctations in coverage levels over time (something the clover history makes very easy). If the threshold level is 100%, then there is no fluctuation; this implies to me that I would be losing some information. With a team that is committed to writing meaningful tests, and with the maturity to do so, this becomes less of an issue, of course. You do have to be somewhat careful with this idea. In a team that doesn’t really buy into the test coverage concept, you find (well, I have found) that people will write silly little tests to give 100% coverage. Also, people seem to stop writing tests when they get 100% coverage, even though a lot of the possible interactions haven’t been explorered yet.

I find it useful to set a threshold level, and then examine the fluctations in coverage levels over time (something the clover history makes very easy). If the threshold level is 100%, then there is no fluctuation; this implies to me that I would be losing some information.

With a team that is committed to writing meaningful tests, and with the maturity to do so, this becomes less of an issue, of course.

]]> By: Sam Newman http://www.magpiebrain.com/blog/2005/01/06/enforcing-test-coverage-in-the-build/#comment-611 Sam Newman Thu, 06 Jan 2005 16:29:41 +0000 http://www.magpiebrain.com/2005/01/06/enforcing-test-coverage-in-the-build/#comment-611 Yeah - a nice idea but from what I've heard is of little practical use. I know the guy who wrote it, and he doesn't even use it :-) Yeah – a nice idea but from what I’ve heard is of little practical use. I know the guy who wrote it, and he doesn’t even use it :-)

]]> By: Simon Brunning http://www.magpiebrain.com/blog/2005/01/06/enforcing-test-coverage-in-the-build/#comment-610 Simon Brunning Thu, 06 Jan 2005 16:26:27 +0000 http://www.magpiebrain.com/2005/01/06/enforcing-test-coverage-in-the-build/#comment-610 Have you seen Jester[1]? It finds out if your code is actually being tested by *mutating* it. If a piece of code can change without causing your tests to fail, then clearly it isn't being tested. Never had a go myself, and I imagine it must take *ages* - an occasional thing, not part of your standard build - but it looks cool. [1] "http://jester.sourceforge.net/":http://jester.sourceforge.net/ Have you seen Jester1? It finds out if your code is actually being tested by mutating it. If a piece of code can change without causing your tests to fail, then clearly it isn’t being tested.

Never had a go myself, and I imagine it must take ages – an occasional thing, not part of your standard build – but it looks cool.

[1] http://jester.sourceforge.net/

]]> By: Howard van Rooijen http://www.magpiebrain.com/blog/2005/01/06/enforcing-test-coverage-in-the-build/#comment-609 Howard van Rooijen Thu, 06 Jan 2005 16:15:53 +0000 http://www.magpiebrain.com/2005/01/06/enforcing-test-coverage-in-the-build/#comment-609 Code Coverage (in the .NET arena) was one of the engineering practices I covered in my Microsoft Architects Forum presentation -"The Value of Agile Methods" for the full slide deck and notes take a look at: http://blogs.conchango.com/howardvanrooijen/archive/2004/12/07/397.aspx Code Coverage (in the .NET arena) was one of the engineering practices I covered in my Microsoft Architects Forum presentation -”The Value of Agile Methods”

for the full slide deck and notes take a look at:
http://blogs.conchango.com/howardvanrooijen/archive/2004/12/07/397.aspx

]]>