XPDay Review

On Thursday and Friday of last week I showed up at XPDay, and attended some cool sessions including:

Nat Pryce on TDD of async code: http://www.natpryce.com/articles/000755.html

Matt Wynne on Lean Engineering: http://blog.mattwynne.net/2008/12/14/slides-from-xp-day-talk/

Had some interesting discussions with some cool people including:

http://gojko.net/tag/xpday08/

So, I was interested in Gojko Anzic's open space session on the new Fitnesse SLIM implemenation. It looks like it's much, much easier to wrap domain objects instead of creating fixture classes for testing. It's not quite ready yet, but I'm looking forward to a stable release.

There was also an interesting keynote from Marc Baker of LeanUK.org (http://www.leanuk.org/pages/about_team_marc_baker.htm) on his experiences of introducing lean principles into the NHS.

Test and Test Again

There has been some discussion of the application of agile principles including test driven development in the office recently. Having been working on a highly concurrent, scalable SIP application server my feet are firmly in the test first, test often camp.

Our testing strategy includes:

* unit testing, usually using mocks and dependency injection
* acceptance testing, using FitNesse
* robustness testing, using custom scenario handlers
* performance testing

These practices are dependent on the type of project, but we've found that although unit testing is done before the functional code is written, the other tests are usually written afterwards. I like to think that acceptance tests demonstrate completed user stories and robustness tests exercise common scenarios, concurrently.

A robust application comes with time, but through concurrent testing, issues can be drawn out earlier and the confidence you have in your application grows.

Software engineers need love too.

It's amazing how a new environment, a ton of pressure and a bunch of uncertainty can make people react. I found out earlier this week how software engineers react. At a three day off site, a large group of people, both developers and 'suits', were told to leave their job titles at the door. Unfortunately for me and the team I was working with, it felt as if we'd left our software engineering principles at the door too.

Having essentially 48 hours to build a product, mostly from scratch, or at least using new products/APIs and services to plug something together seemed to suggest that we threw our best practices out of the window. Now the working environment wasn't set up exactly how we would have liked it, but we could have done something about that. Without even the basic necessity of source control, it felt as if the team were fire fighting from an early stage with USB drives being swapped around at crucial times, along with cries of "hey, can you add this method to the code?" when it could have been done easily and checked in.

I certainly felt like I was developing with one hand without using test driven development, source control, continuous integration et al. If three days being thrown into the deep end has taught me anything, its to be prepared.

I once heard the phrase, "a bad tradesman blames his tools", but conversely, does that mean that a tradesman is only as good as the tools he uses? Perhaps I've grown so used to the tools that I use that I've come to depend on them, and while that encourages best behaviour, does that make me inflexible?

The Trouble of Unit Testing the Consumption of Web Services

Test driven development (TDD) and web services are becoming new standards in the industry, but unit testing against a web service raises issues especially if a hit on the service has a financial or performance cost. During development it may be beneficial to mock the behaviour of a live service.

In the project I am working in at the moment we have had to ask ourselves these questions and we have to continue to ask ourselves how to get the best out of unit testing against a web service. Having been recently introduced to the principle of 'inversion of control', I realised our approach was not that far off of these principles although I questioned whether we still had the right approach.

Issues with building a stub web service

The approach we had taken was too generate web service classes using ThinkTectures' WSCF tool (www.thinktecture.com/WSCF/ ) and ran these web services within local host using fixed ports (on our development and build machines). These generated web methods return a NotImplemetedException by default, so we had to fill these methods out to support our unit tests. This is an architecture discussion in itself: complete these classes; extend them or use helper classes. This discussion arose when the services we were building against were changing on a weekly basis. Completing the classes' meant that every time new classes were generated from the WSDL contract, the code was overwritten which would result in a mass copy and paste job.

It was really a question of extending the web methods or using helper classes. Our thought path was as the web services we were developing were fairly volatile in nature any change could result in a large refactoring task to make sure our mock web service was in step with the live web service.

How stupid should a mock web service be?

Our project had started small; in fact the behaviour from the beginning was more like a proof of concept or an architectural spike then anything else. A small team began building web services and also building a toolkit to consume these services. A four person team had complete control over the project and the architectural awareness may have resulted in overcomplicating our stubbed web services later on.

After the proof of concept, the team continued to build the toolkit to expose the services that new teams taken over to develop. Perhaps the team had too much knowledge of the services they were consuming as the mock web services were considerably complex considering they were to be used for testing only.

A discussion on dependency injection got me thinking that our stubs were too complicated. We should be mocking the behaviour of the live services at the contract/interface level without having to think too hard about how the service is implemented.

These thoughts lead to an increase in a practice that the team had already been practising too a small extend; passing known values in parameters. For this too work, the value would have to be known by the test object and the mock web service. If a certain value was passed the web service would behave in a certain way (throwing an exception for example).

This worked for a time and proved somewhat of a success, especially as debugging a running stubbed web service was a painful exercise; creating a stupid web service achieved the desired behaviour and we could test our toolkit sufficiently without investing too much time in developing a mock framework.

As the services increased in complexity and required data from the SOAP header, this simple passing of a known value was not going to be enough to achieve the desired result. After some discussion the decision was made to pass delimited values across in strings. This way, the service would not have to have the same complexity could behave in the same way by using the delimited data as the live service would if that data was in the SOAP header.

Should unit testing include web service calls?

This leads into a question of should a unit test go as far as actually making a web service call in the first place? Our toolkit contains a logic layer which talks to web service proxy objects and we had a configuration that would create that proxy object with a pointer to the live service, or the mock service on the local machine if the configuration was set to 'test mode'.

We could take this one step further in an attempt to improve our build time. When generating a web proxy class, if our configuration was such, we could instead build a object that contained the stupid mocked behaviour in a known assembly. This would almost certainly cut the build time as the code would not have to go through the web service stack several hundred times to complete the test list.

Using this technique means there could be an inherent risk that the web service stack is not being tested at all. This is bad for code coverage, but if the project also had live integration testing this could be seen as an acceptable risk to take within a unit test.

For this to work, the mock objects would have to be maintained as the live services evolve, although this also holds true for the mock web service.

Sharing test data with a mock web service

There is another advantage to not using a web service stack in the unit testing and that is the ability to share test data. When running unit tests with a mock web service, the code is running in a different assembly, and hence has trouble sharing resource as the only way to communicate is through the web service. Sharing a database for test data is one possible answer to that, but its quite a significant hit to use a database if the project does not use a database like our toolkit.

A file on the same machine or even a key/value paired web service could also work but this comes with its own maintenance issues. Sharing a file seems easy enough, but both assemblies would have to know where that file is, and what if someone comes along and checks out the code from source control it should work straight away without configuring. Will that file always be in the same place; probably not. As for setting up a key/value pair web service; well you might as well set up a database.

If the mock object is running in the same (or referenced) assembly, it can have access to the same data resources as the tests and then that data need only be maintained in one place. This could also open up more dependency injection opportunities for the mock web service where any behaviour could be set during run time.