Test driven development (TDD) and web services are becoming new standards in the industry, but unit testing against a web service raises issues especially if a hit on the service has a financial or performance cost. During development it may be beneficial to mock the behaviour of a live service.
In the project I am working in at the moment we have had to ask ourselves these questions and we have to continue to ask ourselves how to get the best out of unit testing against a web service. Having been recently introduced to the principle of 'inversion of control', I realised our approach was not that far off of these principles although I questioned whether we still had the right approach.
Issues with building a stub web service
The approach we had taken was too generate web service classes using ThinkTectures' WSCF tool (www.thinktecture.com/WSCF/ ) and ran these web services within local host using fixed ports (on our development and build machines). These generated web methods return a NotImplemetedException by default, so we had to fill these methods out to support our unit tests. This is an architecture discussion in itself: complete these classes; extend them or use helper classes. This discussion arose when the services we were building against were changing on a weekly basis. Completing the classes' meant that every time new classes were generated from the WSDL contract, the code was overwritten which would result in a mass copy and paste job.
It was really a question of extending the web methods or using helper classes. Our thought path was as the web services we were developing were fairly volatile in nature any change could result in a large refactoring task to make sure our mock web service was in step with the live web service.
How stupid should a mock web service be?
Our project had started small; in fact the behaviour from the beginning was more like a proof of concept or an architectural spike then anything else. A small team began building web services and also building a toolkit to consume these services. A four person team had complete control over the project and the architectural awareness may have resulted in overcomplicating our stubbed web services later on.
After the proof of concept, the team continued to build the toolkit to expose the services that new teams taken over to develop. Perhaps the team had too much knowledge of the services they were consuming as the mock web services were considerably complex considering they were to be used for testing only.
A discussion on dependency injection got me thinking that our stubs were too complicated. We should be mocking the behaviour of the live services at the contract/interface level without having to think too hard about how the service is implemented.
These thoughts lead to an increase in a practice that the team had already been practising too a small extend; passing known values in parameters. For this too work, the value would have to be known by the test object and the mock web service. If a certain value was passed the web service would behave in a certain way (throwing an exception for example).
This worked for a time and proved somewhat of a success, especially as debugging a running stubbed web service was a painful exercise; creating a stupid web service achieved the desired behaviour and we could test our toolkit sufficiently without investing too much time in developing a mock framework.
As the services increased in complexity and required data from the SOAP header, this simple passing of a known value was not going to be enough to achieve the desired result. After some discussion the decision was made to pass delimited values across in strings. This way, the service would not have to have the same complexity could behave in the same way by using the delimited data as the live service would if that data was in the SOAP header.
Should unit testing include web service calls?
This leads into a question of should a unit test go as far as actually making a web service call in the first place? Our toolkit contains a logic layer which talks to web service proxy objects and we had a configuration that would create that proxy object with a pointer to the live service, or the mock service on the local machine if the configuration was set to 'test mode'.
We could take this one step further in an attempt to improve our build time. When generating a web proxy class, if our configuration was such, we could instead build a object that contained the stupid mocked behaviour in a known assembly. This would almost certainly cut the build time as the code would not have to go through the web service stack several hundred times to complete the test list.
Using this technique means there could be an inherent risk that the web service stack is not being tested at all. This is bad for code coverage, but if the project also had live integration testing this could be seen as an acceptable risk to take within a unit test.
For this to work, the mock objects would have to be maintained as the live services evolve, although this also holds true for the mock web service.
Sharing test data with a mock web service
There is another advantage to not using a web service stack in the unit testing and that is the ability to share test data. When running unit tests with a mock web service, the code is running in a different assembly, and hence has trouble sharing resource as the only way to communicate is through the web service. Sharing a database for test data is one possible answer to that, but its quite a significant hit to use a database if the project does not use a database like our toolkit.
A file on the same machine or even a key/value paired web service could also work but this comes with its own maintenance issues. Sharing a file seems easy enough, but both assemblies would have to know where that file is, and what if someone comes along and checks out the code from source control it should work straight away without configuring. Will that file always be in the same place; probably not. As for setting up a key/value pair web service; well you might as well set up a database.
If the mock object is running in the same (or referenced) assembly, it can have access to the same data resources as the tests and then that data need only be maintained in one place. This could also open up more dependency injection opportunities for the mock web service where any behaviour could be set during run time.
