Introduction
So, Quality Assurance (QA), yawn! Security, double yawn! I'm as guilty as any other developer who wants to work on something cool in that I occasionally sacrifice these in order to 'get the job done'. But when is done really done? What can be the effect of skipping QA from a project or even a class within a project? How can we make QA and security easier so that it just happens?
SOA as an Asset
As we move towards a service oriented architecture world, our services are acting more like assets for our business. There can be many consumers of our services both large and small and they may or may not have service level agreements (SLAs) based on the consumption of a service. Therefore, our service is an asset and has value that we must protect.
Excuses
There are many excuses for poor software; Selip suggests the most popular are complex environments, cultural bias, waterfall methods and missing in-house expertise. Cultural bias is one of the more interesting ones as Selip (p7) raises the point that "there is usually no reward or penalty structure related to software quality". Personally, I'm not one for a penalty structure but I'm all for a metric based performance reward either through peer recognition (and a little something extra of course) rather then hidden personal goal/development documents.
Three Facets to Secure Software
Lipner and Howard (p2) discuss what they see as the three facets of secure software: repeatable process, engineering education and metrics and accountability.
Metrics are most impressive and responsive in a continuous integration environment running against the code and not against a document. Static analysis and security based tools should be included in a build as well as using code reviews, threat modeling and paired programming to eliminate issues.
Selip (p8) mentions that all the engineers at Microsoft attend an annual training program in the Security Development Lifecycle (SDL). Engineer education is equally important to running these tools in a continuous integration environment. If the engineers are more informed, they are likely to make better decisions and improve the quality of a project.
Commonly Missed Types of Testing
Continuous integration is great for unit and integration tests, but these tests only go so far. It can't test how easy a system is to use or whether the system is backwards compatible. Projects need to be more conscious of functional and user acceptance testing where the program is handed out for feedback. Projects also need to be conscious of strategies to perform regression testing so breaking changes can be tracked.
When is Beta, Alpha?
Everything these days seems to have a beta label slapped on it, but Selip defines the difference between people bases testing techniques (p20). Alpha testing is performed by those friendly to the development team while beta testing is performed by those in a similar demographic to that of the target audience. So unless you have people you don't know testing your software, you're probably in alpha.
Agile Approach to QA
Depending on their structure, stories generally relate to a piece of functionality. If stories are expressed with Volere Shells, the QA can be easier defined as these are written more with a specific test and priority in mind. These stories relate to a set of unit and integration tests that prove the story and feedback into the tools used within the CI environment to prove the QA.
Conclusions
I think Selip (p8) says it best so I'll leave it to him.
"Believe that all software, whether built in-house, purchased as COTS, or developed by outsourcing, is the work of fallible humans who are prone to making mistakes. Understand that the organization and its stakeholders are highly dependent on high-quality, fault-free software. Expect that requirements, design, and software development mistakes will result in risky software faults. Invest in people, processes, and tools to avoid, detect, and neutralize software faults. Mandate and create incentives for a culture that takes pride in quality, has the skills and knowledge to achieve it, and that aggressively finds and fixes faults."
Lipner & Howard (p11) also suggest that project teams should concentrate on threat modeling, code reviews, using automated tools and fuzz testing throughout the lifecycle. This should be done as a higher priority then penetration testing which is used to define production readiness rather then used to find bugs.
References
Selip, S. (2005), To Err Is Human, So Test That Software,
Lipner, S & Howard, M (2005), The Trustworthy Computing Security Development Lifecycle, MSDN, http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnsecure/html/sdl.asp
No comments:
Post a Comment