Agile Methods, Team Culture and the Bigger Picture

There seems to be a lot of bad air about agile methods at the moment and a lot of gushing over Google’s working practices. From what I’ve read working at Google sounds great, but for better or worse, most of us working a customer driven industry which won’t accept development and management styles like those employed at Google.

After contemplating on this for a while, I was going to write how I felt it was Google’s culture which affected it’s ways of working and hence following a method probably doesn’t make sense for them. I then thought about how practices from companies such as Google along with agile methods such as Scrum could create a high performing software engineering culture. My thoughts diverged into a group of practices which when used together could influence the culture of a group toward a high performing software engineering team, regardless of the method they apply.

Perpetual Learning

Software engineering and agile both promotes knowledge sharing. Pair programming, especially between different levels of experienced developers can improve the knowledge of those developers. Not to mention the other benefits of pair programming with more thought out code with hopefully better design, better tests and better code.

Within our group, every developer takes turns to present a tool, a technology or host a discussion once a week. This is a great way to transfer knowledge within a group, especially if these sessions are captured and archived for future members of the team or for reference. These sessions have been given the name ‘Brown Bags’ after originally being brought from another group doing the same thing, but over lunch.

Every quarter, Google projects show themselves off their peers. This can be a great way of exposing the great work, new tools and new techniques that have been developed over the quarter to the rest of the group. This could enhance the learning of the group as a whole as well as give teams something to strive towards for peer recognition.

Individual learning is also important. Google give their developers 1 day a week to work on their own projects or their own learning encouraging new projects to spin out of that work. 20% of a developer’s time is a lot to lose, but it should be encouraged to spend a certain amount of time on such endeavours. As well as this, in-between releases agile workers are encouraged to have a period of downtime to unwind, recharge the batteries, do some learning and work on some side projects. Hopefully this will feed back into the teams through new ideas during a project and through the weekly brown bags.

Shared Tools

I’m sure we’ve all wrote small applications to perform a specific task. If a group had a shared, searchable repository this could improve the productivity of the group. Or how about taking that one step further and applying a BSD (http://www.opensource.org/licenses/bsd-license.php) licence? Applying that to the code base and host it on a place such as Source Forge could help developers in the team and all over the world. If your company had it’s own share repository and hosted online this could enhance the company’s reputation in the engineering world.

Common Build, Common Code

Common builds and common code can help the engineering practice of being able to move onto different projects with minimal fuss. If all the code looked the same (as Google employees’ claims) that’s going to help when a team member moves team or a new team is formed. Pair programming can be a good way to converge onto a common code style.

A common build should most definitely include a continuous integration environment on a build box with metrics to fail the build such as code coverage and static analysis. Code should be developed in a test driven development (TDD) fashion to ensure only the function that needs to get written is written.

Prioritised Stack, Failing Fast and Team Communications

Google use queuing theories for their work stack; agile uses prioritised backlogs. These aren’t so far apart. Both are pulling mechanisms where the developer pulls the work off the stack that they want to complete. The only real difference is that a sprint is for a set time period with a prioritised stack so the customer can drive features and delivery.

Short sprints and regular releases also enable a team to ‘fail fast’. It may be feasible to set a time for a team to start a project and after a release it may be seen that a project isn’t working for whatever reason. Surely it’s better to realise this after one release and not after a few years of a project. It’s quite a simplistic view, but the sooner a project can fail, the quicker the business can cut the loses on that and move the teams onto projects which can provide value to the business. Architecture and development spikes are also great for this, setting a fixed time period to ‘play’ with a technique or tool and bring back that learning into the project.

Agile, and particularly Scrum uses daily stand up’s so a team member can communicate the work they are undertaking. This is a valuable time for the team to be able to find any issues early on. Some have said that a daily stand up is a triage with protocol. They may be right, but nether-the-less the team benefits from frequent communication from peers.

Conclusions

Every workplace has a culture, every team follows a methodology. But regardless of those, if practices such as those outlined above are followed a team should hopefully increase its value to the business.

QA in an Agile World

Introduction

So, Quality Assurance (QA), yawn! Security, double yawn! I'm as guilty as any other developer who wants to work on something cool in that I occasionally sacrifice these in order to 'get the job done'. But when is done really done? What can be the effect of skipping QA from a project or even a class within a project? How can we make QA and security easier so that it just happens?

SOA as an Asset

As we move towards a service oriented architecture world, our services are acting more like assets for our business. There can be many consumers of our services both large and small and they may or may not have service level agreements (SLAs) based on the consumption of a service. Therefore, our service is an asset and has value that we must protect.

Excuses

There are many excuses for poor software; Selip suggests the most popular are complex environments, cultural bias, waterfall methods and missing in-house expertise. Cultural bias is one of the more interesting ones as Selip (p7) raises the point that "there is usually no reward or penalty structure related to software quality". Personally, I'm not one for a penalty structure but I'm all for a metric based performance reward either through peer recognition (and a little something extra of course) rather then hidden personal goal/development documents.

Three Facets to Secure Software

Lipner and Howard (p2) discuss what they see as the three facets of secure software: repeatable process, engineering education and metrics and accountability.

Metrics are most impressive and responsive in a continuous integration environment running against the code and not against a document. Static analysis and security based tools should be included in a build as well as using code reviews, threat modeling and paired programming to eliminate issues.

Selip (p8) mentions that all the engineers at Microsoft attend an annual training program in the Security Development Lifecycle (SDL). Engineer education is equally important to running these tools in a continuous integration environment. If the engineers are more informed, they are likely to make better decisions and improve the quality of a project.

Commonly Missed Types of Testing

Continuous integration is great for unit and integration tests, but these tests only go so far. It can't test how easy a system is to use or whether the system is backwards compatible. Projects need to be more conscious of functional and user acceptance testing where the program is handed out for feedback. Projects also need to be conscious of strategies to perform regression testing so breaking changes can be tracked.

When is Beta, Alpha?

Everything these days seems to have a beta label slapped on it, but Selip defines the difference between people bases testing techniques (p20). Alpha testing is performed by those friendly to the development team while beta testing is performed by those in a similar demographic to that of the target audience. So unless you have people you don't know testing your software, you're probably in alpha.

Agile Approach to QA

Depending on their structure, stories generally relate to a piece of functionality. If stories are expressed with Volere Shells, the QA can be easier defined as these are written more with a specific test and priority in mind. These stories relate to a set of unit and integration tests that prove the story and feedback into the tools used within the CI environment to prove the QA.

Conclusions

I think Selip (p8) says it best so I'll leave it to him.

"Believe that all software, whether built in-house, purchased as COTS, or developed by outsourcing, is the work of fallible humans who are prone to making mistakes. Understand that the organization and its stakeholders are highly dependent on high-quality, fault-free software. Expect that requirements, design, and software development mistakes will result in risky software faults. Invest in people, processes, and tools to avoid, detect, and neutralize software faults. Mandate and create incentives for a culture that takes pride in quality, has the skills and knowledge to achieve it, and that aggressively finds and fixes faults."

Lipner & Howard (p11) also suggest that project teams should concentrate on threat modeling, code reviews, using automated tools and fuzz testing throughout the lifecycle. This should be done as a higher priority then penetration testing which is used to define production readiness rather then used to find bugs.

References

Selip, S. (2005), To Err Is Human, So Test That Software, Burton Group, http://www.burtongroup.com/content/printver.aspx?cid=729

Lipner, S & Howard, M (2005), The Trustworthy Computing Security Development Lifecycle, MSDN, http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnsecure/html/sdl.asp

Futher Reading

http://www.volere.co.uk