Agility for Complex Systems

At SPA2008, there was much debate amount the use of agile practices in developing software.

A comment that I heard was 'that for all the noise that the agile community makes us believe that agile is mainstream and the most common approach in developing software today, the reality is somewhat different'. Certainly there are many good examples of where agile approaches have been successfully applied but I guess that many of these are in 'young' and 'smallish' companies and not in large, corporate companies which have many decades of history.

It is not easy to convince these large and well-established companies that there are alternative ways of developing software. One reason, is often the 'process' police. Large companies, in my experience, are very keen on following a tried and tested approach, with some limited tailoring to cater for different sizes of developments.

So what are the challenges in applying agile techniques to complex systems?

SPA2008 - some final thoughts

It is now 2 weeks since the end of the SPA 2008 Conference and I have finally finished writing my notes. (Yippee!) So what are my final thoughts and what I am going to take away for exploring in the next few weeks and months?

I thought the conference was excellently run with a good mix of software practitioners attending. I was spoilt for choice in selecting the sessions to attend but there are some good notes appearing which summarise the sessions I didn't attend. There are also a number of articles appearing on various blogs. The dialogue during and between sessions was also stimulating and I hope to continue this when time permits.

Technically, I am particularly interested in following up the following topics:

• Python, Django and other frameworks


• Erlang, particularly as part of a multi-language development probably with C, C++ or Java


• Further development of metric dashboards (which is work in progress ) now that I have got some further input to consider


• Look more at DSM, DSL's and domain specific code generation


• Decaying architecture and legacy code bases, and attempting to detect when to intervene

I also learnt a lot about me. The conference confirmed that generally, the work I do is pretty well in line with what the rest of the software world in the UK is currently doing (or tyring to do!). There are differences but these appeared to be related to the domains, size of developments and team dynamics.

Describing Software Architectures

My last session at the SPA 2008 Conference was a workshop to explore the approach to describing a software architecture.

Nick Rozanski, Eoin Woods and Andy Longshaw asserted that any architecture requires a lot of selling with many different stakeholders and customers interested in the architecture. Communicating the architecture is obviously key and a group session explored what should be produced in an architectural description document. The varied experience of the groups produced a diverse set of content, with the more experienced architects recognising the importance of a clear baseline of requirements from which the architecture can be derived and confirming an understanding of how the delivered system is to be used. Clearly, the project size and life cycle approach has an impact on the production of the architecture; projects following a traditional waterfall life cycle (tending to be part of larger programmes) placed a increased emphasis on the requirements whereas the more agile projects placed much less emphasis on this and were keen to start producing some tangible software. This discrepancy also appeared in another SPA 2008 session that I attended.

There was also an interesting exploration in the tools and techniques used to produce the content of an architecture document. A recurring message was that the document must be in a format suitable for all of the readership, so clear preference for using a variety of standard office based tools such as MS-Word, Powerpoint or web pages on an Intranet rather than specific architecture tools e.g. modelling tools for UML which may not be available to all readers (one solution to mitigate this would be to ensure that such content is exported into one of the more accessible formats e.g. HTML). There was the recommendation that the tools used in the architecture should form part of a integrated toolset with good traceability between the requirements, architecture and design being available. The availability and use of such integrated toolsets appeared to be limited to larger programmes, particularly where the software was only a part of the eventual solution.

Once an architecture is produced, it needs to be publicised and communicated to everyone who is impacted or interested in the architecture. While there was general agreement that the architecture must be peer-reviewed before this wider communication is performed, the approach to communicating (or socialising, as Andy called it) the architecture varied from simple 1-1 chats to major presentations. What was clear was that the architecture must continue to live and that it needs to be actively maintained, remembering that the original approach of selling the architecture also applies to any subsequent changes.

Overall, I felt the workshop confirmed that the approach to software architecture that I follow continues to be appropriate and picked up some good ideas for trying on a future architectures.

A summary of the output produced by the workshop is available on the SPA 2008 Conference wiki.

Smelly Software Architectures

At the SPA 2008 Conference, Mark Dalgarno led an interesting workshop which explored the deterioration of software architecture over time. Mark outlined a number of different conditions which could indicate that the software architecture (assuming that there was one to start with!) is decaying from the as-intended architecture. There was an interesting debate about the importance of architecture and there was a clear distinction about architecture minimalism from those practicing agile techniques, to those engaged in large and complex systems of which software is only a small part of the overall solution.

I firmly believe that a key principle is that any architecture should be resilient to change and should be an enduring artifact. It should also be flexible and scalable, although this can depend on the expected lifetime of the architecture (architectures that are only expected to last for a year or two clearly don't need to exhibit the same attributes as an architecture which is intended to last for 10 years or more). This also requires that the architecture and design are seen as distinct activities - I have seen many examples where the architecture has been assumed and the detailed design and implementation has proceeded without any thought about the architectural options or future maintenance requirements.

How an architecture can be assessed was an interesting discussion with metrics being used (e.g. dependency counts) or consideration of some potential change scenarios although the more agile developments considered this to be unnecessary. Example scenarios were offered such as a change in the operating system version or a database upgrade and the resilience of the architecture to this change. However, it is clearly unachievable to think of all such change scenarios that could occur. Some examples of changes that weren't originally anticipated but were then implemented with some significant pain included the addition of error messages in multiple languages. The impact of a project driven by time pressures (time driven coding) were also likely to lead to a less enduring solution and a less robust architecture.

The economics of architectural decay was considered with a view that 'rewriting is considered harmful' particularly for solutions which are deployed widely where there is no acceptable alternative to maintaining what is currently implemented. While I sympathise with this as a view, any maintenance regime should always consider UUD (upgrade, update and disposal) of an in-service solution and to recognise that there may be some time when multiple versions of the same solution operating on potentially different architectures may have to be maintained in parallel.

While the session didn't offer any solutions to a difficult problem, it did pose some interesting thoughts which will require some further consideration for future software architectures.

What's the point of Software Architecture?

At the SPA 2008 conference, I attended a session entitled 'Does Architecture Help?' in which Eric Nelson from Microsoft identified 9 observations on architecture that he has encountered from his work with Independent Software vendors (ISVs). He noted that there was no correlation to project success if there isn't an architecture and that there are many different ways (read architecture) of solving the same problem. Although there isn't a perfect architecture, there are clearly examples of bad architectures, which are often demonstrated by symptons such as change being very expensive, that have delivered a great service possibly through the massive advances in hardware technology being 'the get out of jail card'.

The rise of agile developments has clearly impacted the importance of architecture with perhaps the removal of the formality of this activity. Clearly the availability of technology can have significant impacts on software architecture. While there is always the tempatition to try and accomodate the latest technology in your next project, I think this is a risky strategy as architectures normally need to be based on firm foundations (there are always exceptions). With a good architecture (not a perfect one!), it is possible to keep technology choice separate from the architecture. In my experience of Model Driven Architectures (MDA), it is possible to separate the technology from the architecture through the use of separare models:

• Platform Independent Models (PIM) which is independent of a the technical choice


• Platform Specific Model (PSM) in which the technical choice is made

The slides for the session are available from Eric's blog.

I discovered something when I downloaded the slides, a new file extension .PPTX. My computer didn't recognise the extension (I only had Microsoft Powerpoint Viewer 2003 installed). A hunt around Microsoft's website revealed a free download for Microsoft Powerpoint Viewer 2007 which adds the capabilty to view .PPTX files, which I discovered are Powerpoint files stored in Microsoft's new XML file format.