Performance Management for a Complex World

A thought provoking presentation was recently hosted by the Manchester branch of the IET on Performance Management in a Fast Moving World presented by Dr Therese Lawlor-Wright (University of Manchester), Elizabeth Jovanovic (EEF) and Andrew Wright from Dynamic Technologies Ltd.

After a brief introduction in to what performance measurement was (determining the quality of execution, competence and effectiveness of a actor or operation compared against a standard) and how it could be applied to both people (typically in the form of appraisals) and processes and systems (typically in the form of project reviews), the presentation made a few key observations about how performance measurements are currently performed.

• Organisations define performance measurements at the strategic level to see how it can achieve goals, often stated in terms of cost, quality, time and reported against a number of key performance indicators (KPIs). These can then be used to communicate and confirm (or adjust) progress.
• Any KPIs used must be balanced, fair and transparent
• The performance measurements must include indicators of the effectiveness of the organisation (the extent at which the strategic objectives are being met) and the efficiency of the organisation (how economically the resources of the organisation are being used to provide the level of performance).

Whilst the measurements can be useful, there was a word of caution in the unwanted effects of performance management systems (cited by De Bruijn (2002))

• The measurements will take too long to accumulate and will soon be out of date (data lag)
• Competition between teams or business units can result in a tendency to not share valuable data
• The measurements can stifle innovation at the expense of efficiency
• There will be 'game' playing to maximise the 'score'

These effects will dominate in the long term. To counter this, the measurements must be refreshed regularly. This should ideally be every 2 to 3 years, as this gives sufficient time for the data to be used for benchmarking/comparison purposes but short enough to counter the complacency that can arise.

Managing Performance

Having established the measurements to be made, the presentation moved on to the application of the measures in the form of performance management. Performance management needs to be both process-oriented and people-oriented and must be a continuous process (and not just an annual activity which is often the case). By being continuous it helps to clarify expectations, standards and targets and allows for any corrective actions to be addressed as soon as they arise. The most common approach for people-oriented performance management, the appraisal, should link individual targets to organisation targets but must be an opportunity to praise and develop. As expected, the audience was reminded that all objectives must be SMART. However, an alternative method of specifying a target was proposed - 'Positive - Personal - Present' which can be used to change behaviour. It can improve staff morale if done correctly, as it apparently tricks your subconscious mind into acting positively (the targets should be written starting 'I ...'). There was a strong suggestion that contrary to many organisations, performance management must not be linked to remuneration since it results in a warped approach in order to fit in with the inevitable budgetary constraints.

The Complex World

The complex world was defined as fast moving (continual change, increasing hierarchies of complexity) together with increasing challenges (timescales, budgets, mergers, multiple dependencies). These require that performance management is aligned with the business strategy. However, the classical approach to strategic management with a top-down controlling hierarchy was considered unsuitable. For complex systems,a more holistic approach is required with a thoughts being contributed from the bottom upwards.

With a fast changing environment, long-term plans quickly lose touch with reality with inflexible KPIs driving behaviours that fail to respond to the real-world challenges. Inconsistency between the strategy, real-world reality, the KPIs and the objectives inevitably quickly leads to poor performance. Clearly there needs to be an alternative approach to performance measurement which is both flexible and efficient.

Taking some of the ideas from the Agile Manifesto, a more lively and dynamic approach developed by consensus which adapts to the changing environment was proposed. This approach addressed many of the unwanted effects with the traditional performance measurement schemes by being much more efficient and flexible with an empowered organisation with a shared vision. Use of such techniques such as the Balanced Scorecard and the EFQM Excellence Model can clearly help in communicating a comprehensive view of an organisation.

Key Conclusions

• Strategy must become change oriented with a dynamic response in a controlled manner. The route to the strategic vision may change.
• Long-term plans leave companies without direction
• KPIs and objectives must respond to the changing needs
• The measurement process must be flexible and efficient
• Good performance must be encouraged by reducing uncertainty
• Organisations need to move from optimising the 'simple' status quo to optimising 'complex' continuous change

Open Source Certification

I have just been browsing the relaunched website of the British Computer Society and came across an interesting article on Open Source Certification. Now there are some pretty important and successful open source applications out there, but there is limited experience of 'certification' in the same way that you can be become, for example, Microsoft certified. Red Hat does offer some courses for you to become a Red Hat Certified Engineer (RHCE) but this is an exception for open source applications.

The big question is does it matter? It all depends on your point of view regarding certification. Does the fact that a product is 'certified' make it a better product? Does the fact that an engineer is 'certified' make him a better engineer compared to one who isn't? As in all cases it depends. A certified engineer should certainly have independently demonstrated a degree of competence in using or configuring a product. However certification without experience to back up the qualification is no use to anyone. Similarly a certified product might demonstrate that the product has become too large and cumbersome that it really needs to be entrusted to a select band of engineers who have demonstrated that they understand the product better than those who have just learned to tame the product to met their specific requirements. A certified engineer should also probably be aware of a few tricks and tips which are not widely known.

So should all open source products offer a certification programme? In my view, no. However there is clearly a point at which certification becomes necessary or expected by the customer community. I would suggest that this can occur in a number of cases:

• When the product is becoming widely accepted as one of the market leaders across multiple platforms.
• When the product is now developed on 'commercial' lines with a funding line.

In either case, a professional certification programme should be promoted and managed, but recognizing that significant experience of a product should be automatically rewarded (on request) with certification, particularly if the experience has been gained through the formative years of the product.

A similar approach was adopted a few years ago by the BCS when it launched the Chartered IT Professional (CITP) qualification. To date, this has yet to become a widely accepted, recognized (and demanded) qualification for key roles within the IT industry. Until recognized qualifications or certifications within the IT industry become a pre-requisite for certain roles, the certifications people achieve will be little more than another the certificate to put on the wall or in the drawer. Until this is the case open source certification will become little more than a commercial exercise in raising funds for future product developments.

Project Management blogging

From time to time, I anonymously contribute to the BCS ProjectEye Blog on a variety of subjects related to Project Management. Some of my contributions include:

• Open Source Project Management
• Skill Deficits
• The Manchester City Project
• Agile Project Management
• Festive Thoughts
• Ensuring Timely Success
• The Sorcerer's Apprentice

Writing poor code requires skill

Recently I read an interesting post on decaying code and the various ways that could be adopted in order to improve code quality. It briefly outlined three approaches to improving code (presumably on a code base which has been evolved due to enhancements and bug fixes):

• Detect the bad code and fix it. But this is too expensive...
• Don't write it in the first place. But this requires you (or a tool) to be able to spot bad code (consistently) in the first place.
• Formal Training. This is fine, but how do you ensure that that 'training' is put into practice correctly? And it is all too easy to fall back into bad habits which won't get spotted.

This got me thinking. Are there any other approaches to prevent bad code (or code smells)? Well I reckon there are. In fact it should take skill to write poor code given the amount of help that there is at constructing software now.

Increase use of automatic code generation

When I started programming last century, there were two languages I could choose to write my software in. One was a high-level language and the other was assembly language. While all but the most time critical code was written in the high-level language, I sometimes found it necessary to look at the generated assembly code to understand why my program was failing or to optimise the code. Now, I never look at the object code as I never question the quality of the object code generated by the compilers. However badly the code is written, the compiler would normally ensure that the generated code is efficient; this often means that engineers can get away with sloppy or bad code as it is often automatically refactored in the background

by the compiler to something more optimal. It also encourages, in my opinion, the lazy programmer who understands that the compiler does all the hard work in terms of 'writing' good code.

The increasing use of model driven development (MDD) as part of a model driven architecture (MDA) as a way of improving software productivity is moving the goalposts again. In its purest form, the development uses visual tools and the generated code (in a high level language) and subsequent object code is never seen (by a human). However, I have yet to see any real world MDD which doesn't involve some algorithmic code still being written by hand in a conventional high-level language. Over time, the increasing use of MDD should result in less 'algorithmic' code being written which will, by implication, reduce the potential for less code to decay.

MDD offers the opportunity for engineers to focus on the design rather than the implementation which should result in more maintainable systems which can readily be adapted to include enhancements and changes throughout their life. As the generated code is not the primary artifact which engineers work with, the quality of the generated code becomes less important. However, a key question to consider is does the use of MDD lead to the potential of (over time) of creating a decaying model and if so, how do you prevent it? And can you perform 'bad' MDD?

Reuse (of code snippets)

There are an increasing number of source code repositories (for example see here) now available on the Internet offering a variety of resources from simple algorithms to reusable components. All of these repositories help in providing a (hopefully!) proven way of solving a particular problem, and should also be written in such a way to be readable in case the snippet needs to be tweaked in the future. Although I cannot advocate that the quality of all snippets will meet the criteria of not being 'bad' code, it is reasonable to expect that the many will be examples of 'good' code. If the code snippets can be used 'as is' without modification the code should remain maintainable; if the code is modified, the style of the original code should normally be preserved so that the code remains 'good'.

Although the repositories do not require that the code passes any quality checks with regards maintainability etc, it should become obvious that the better code will be downloaded more frequently.

Use Multiple Compilers

I have always advocated compiling code with two different compilers as a way of improving code quality. No two compilers are ever the same as each one has different strengths and weaknesses. I have also always promoted 'clean' compilation i.e. ensuring that all code compiles without warnings once the set of compile options have been defined. If the code compiles cleanly with two different compilers, there is an increased probability that the code is well-structured, which IMO implies that the code will be more maintainable. It should also help testing as a number of latent faults can often be removed prior to run-time.

Clearly if the multiple compiler approach is adopted, this must be used at all subsequent code evolutions; if not the code will clearly decay albeit more slowly than if a single compiler has been used.

Conclusion

Writing maintainable software requires skill. With a bit of thought (and resisting the temptation to code the first thing that comes into your head), quality code can be produced using one or more of the techniques outlined which will support future product evolutions in a cost effective manner.

Some thoughts on Kanban in Software Development

I recently watched a presentation given by David Anderson at QCon 2008 on his experience and observations of a kanban system applied in a software engineering environment. I found some of the observations very interesting, particularly the approach that optimising the system for lowest cost didn't actually result in the most efficient system in terms of optimal throughput. As with many great ideas, kanban appears to be a very simple concept; however clearly there is more to it than just a simple approach at managing a flow of post-it notes on a whiteboard.

I was keen to understand more about kanban and how it might be applied in various project scenarios. At the SPA2009 conference, there was a session presented by Karl Scotland on his take on kanban (Kanban, Flow and Cadence) together with a very interactive BoF session on Lean and Kanban. Whilst the comparison of lean, based on a model, originally promoted by Toyota, of optimising production, against kanban, which limits the work in progress was a  simple and very understandable comparison of the two approaches, it was when the debate moved to seeing how both lean and kanban could be applied to software developments that the debate got really interesting. 

Clearly software developments come in all shapes and sizes and the audience clearly represented a good cross section of developments and associated practices. Both approaches are clearly well suited to an agile approach in which features are developed to provide a flow of (increasing) value to the end user. This works particularly well in an iterative development where frequent delivery is encouraged, especially where the features are being evolved based on user feedback. This also requires a tolerant customer who can accommodate some failures. An interesting statement of 'get it right second time' was  promoted as acceptable provided you learn from mistakes. I find that hard to accept as the 'norm' because it is completely ignoring any recognition of embedding quality in a delivered product. It might be acceptable in a prototype development but not in a production environment. Clearly there was something missing. There was! The integration step. Now I consider integration to be a very important (and potentially very expensive) stage, which is often squeezed in terms of time (how do you know you have finished?), and can't be omitted.

So could kanban work in a more traditional waterfall development approach? Maybe. In a traditional approach, the requirements are analysed before design, implementation and testing are performed. Using a kanban system, a flow of features could be pushed through the separate stages of design, implementation and testing before handing over to integration, with limits being imposed at each stage. This would probably require that the 3 life cycle stages are performed by different team members, a not unreasonable expectation in a traditional development. Provided the requirements are relatively stable and the ordering and independence of features is organised to support a sensible integration approach in order to deliver increasing value (or functionality) then this could be an interesting approach. Managing the queues at each stage of the life cycle needs to be carefully managed since the costs of managing each queue must be proportional to the size of the queue otherwise the process becomes very inefficient. 

Now for the challenge. How likely is it that each feature will be take approximately the same time through the life cycle? - probably not. In my experience, it is highly unlikely that each requirement (or feature) will be equivalent in terms of effort expended, particularly when the software forms part of a complex system. This will result in some initial  inefficiency as it would take some time for the queues to become populated with tasks. There is the temptation to process easy requirements first as a way of seeding the work queues quickly. However, from a project management perspective this is probably not what is required as it leaves the difficult tasks (and hence risks) to the end (and most difficult tasks will probably impinge on the easier tasks in some form resulting in some additional unplanned work).

Looking back at the David Anderson presentation, it is clear why a kanban approach worked well in one of the case studies since the approach was only applied to bug fixing rather than  in a new development. I believe that looking at kanban in a multiple life cycle development needs a different approach for development with the queues being placed at different points in the system life cycle (as distinct from the software life cycle) to avoid bottlenecks building up. I would advocate that the same developer should be responsible for design, implementation and testing of each feature so that the queue limits are placed on the exit from analysis (or requirements definition) and entry into integration rather than within the different stages of software development. In a lean or agile development this approach is normal since the developers are performing all stages of the life cycle; in waterfall developments, and particularly large systems, this is not the norm. 

Adopting kanban in waterfall certainly isn't new (see Kenji Hiranbe's article) but there is little evidence currently to demonstrate if it can deliver tangible benefits for all types of project approaches. Are there any case studies to prove or disprove my hypothesis?

Web Security - an eyeopener

An impromptu Birds of a Feather (BoF) session given by Dave Misell at the SPA Conference admirally demonstrated what happens behind your back when you visit a website. Key to the demonstration was showing (very simply) how much data is passed between sites using cookies and how much data is retained (for considerable length of time in some cases) between site visits.

The session used the Paros proxy server with Firefox which easily showed the web transactions when entering a simple URL (I won't disclose the URLs used in the demonstration, suffice to say that the sites were well-known). Using Paros, it is easy to see how much information is passed via cookies. Stopping cookies results in the same information being passed via URL (although this is more obvious since the data appears in some form in the address bar). In many cases the same data is passed to a number if websites regardless of whether the data is appropriate or useful to the receiving website.

Although it is possible to see the information being sent to a website, it is not possible to determine what is done with the data by the receiving site. In most cases the data is used to trace a journey through website (so that the 'user experience' can be improved) which is totally transparent to the user since these are performed server side.

So is it possible to stop the information being transferred? Not easily, but choose your sites carefully. There is some legislation in the EU which has tightened up the exchange of information through websites, particulalry to third parties, without the express permission of the user. Unfortunatley this legislation is dependent on where the website is hosted, which isn't always obvious from a simple URL. There is always 'education' which applies to both the end user and also to the developers of the site, and there are some good courses now which can help (e.g. ethical hacking and MSc in Information Security at Royal Holloway College in London) increase awareness.

Can you scrum on your own?

An interesting question posed by Peter Bell at this year's SPA conference. His premise was that some of the lean and agile approaches can be equally applicable when you are THE team. He called his approach Solo Scrum.

Peter identified a couple of scenarios where he felt that 'Solo Scrum' could be appropriate:

• A full time developer doing open source development part-time
• A solo consultant developing applications for non-technical clients
• A 'work at home' contractor managing multiple clients

Clearly working on your own brings a number of challenges (note that I didn't say problems!) which need to be considered when working on your own. I think that probably one of the biggest challenges to overcome, (and this is based on personal experience), is to try and avoid too many distractions within the 'home' work area. The second challenge is that of loneliness! In the many (professional) development projects that I have worked, talking to colleagues has been a great way of confirming your thoughts for a solution or helping to find a way through a problem. When you are on your own, this isn't possible, or is it? However the use of technology can certainly help (think of IM, email, VOIP (with video)) and it can certainly be seen as 'virtual' pair programming. This isn't a new idea (see here); however, there is still the problem of finding a suitable 'pair' to link up with.

The traditional problems of planning don't go away when working on your own (as I said at the session 'Fail to plan, plan to fail'). Working on your own necessitates an efficient process for planning and managing the plan (a bit more than the 'back of an envelope' note). There are a number of Eclipse plugins (e.g. mylyn) which offer the right balance in terms of monitoring the work activity without distracting from the task-at-hand.

Some interesting ideas were discussed, including the observation that many team collaboration software packages often includes a free version for a small team (where small can be up to 5). This can overcome a often heard comment which was that I don't have the time/experience/etc to set up a 'professional' development environment (the minimum being a source control system, an IDE and a bug tracking system) from scratch.

Peter has now started a Google group to further the debate. I will watch with interest.