Notes from a Software Architect, on a small island

TDD for embedded development

2012-01-01T18:31:00.000Z

I recently attended an interesting talk on test driven development (TDD)
for embedded development given by Zulke UK for the BCS SPA specialist group.

Robotshop Rover

The talk was the result of some experiments using an Arduino board, a bluetooth interface and an android tablet. The embedded platform was a Robotshop Rover with a number of sensors and controllers. The sensors were used to guide the robot along a track indicated by a solid black line; the motors were used to control the direction of the robot and also its speed.

Although the standard arduino development environment isn't a full IDE and is limited in its functionality, does come with some good code
examples. An alternative environment is the Eclipse CDT with the AVR plugin added to handle the download of the arduino image to the target platform. To support the development using TDD, CPPUTEST was used as the test framework. CPPUTEST is recommended as a framework suitable for embedded development (see James Greening book on TDD for Embedded Development) and appeared to the presenters to be more effective than CPPUNIT. It was noted by members of the audience that there are few tools which have good integration with the continuous integration platforms such as Jenkins.

An overview of TDD was given, and its application for an embedded target environment

1/ TDD needs to be able to test on both the development and target environments. This requires that two projects are created, one with arduino as the target environment and an other targeted at an x86 environment

2/ The cycle of test->code->refactor needs to be followed with the tests being chosen from a backlog. Code shouldn't be written unless it is to satisfy an existing test.

3/ The cycle should be choose test->write test->run-test->fail! If it doesn't fail (and it could be due to compiler or link failure), then it normally indicates that the test has been incorrectly written.

4/ Limited design is required although normal good software engineering practice should still be adopted (no monolithic functions etc).

5/ Mock interfaces should be used to unit test sensor interfaces. This allows the logic to be tested and debugged first before loading on to the target.

6/ If a state machine is required, some design is essential before any test cases can be identified and written.

The exercise in TDD with the arduino target resulted in no logic errors once the code was exercised on the target. However, the behaviour of the robot, in particular the speed of the motors required some further development and enhancement to the codebase. Given the fact that arduino boards are typically focused on the school market, I was surprised that C++ was the chosen development language. However, the C++ used on the arduino is a cut down version which removes many of the complexities of the full C++ language.

Whilst the associated Android development (on a Motorola XOOM tablet) appeared to be successful in terms of developing a simple user interface to send commends via Bluetooth to control the robot (e.g. stop, start,...), the development of the application revealed some shortcomings. Although the Android development kit works very well with Eclipse (the code is a mixture of Java and xml) and allows on target debugging, the use of TDD is less appropriate given the extensive use of callbacks (e.g.onclick, ....). The testing of GUI applications cannot be adequately tested within a development environment; fortunately the android debugger is excellent at pinpointing issues (typically null pointer exceptions). Android emulators can help to a limited extent but are not a sufficient replacement to an actual device. Android development for tablets is just evolving as the platform moves from a phone, with relatively simple applications, to potentially far more complex applications. The launch of Android 4.0 development kit (aka Ice-Cream sandwich) will clearly accelerate the development of more complex applications which will necessitate the employment of sound software engineering principles in delivering quality products.

In summary, the session demonstrated successfully that TDD could be applied in an embedded environment and that through the use of appropriate open source tools, software development for the expanding android market can follow tried and tested techniques.

Open Source Software - the legal implications

2011-11-17T23:25:00.004Z

I attended the recent BCS Manchester/Society for Computers and Law event on the use of Open Source Software and the legal implications. It was given by Dai Davis, an information technology lawyer who is also a chartered engineer, a very unusual combination but he clearly knew his material.

Dai started by explaining the basic aspects of copyright law and the implications that this

had on software. It was clear that some of the original purposes of copyright law (to prevent

copying) were applicable to software but that the period that copyright lasts (70 years after

the authors death) clearly made little sense for software. However a number of points caught my eye:

Copyright protects the manifestation and not the subject matter. This means that look and feel is not normally subject to copyright although fonts are.
Copyright infringement also includes translating the material. Translation in the software case includes compiling source code as well as rewriting the source code into another language.
Copyright protects copying some or all of the material. The amount does not normally matter.
Moral rights do not extend to software but do apply to documentation
Copyright infringement is both a civil and criminal offence with a maximum of 10 years imprisonment and an unlimited fine.

Dai then explained that the first owner (or creator) of the material owns the copyright. Misunderstanding this is the major cause of disputes about copyright. Clearly there are exceptions if the material is created in the course of employment (copyright rests with the employer) or if the contract under which the material is being created 'assigns' the copyright to the purchaser.

All software licences grants the purchaser permission to use the software otherwise the purchaser would be in breach of the copyright. Licences can be restrictive e.g. by time, number of concurrent users and all licences are transferable according to EU law.

Copyright of Open Source Software is no different to normal copyright of software but the approach to licencing is very different:

Nearly all OSS do not require payment to acquire
Free relates to restrictions on use (non-OSS can place restrictions)
Open access to source and usage is required (not normally available with non-OSS)

However, the licences are very difficult to enforce mainly because there has been no loss in terms of monetary value. There has never been a successful prosecution in the UK although there are a number of examples in Germany (where litigation is cheaper than the UK) and an example in the US (Jacobson v Katzer in 2009) where a 'token' settlement of $100000 was awarded.

Whilst there may be little prospect of getting sued for use of Open Source Software the biggest issue often comes when businesses are sold and OSS is found within a product - this often affects the eventual purchase price of the company. Many businesses don't know where Open Source Software is being used and included within its own products because it is very difficult to police and manage.

A video of the session was filmed by students from Manchester Metropolitan University the resulting video being made available via the BCS Manchester website.

How good are your requirements?

2011-07-27T23:03:00.003+01:00

I read an interesting post which was trying to determine how much detail a requirement should contain. As with any question like this, it all depends on a number of factors and it is not possible to give a rule which can be religiously followed. Experience over time will determine what works for you - there is always a cost if the balance isn't right. Too few requirements and there is likely to be a very difficult verification phase; too many requirements and it will prolong the development and testing phases.

The first rule of requirements is to ask your customer 'why?'. If they can't explain simply why the requirement is needed then it isn't essential and it should be discounted. If the response is acceptable, the next question is 'How am I going to validate that I have satisfied the requirement'? If you can't agree on how the requirement is to validated, then the requirement clearly needs clarifying, maybe with some constraints explicitly included in the requirement wording. The language of requirements is very important and the differences between shall, should, will, may are VERY important. Any requirement with phrases such as 'such as', 'for example' should always be rejected as they are open-ended and can never be fully satisfied.

As an example, I encountered a requirement recently which included the phrase 'any printer shall be supported'. After discussions, and any customer that doesn't engage in discussions is clearly an indication of a very difficult customer!, it became very clear why the requirement had be written in the way - a third party would be supplying the as yet unspecified printer. However, the discussion did enable the requirement wording to became slightly more achievable by rephrasing as 'a network connected printer supporting postscript'. At least I would have a chance of testing this (and have constrained it to exclude printers with parallel or usb interfaces).

The hardest part of requirements is not the accepting the requirement, it is the validation at the end. How often have you encountered 'This isn't what I wanted'. The only way to avoid this is to remain in constant contact with your customer to validate any assumptions throughout the development so that there aren't any surprises at the end.

Other than experience, I have yet to find a reliable and objective way of assessing the 'goodness' of a requirement set. It would clearly be possible to perform some analysis of the language used, looking for ambiguous phrases for example, but would this be a sufficient measure?

Computer Forensics

2011-06-04T22:57:00.006+01:00

BCS Manchester recently hosted an interesting evening on the growing importance of computer forensics. The session was led by Sam Raincock, an experienced expert in performing forensic analyses. Whilst the session did not reveal the secrets behind a successful analysis (or give hints of how to make an analysis more difficult), it did explore some of the approaches (in general) that can be used in establishing evidence. Whilst a typical forensic investigation does include a significant amount of technical information this only accounts for about 20% of an analysis as the remaining time is concerned with communication with lawyers and report writing. As in all legal cases, it is crucial to review and present all the evidence to piece together a coherent case rather than circumstantancial evidence.

While Computer forensics is primarily the examination of any device that permanently stores data (the range of devices is ever-expanding from the traditional hard disc drives, CD-ROMS and USB memory sticks to mobile phones and cameras), it also includes reviewing system functionality in its goal to try to establish what happened and who did it. It is used in a variety of cases include criminal, civil and fraud cases.

It was stated that 'Every contact leaves a trace' by Edmond Locard, an early pioneer of forensic science. This is very true with all computer usage as every time a file is created, every web page that is browsed, every document that is printed is recorded somewhere although computer usage is unique to everyone.

Some key points that I took away from the session included:

Never assume anything
All humans are unpredictable, and different
Personnel cause more damage than they discover
Do not assume that common sense prevails
The IT department are not forensically trained and don't necessarily understand the value of every piece of data
Forensics is not about data recovery
Ownership of data must be established

A forensic examination is looking at where the offence was allegedly committed, how the event occurred and who performedthe activity. A typical examination can normally be performed on a single device (once a forensic image has been taken) by an appropriate expert and does not normally need to consult with outside agencies (e.g. internet service providers) to obtain specific information. The examination will review such data as cookies, the various system logs, network connections (IP addresses, type of connection particularly whether it was local, remote, fixed, wireless etc). The usage patterns of a computer will reveal a significant amount as every human has particular behaviour traits. The use of the various system logs that reside on a computer or within a network can reveal significant and valuable data; these logs should be actively monitored as they can often be the first sign that something unusual is being performed that may merit investigation. The sooner something is detected, the greater the chance of limiting the damage (or increasing the evidence in order to establisha conviction). In the case of an incident being detected within a business, the primary aim is to return a business to normal as quickly as possible. This is where policies are vitally important; it is equally important that they are actively used, policed and maintained.

Whilst there is no formal qualification required to become a forensic expert (an inquiring mind would probably be useful), it is clearly a growing and important aspect of computing. There are clearly many challenges with the continually evolving usage of computers; the growing importance of the cloud will clearly require different techniques to those employed when examining a physical item such as a laptop. The session left me wondering what traits my computer usage would reveal about me but also wanting to find out more about what is being recorded without me having any knowledge.

Open Source Document and Content Management

2010-10-11T23:34:00.004+01:00

BCS Manchester recently hosted a meeting on open source content management and document management in the Public Sector. The speaker. Graham Oakes, explained that the catalyst for this had been an article in the Guardian about the use of commercial software by Birmingham City Council and its attempt at building websites. They had already spent £2.8M and the question was asked ‘Why not use Open Source Software instead?’ Graham stated that there is an awful lot of mis-information around, in particular that the use of OSS does not mean it costs nothing! This led the BCS Open Source Software Group to run a conference in January 2010 looking at the use and adoption of OSS in the public sector.

After briefly outlining what OSS is (source code owned by the community which can derive new works), Graham outlined a typical software stack found in many organisations. At every level, there are good open source solutions available. Content Management normally fits in at the application layer below portals and webservers. Content Management has a number of very strong options including Plone, Hippo, TYPO3, Drupal, Joomla, eZ and Umbraco. Many of these have already been adopted by public sector in developing websites, for example a number of Police forces.Document Management is not as well developed as Content Management and there are fewer options.

Gartner and Forrester enterprise software reviews both report that OSS should be adopted and it is becoming more amenable to use in the UK but it is still necessary to consider the full life-cycle costs. OSS should be considered equivalent to proprietary – public sector should now consider and event contribute to OSS projects. However UK is some way behind other european nations (specifically Netherlands, Germany, Italy and Denmark) with the OSOSS project in Netherlands urging public administration to use more OSS and open standards.

Advantages

The key advantages of adopting OSS within the public sector were identified as

The reason for adoption is low up-front costs. Low costs of initial ownership, but the organisation needs to consider normal software selection processes and consider the risks, requirements for the software etc. OSS should be considered no differently to commercial software. It is still necessary to look at the total cost of ownership (TCO) and an organisation may need to still involve a system integrator in order to deploy effectively.
OSS applications do not constrain the design. The public sector can can use it to start small, think big.
Many OSS is easier to work with because access to source code (only useful if you have skills to use it!) is always available. This can provide an additional option to documentation. OSS is also increasingly important with the cloud as proprietary license models don’t adapt readily.
Good to help the public sector to demonstrate openness (committed to visibility and open to the public)

Apart from the last one, the advantages are not particularly specific for public sector adoption.

Risks

Of course, there is always a downside, commonly called risks in procurement circles. The risks identified included

Getting over the perception that everything is free. This misunderstands the true costs of OSS. Most costs in using software are not in the licenses – it is actually in the content created/managed by the software. Most project costs are less than 10% on technology/license and the migration costs must always be considered as these can often be many more times more expensive than the base software costs.
Misperception around content management and reusability of OSS. Just because some OSS can be used securely, does not mean that all OSS is secure.
Public sector are use to working with large organisations. OSS needs a different approach which the public sector may not readily adopt due to the mismatch of scale. OSS developers often move at a far faster pace than the public sector does (or can). This can be difficult at the procurement stage as the existing procurement model (level playing field) is broken. The procurement approach needs to recognise that OSS is no more less secure than proprietary solutions
Unreasoned decisions still dictate the major procurement decisions. OSS might not be a perfect match to the requirements but may provide a suitable solution.

Conclusions

Graham presented a set of conclusions, which if I am honest are no different to proprietary software.

Remember not all OSS are not the same, different quality levels and capabilities
Always chose carefully – consider usages scenarios
Look beyond licensing costs, user adoption, change management, migration
A team still creates the success over the technology. Choose the right team!
OSS supports evolutionary delivery, try before buy, which encourages innovation and supports agile and lean practices. However, this is good practice for all software.
License fees for software bring costs forward and commit project for the duration (unless trial licenses are available). OSS does not have this commitment and it is (relatively) easy to change OSS software without excessive upfront costs.

So would OSS have solved Birmingham’s problem. No. The problem was not a cost of licences issue; it was not understanding the issue well enough. OSS would have helped to examine the problem in the small before the initial financial commitment was identified which might have produced a more realistic budget.

Lessons in measurement and data analysis

2010-07-20T20:04:00.003+01:00

Recently I attended a very interesting and entertaining lecture by Peter Comer, from Abellio Solutions, to the BCS Quality Management Specialist Interest Group (NW) on lessons learnt in measurement and data analysis following a recent quality audit of an organisation’s quality management system (QMS).

The talk started by highlighting the requirements for measurement in ISO9001 (section 8). Key aspects that were highlighted included

Measure process within QMS to show conformity with and effectiveness of QMS
Monitoring and measurement of processes, products and customer satisfaction with QMS
Handle and control defects with products and services
Analyse data to determine suitability and effectiveness of QMS
Continual improvements through corrective and preventative actions

It was noted that everyone has a KPI (Key Performance Indicator) to measure the effectiveness of products and services although every organisation will use the KPIs slightly differently.

Peter outlined the context of the audit, which was an internal audit in preparation for a forthcoming external audit. The audit was for a medium sized organisation with small software group working in transport domain. A number of minor non-conformances which were relatively straightforward to correct. However, after the audit an interesting discussion ensued regarding the software development process which stated that they were finding more bugs in bespoke software development than anticipated and a lot harder to fix. Initial suggestions included:

Look at risk management practices. However, the organisation had already done this by reviewing a old (2002) downloaded paper looking at risk characteristics.
Look at alternative approaches to software development.

It was the approach to risk which intrigued Peter. The quality of the paper was immediately considered. What was the quality of the paper? Has it been peer-reviewed? Is it still current and relevant?

Peter then critiqued the paper. The paper proposed a number of characteristics supplemented by designators; it was quickly observed that there was considerable overlap between the designators. The analysis of the data was across a number of different sources although no indication of what the counting rules are (and no indication if they were rigorous and consistent). The designators were not representative of all risk factors that may affect a development and said nothing about their relevance to the size of development. The characteristics focused on cultural issues rather than technical issues - risk characteristics should cover both. Just counting risk occurrences does not demonstrate the impact that the risk could have on the project.

Turning to the conclusions, Peter considered if the conclusions were valid. What would happen if you analysed the data in a different way, would the conclusions be different? Can we be assured that the data has been analysed by someone with prior experience in software development? It was observed that designators were shaped to criteria which is appropriate, but one size doesn’t fit all. Only by analysing the data in a number of different ways can the significance of the data can be established. It can also show if the data is not balanced which can in turn lead to skewed results. In the paper under review, it was clear that qualitative data was being used quantatively.

Peter concluded by stating that by ignoring simple objective measures can lead to the wrong corrective approach which might not be appropriate to their process and product. This is because ‘you don’t know what you don’t know’. It is essential to formally define what to count (this is a metric) with an aim to make the data to be objective. Whatever the method for collection, it must be stated to ensure that it is consistent.

The talk was very informative and left much food for thought. I have always aimed to try and automate the collection process to try and make this consistent. However this does nothing if the data is interpreted incorrectly or inconsistently. It is also difficult to know if you are collecting the right data but that is what experience is for!

Creating Intelligent Machines

2010-03-17T22:50:00.007Z

I have just attended the excellent IET/BCS 2010 Turing Lecture 'Embracing Uncertainty: The New Machine Intelligence' at the University of Manchester which was given this year by Professor Chris Bishop who is the Chief Research Scientist at Microsoft Research in Cambridge and also Chair of Computer Science at the University of Edinburgh. The lecture allowed Chris to share his undoubted passion for machine learning, and although there were a number of mathematical aspects mentioned during the talk, Chris managed to ensure everyone was able to understand the key concepts being described.

Chris started by explaining that his interest is in building a framework for building intelligence into computers, something which has been a goal for many researchers for many years. This is now becoming increasingly important due to the vast amounts of data which is now available for analysis. With the amount of data doubling every 18 months, there is an increasing need to move away from purely algorithmic ways of reviewing the data to solutions which are based on learning from the data. This has traditionally been the goal for machine (or artificial) intelligence and despite what Marvin Minsky wrote in 1967 in 'Computation: Finite and Infinite Machines' that "within a generation ... the problem of creating 'artificial intelligence' will substantially be solved", the problem still does not have a satisfactory solution for many classes of problem.

A quick summary of the history of artificial intelligence showed that expert systems, which were good at certain applications but required significant investment in capturing and defining the rules, and neural networks which provide a statistical learning approach but have difficulty in capturing the necessary domain knowledge within the model, were not adequate for today's class of problems. An alternative approach which was able to integrate domain knowledge with statistical learning was required and Chris's approach was to use a combination of approaches:

Bayesian Learning which uses probability distributions to quantify the uncertainty of the data. The distributions are amended once 'real data' is applied to the model which results in a reduction in the uncertainty.
Probabilistic Graphical Models which enables domain knowledge to be captured in directed graphs with each node having a probability distribution.
Efficient inference which ensures efficiency in computation

To explain the approach, Chris sensibly used real-life case studies to demonstrate the application of the theory in three very diverse applications.

His first example was of Bayesian Ranking system to be used in producing a global ranking from noisy partial rankings. The conventional approaches is to use the Elo rating system which is a method for calculating the relative skill levels of players in two-player games. The Elo system could not handle team games or more than 2 players. As part of the launch of the Xbox 360 Live online playing solution, Microsoft developed the TrueSkill algorithm to match opponents of similar skill levels. The TrueSkill algorithm converges far faster than Elo by managing the uncertainty in a more efficient way; it also operates quickly so that users can find suitable opponents in a few seconds out of a user population of many million. Further details on TrueSkill(TM) are available at http://research.microsoft.com/en-us/projects/trueskill/

The next example was for a website serving adverts and how to determine which advert to show based on the probability of being clicked and the value of click. The proposed approach was to use gausian probability in order to assign a weight to a number of features which is used to determine the ranking. However it is important to ensure that the system continually learns in order to re-evaluate the ranking to ensure that the solution accurately reflects the dynamics of the adverts. If this was not the case, it would be very difficult for a new advert to be be served.

The final example was the Manchester Asthma and Allergy Study which is working with a comprehensive data set acquired over 11 years. The data set is continually being augmented with new types of data (recently genetic data has been added) and the study has been successful at establishing the important variables and features and their relationships. By defining a highly structured model of the domain knowledge, it has been possible to assign each variable a probability distribution. By placing the data at the heart of the study and applying some machine learning techniques, a number of key observations are now being reported which might not have been apparent if more traditional statistical techniques had been used.

As a closing remark, Chris promoted a product from Microsoft Research (Infer.net) which provides a framework for further experimentation in developing Bayesian models for a variety of machine learning problems.

As is now traditional with the Turing Lecture, it is presented at several locations around the country. A webcast of the version presented at the IET in London is available on the IET TV channel.

Performance Management for a Complex World

2009-11-27T22:53:00.005Z

A thought provoking presentation was recently hosted by the Manchester branch of the IET on Performance Management in a Fast Moving World presented by Dr Therese Lawlor-Wright (University of Manchester), Elizabeth Jovanovic (EEF) and Andrew Wright from Dynamic Technologies Ltd.

After a brief introduction in to what performance measurement was (determining the quality of execution, competence and effectiveness of a actor or operation compared against a standard) and how it could be applied to both people (typically in the form of appraisals) and processes and systems (typically in the form of project reviews), the presentation made a few key observations about how performance measurements are currently performed.

Organisations define performance measurements at the strategic level to see how it can achieve goals, often stated in terms of cost, quality, time and reported against a number of key performance indicators (KPIs). These can then be used to communicate and confirm (or adjust) progress.
Any KPIs used must be balanced, fair and transparent
The performance measurements must include indicators of the effectiveness of the organisation (the extent at which the strategic objectives are being met) and the efficiency of the organisation (how economically the resources of the organisation are being used to provide the level of performance).

Whilst the measurements can be useful, there was a word of caution in the unwanted effects of performance management systems (cited by De Bruijn (2002))

The measurements will take too long to accumulate and will soon be out of date (data lag)
Competition between teams or business units can result in a tendency to not share valuable data
The measurements can stifle innovation at the expense of efficiency
There will be 'game' playing to maximise the 'score'

These effects will dominate in the long term. To counter this, the measurements must be refreshed regularly. This should ideally be every 2 to 3 years, as this gives sufficient time for the data to be used for benchmarking/comparison purposes but short enough to counter the complacency that can arise.

Managing Performance

Having established the measurements to be made, the presentation moved on to the application of the measures in the form of performance management. Performance management needs to be both process-oriented and people-oriented and must be a continuous process (and not just an annual activity which is often the case). By being continuous it helps to clarify expectations, standards and targets and allows for any corrective actions to be addressed as soon as they arise. The most common approach for people-oriented performance management, the appraisal, should link individual targets to organisation targets but must be an opportunity to praise and develop. As expected, the audience was reminded that all objectives must be SMART. However, an alternative method of specifying a target was proposed - 'Positive - Personal - Present' which can be used to change behaviour. It can improve staff morale if done correctly, as it apparently tricks your subconscious mind into acting positively (the targets should be written starting 'I ...'). There was a strong suggestion that contrary to many organisations, performance management must not be linked to remuneration since it results in a warped approach in order to fit in with the inevitable budgetary constraints.

The Complex World

The complex world was defined as fast moving (continual change, increasing hierarchies of complexity) together with increasing challenges (timescales, budgets, mergers, multiple dependencies). These require that performance management is aligned with the business strategy. However, the classical approach to strategic management with a top-down controlling hierarchy was considered unsuitable. For complex systems,a more holistic approach is required with a thoughts being contributed from the bottom upwards.

With a fast changing environment, long-term plans quickly lose touch with reality with inflexible KPIs driving behaviours that fail to respond to the real-world challenges. Inconsistency between the strategy, real-world reality, the KPIs and the objectives inevitably quickly leads to poor performance. Clearly there needs to be an alternative approach to performance measurement which is both flexible and efficient.

Taking some of the ideas from the Agile Manifesto, a more lively and dynamic approach developed by consensus which adapts to the changing environment was proposed. This approach addressed many of the unwanted effects with the traditional performance measurement schemes by being much more efficient and flexible with an empowered organisation with a shared vision. Use of such techniques such as the Balanced Scorecard and the EFQM Excellence Model can clearly help in communicating a comprehensive view of an organisation.

Key Conclusions

Strategy must become change oriented with a dynamic response in a controlled manner. The route to the strategic vision may change.
Long-term plans leave companies without direction
KPIs and objectives must respond to the changing needs
The measurement process must be flexible and efficient
Good performance must be encouraged by reducing uncertainty
Organisations need to move from optimising the 'simple' status quo to optimising 'complex' continuous change

Open Source Certification

2009-09-20T21:09:00.010+01:00

I have just been browsing the relaunched website of the British Computer Society and came across an interesting article on Open Source Certification. Now there are some pretty important and successful open source applications out there, but there is limited experience of 'certification' in the same way that you can be become, for example, Microsoft certified. Red Hat does offer some courses for you to become a Red Hat Certified Engineer (RHCE) but this is an exception for open source applications.

The big question is does it matter? It all depends on your point of view regarding certification. Does the fact that a product is 'certified' make it a better product? Does the fact that an engineer is 'certified' make him a better engineer compared to one who isn't? As in all cases it depends. A certified engineer should certainly have independently demonstrated a degree of competence in using or configuring a product. However certification without experience to back up the qualification is no use to anyone. Similarly a certified product might demonstrate that the product has become too large and cumbersome that it really needs to be entrusted to a select band of engineers who have demonstrated that they understand the product better than those who have just learned to tame the product to met their specific requirements. A certified engineer should also probably be aware of a few tricks and tips which are not widely known.

So should all open source products offer a certification programme? In my view, no. However there is clearly a point at which certification becomes necessary or expected by the customer community. I would suggest that this can occur in a number of cases:

When the product is becoming widely accepted as one of the market leaders across multiple platforms.
When the product is now developed on 'commercial' lines with a funding line.

In either case, a professional certification programme should be promoted and managed, but recognizing that significant experience of a product should be automatically rewarded (on request) with certification, particularly if the experience has been gained through the formative years of the product.

A similar approach was adopted a few years ago by the BCS when it launched the Chartered IT Professional (CITP) qualification. To date, this has yet to become a widely accepted, recognized (and demanded) qualification for key roles within the IT industry. Until recognized qualifications or certifications within the IT industry become a pre-requisite for certain roles, the certifications people achieve will be little more than another the certificate to put on the wall or in the drawer. Until this is the case open source certification will become little more than a commercial exercise in raising funds for future product developments.

Project Management blogging

2009-09-06T14:02:00.005+01:00

From time to time, I anonymously contribute to the BCS ProjectEye Blog on a variety of subjects related to Project Management. Some of my contributions include:

Writing poor code requires skill

2009-06-23T23:47:00.008+01:00

Recently I read an interesting post on decaying code and the various ways that could be adopted in order to improve code quality. It briefly outlined three approaches to improving code (presumably on a code base which has been evolved due to enhancements and bug fixes):

Detect the bad code and fix it. But this is too expensive...
Don't write it in the first place. But this requires you (or a tool) to be able to spot bad code (consistently) in the first place.
Formal Training. This is fine, but how do you ensure that that 'training' is put into practice correctly? And it is all too easy to fall back into bad habits which won't get spotted.

This got me thinking. Are there any other approaches to prevent bad code (or code smells)? Well I reckon there are. In fact it should take skill to write poor code given the amount of help that there is at constructing software now.

Increase use of automatic code generation

When I started programming last century, there were two languages I could choose to write my software in. One was a high-level language and the other was assembly language. While all but the most time critical code was written in the high-level language, I sometimes found it necessary to look at the generated assembly code to understand why my program was failing or to optimise the code. Now, I never look at the object code as I never question the quality of the object code generated by the compilers. However badly the code is written, the compiler would normally ensure that the generated code is efficient; this often means that engineers can get away with sloppy or bad code as it is often automatically refactored in the background

by the compiler to something more optimal. It also encourages, in my opinion, the lazy programmer who understands that the compiler does all the hard work in terms of 'writing' good code.

The increasing use of model driven development (MDD) as part of a model driven architecture (MDA) as a way of improving software productivity is moving the goalposts again. In its purest form, the development uses visual tools and the generated code (in a high level language) and subsequent object code is never seen (by a human). However, I have yet to see any real world MDD which doesn't involve some algorithmic code still being written by hand in a conventional high-level language. Over time, the increasing use of MDD should result in less 'algorithmic' code being written which will, by implication, reduce the potential for less code to decay.

MDD offers the opportunity for engineers to focus on the design rather than the implementation which should result in more maintainable systems which can readily be adapted to include enhancements and changes throughout their life. As the generated code is not the primary artifact which engineers work with, the quality of the generated code becomes less important. However, a key question to consider is does the use of MDD lead to the potential of (over time) of creating a decaying model and if so, how do you prevent it? And can you perform 'bad' MDD?

Reuse (of code snippets)

There are an increasing number of source code repositories (for example see here) now available on the Internet offering a variety of resources from simple algorithms to reusable components. All of these repositories help in providing a (hopefully!) proven way of solving a particular problem, and should also be written in such a way to be readable in case the snippet needs to be tweaked in the future. Although I cannot advocate that the quality of all snippets will meet the criteria of not being 'bad' code, it is reasonable to expect that the many will be examples of 'good' code. If the code snippets can be used 'as is' without modification the code should remain maintainable; if the code is modified, the style of the original code should normally be preserved so that the code remains 'good'.

Although the repositories do not require that the code passes any quality checks with regards maintainability etc, it should become obvious that the better code will be downloaded more frequently.

Use Multiple Compilers

I have always advocated compiling code with two different compilers as a way of improving code quality. No two compilers are ever the same as each one has different strengths and weaknesses. I have also always promoted 'clean' compilation i.e. ensuring that all code compiles without warnings once the set of compile options have been defined. If the code compiles cleanly with two different compilers, there is an increased probability that the code is well-structured, which IMO implies that the code will be more maintainable. It should also help testing as a number of latent faults can often be removed prior to run-time.

Clearly if the multiple compiler approach is adopted, this must be used at all subsequent code evolutions; if not the code will clearly decay albeit more slowly than if a single compiler has been used.

Conclusion

Writing maintainable software requires skill. With a bit of thought (and resisting the temptation to code the first thing that comes into your head), quality code can be produced using one or more of the techniques outlined which will support future product evolutions in a cost effective manner.

Some thoughts on Kanban in Software Development

2009-04-17T21:04:00.008+01:00

I recently watched a presentation given by David Anderson at QCon 2008 on his experience and observations of a kanban system applied in a software engineering environment. I found some of the observations very interesting, particularly the approach that optimising the system for lowest cost didn't actually result in the most efficient system in terms of optimal throughput. As with many great ideas, kanban appears to be a very simple concept; however clearly there is more to it than just a simple approach at managing a flow of post-it notes on a whiteboard.

I was keen to understand more about kanban and how it might be applied in various project scenarios. At the SPA2009 conference, there was a session presented by Karl Scotland on his take on kanban (Kanban, Flow and Cadence) together with a very interactive BoF session on Lean and Kanban. Whilst the comparison of lean, based on a model, originally promoted by Toyota, of optimising production, against kanban, which limits the work in progress was a simple and very understandable comparison of the two approaches, it was when the debate moved to seeing how both lean and kanban could be applied to software developments that the debate got really interesting.

Clearly software developments come in all shapes and sizes and the audience clearly represented a good cross section of developments and associated practices. Both approaches are clearly well suited to an agile approach in which features are developed to provide a flow of (increasing) value to the end user. This works particularly well in an iterative development where frequent delivery is encouraged, especially where the features are being evolved based on user feedback. This also requires a tolerant customer who can accommodate some failures. An interesting statement of 'get it right second time' was promoted as acceptable provided you learn from mistakes. I find that hard to accept as the 'norm' because it is completely ignoring any recognition of embedding quality in a delivered product. It might be acceptable in a prototype development but not in a production environment. Clearly there was something missing. There was! The integration step. Now I consider integration to be a very important (and potentially very expensive) stage, which is often squeezed in terms of time (how do you know you have finished?), and can't be omitted.

So could kanban work in a more traditional waterfall development approach? Maybe. In a traditional approach, the requirements are analysed before design, implementation and testing are performed. Using a kanban system, a flow of features could be pushed through the separate stages of design, implementation and testing before handing over to integration, with limits being imposed at each stage. This would probably require that the 3 life cycle stages are performed by different team members, a not unreasonable expectation in a traditional development. Provided the requirements are relatively stable and the ordering and independence of features is organised to support a sensible integration approach in order to deliver increasing value (or functionality) then this could be an interesting approach. Managing the queues at each stage of the life cycle needs to be carefully managed since the costs of managing each queue must be proportional to the size of the queue otherwise the process becomes very inefficient.

Now for the challenge. How likely is it that each feature will be take approximately the same time through the life cycle? - probably not. In my experience, it is highly unlikely that each requirement (or feature) will be equivalent in terms of effort expended, particularly when the software forms part of a complex system. This will result in some initial inefficiency as it would take some time for the queues to become populated with tasks. There is the temptation to process easy requirements first as a way of seeding the work queues quickly. However, from a project management perspective this is probably not what is required as it leaves the difficult tasks (and hence risks) to the end (and most difficult tasks will probably impinge on the easier tasks in some form resulting in some additional unplanned work).

Looking back at the David Anderson presentation, it is clear why a kanban approach worked well in one of the case studies since the approach was only applied to bug fixing rather than in a new development. I believe that looking at kanban in a multiple life cycle development needs a different approach for development with the queues being placed at different points in the system life cycle (as distinct from the software life cycle) to avoid bottlenecks building up. I would advocate that the same developer should be responsible for design, implementation and testing of each feature so that the queue limits are placed on the exit from analysis (or requirements definition) and entry into integration rather than within the different stages of software development. In a lean or agile development this approach is normal since the developers are performing all stages of the life cycle; in waterfall developments, and particularly large systems, this is not the norm.

Adopting kanban in waterfall certainly isn't new (see Kenji Hiranbe's article) but there is little evidence currently to demonstrate if it can deliver tangible benefits for all types of project approaches. Are there any case studies to prove or disprove my hypothesis?

Web Security - an eyeopener

2009-04-13T23:21:00.004+01:00

An impromptu Birds of a Feather (BoF) session given by Dave Misell at the SPA Conference admirally demonstrated what happens behind your back when you visit a website. Key to the demonstration was showing (very simply) how much data is passed between sites using cookies and how much data is retained (for considerable length of time in some cases) between site visits.

The session used the Paros proxy server with Firefox which easily showed the web transactions when entering a simple URL (I won't disclose the URLs used in the demonstration, suffice to say that the sites were well-known). Using Paros, it is easy to see how much information is passed via cookies. Stopping cookies results in the same information being passed via URL (although this is more obvious since the data appears in some form in the address bar). In many cases the same data is passed to a number if websites regardless of whether the data is appropriate or useful to the receiving website.

Although it is possible to see the information being sent to a website, it is not possible to determine what is done with the data by the receiving site. In most cases the data is used to trace a journey through website (so that the 'user experience' can be improved) which is totally transparent to the user since these are performed server side.

So is it possible to stop the information being transferred? Not easily, but choose your sites carefully. There is some legislation in the EU which has tightened up the exchange of information through websites, particulalry to third parties, without the express permission of the user. Unfortunatley this legislation is dependent on where the website is hosted, which isn't always obvious from a simple URL. There is always 'education' which applies to both the end user and also to the developers of the site, and there are some good courses now which can help (e.g. ethical hacking and MSc in Information Security at Royal Holloway College in London) increase awareness.

Can you scrum on your own?

2009-04-09T23:45:00.006+01:00

An interesting question posed by Peter Bell at this year's SPA conference. His premise was that some of the lean and agile approaches can be equally applicable when you are THE team. He called his approach Solo Scrum.

Peter identified a couple of scenarios where he felt that 'Solo Scrum' could be appropriate:

A full time developer doing open source development part-time
A solo consultant developing applications for non-technical clients
A 'work at home' contractor managing multiple clients

Clearly working on your own brings a number of challenges (note that I didn't say problems!) which need to be considered when working on your own. I think that probably one of the biggest challenges to overcome, (and this is based on personal experience), is to try and avoid too many distractions within the 'home' work area. The second challenge is that of loneliness! In the many (professional) development projects that I have worked, talking to colleagues has been a great way of confirming your thoughts for a solution or helping to find a way through a problem. When you are on your own, this isn't possible, or is it? However the use of technology can certainly help (think of IM, email, VOIP (with video)) and it can certainly be seen as 'virtual' pair programming. This isn't a new idea (see here); however, there is still the problem of finding a suitable 'pair' to link up with.

The traditional problems of planning don't go away when working on your own (as I said at the session 'Fail to plan, plan to fail'). Working on your own necessitates an efficient process for planning and managing the plan (a bit more than the 'back of an envelope' note). There are a number of Eclipse plugins (e.g. mylyn) which offer the right balance in terms of monitoring the work activity without distracting from the task-at-hand.

Some interesting ideas were discussed, including the observation that many team collaboration software packages often includes a free version for a small team (where small can be up to 5). This can overcome a often heard comment which was that I don't have the time/experience/etc to set up a 'professional' development environment (the minimum being a source control system, an IDE and a bug tracking system) from scratch.

Peter has now started a Google group to further the debate. I will watch with interest.

SPA2009

2009-04-09T23:29:00.004+01:00

Having attended last year's conference and thoroughly enjoyed it, I was disappointed that my work commitments meant that I could only attend a single day of this year's conference. Attending a single day meant that I was keen to catch as much of the buzz as possible as well as catching up with some friends. I managed to attend 2 sessions, on AJAX web testing using Selenium and Solo Scrum, together with 2 BoF sessions on Web security (or lack of it) and Lean/Kanban in a stimulating day. I will summarise my notes over the next few days.

Case Studies in Enterprise Architecture

2009-03-20T21:12:00.004Z

A few nights ago, I attended a fascinating evening with Wayne Horkan, the CTO of Sun Microsystems for UK and Ireland, who presented his views on Enterprise Architecture. The event was jointly organised by IET, the BCS Manchester branch with a pitch for the recently formed BCS Enterprise Architecture Specialist Group forming the warm act before Wayne's presentation.

Wayne presented his experience in 3 (anonymous) organisations who experienced difficulties in their approach to delivering an enterprise architecture. The problems could easily be addressed by recognising the key and important role of enterprise architects who maintain an understanding of the business context, provides domain knowledge and ensure that the architecture and supporting IT remains aligned with the business.

In assessing any programme, a simple 4 stage approach is followed:

Gather the facts by a series of interviews and information gathering. This includes understanding the current 'as is' enterprise architecture, the current IT/IS setup and the 'vision' for the future.
Analyse the data
Determine the key conclusions/recommendations and the way foward. This includes establishing the order of change to ensure that the changes deliver value to the organisation without destablising the exsiting organisation.
Assess the project against the five key areas of likely project failure:

Strategy. Is it really needed? Why is it needed now? Will it deliver a realistic ROI?
Contractual. Is the contract, including the financial elements, appropriate defined and achievable?
Process/procedures. Are we in control or out of control?
Politics and personnel. Is there buy-in from all concerned/affected?
Technical and architectural. The least likely cause of project failure (apparently) but need to ensure that the right resources are available in order to deliver the right solution.

Wayne concluded with a number of best practices based on his experiences with these (and other) organisations:

Ignore the religious arguments about which framework to use. Just pick a single framework and stick with it. Most organisations use a blend of Zachman (artefact focused) and TOGAF (procedurally focused). None of the frameworks provide a magic bullet with which success is guaranteed.
Obtain sponsorship from key stakeholders. The stakeholders must have a strategy for the business, which may change over time, that is communicated to the EA team.
The EA team must continually:

Remain delivery focused, providing value to the organisation
Involve the whole team (consisting of Enterprise, Infrastructure and Application architects) in all decisions, ensuring that cross-team communication is integral to the team (no silos)
Ensure that there is the right mix of technical skills to deliver the right architecture
Obtain and foster Sponsorship, Sponsorship, Sponsorship without which a programme is likely to fail
Keep aligned (and checking) with the business strategy

As a reality check, Wayne highlighted that the current economic climate now requires that the investment in enterprise architecture must be seen to deliver value far more quickly (i.e. in the matter of months rather than years) to the business than has previously been tolerated.

The session was well attended and left the audience with much to consider in future projects.

The slides, and Wayne's personal views on the session are contained here.

Going forward with PHP

2009-03-01T16:53:00.013Z

I run a number of websites which provide a daily update of a shareprice together with the value of the shareprice converted to a loical currency (e.g. £, $, Aus$). When I started, I selected Perl as my scripting language which provided me the necessary facilities (the LWP package) to retrieve a shareprice from the Yahoo finance site. I used the URL

http://finance.yahoo.com/d/quotes.csv?f=l1d2c1p2&s='.$stock_symbol

as my data page which returned the share's closing price and other useful information.

I also generated a graph which showed the historical share price, relative to one of the major indices (e.g. FTSE, CAC40).

As I couldn't host my scripts on my web site, I was required to run a number of Perl scripts to generate updated files and then upload the files to the webhost. I wrapped the scripts in a DOS batch file and set up a scheduled task (under Windows) to run the scripts every weekday after the markets had closed.

This worked very well, most of the time. However it suffered from a number of problems. If I wasn't around, and didn't turn my PC on, then the scripts didn't run and the shareprice wasn't updated. I also suffered from some problems with my web connection which meant that the scripts didn't run as expected and I obtained a zero shareprice.

I had been wondering how to make the system better and less dependent on my PC being turned on. Most ISPs that I have experienced don't allow the LWP package to be installed as part of the standard Perl installation, presumably based on security concerns. I therefore considered alternative scripting languages. I had already seen Python at SPA2008 and was keen to try and see Python was a suitable replacement. However, my web host didn't offer Python in its standard configuration, but did offer the other P language, PHP.

PHP treats a URL as a filename and does not appear (in the standard configurations installed on webhosts) to suffer from the security constaints that Perl installations suffer. A simple function retrieves the values for a stock symbol returning the parameters in an array.

function get_share_price($stock_symbol)
{
$url="http://finance.yahoo.com/d/quotes.csv?f=l1t1d1c1p&s=".$stock_symbol;
$filesize = 2000;
$handle = fopen($url, "r");
$arr = fgetcsv($handle , $filesize , ',');
fclose($handle);
return $arr;
}

This resolved the problem of updating a daily share price, and also offered the opportunity for offering live share price updates (subject to the market delays which free services such as Yahoo suffer - at least 15 minutes delay).

To resolve the historic share price required PHP again but this time with a different Yahoo feed which I had only recently discovered, the ichart interface which returns the data as a stream of CSV separated values.

function plot($numdays, $stock_symbol)

{

// Get current date

list($e,$d,$f)=split('-',date("j-n-Y"));

$d--; // Months are 0-11

$yahoostr="http://ichart.yahoo.com/table.csv?s=".$stock_symbol."&a=0&b=1&c=2003&d=".$d."&e=".$e."&f=".$f."&g=d&ignore=.csv";

$arrResult = array();

$min=1000;

$max=0;

$handle = fopen($yahoostr, "r");

if( $handle )

{

$n=0;

while (($data = fgetcsv($handle, 1000, ",")) !== FALSE)

{

if ($n == 0)

{

$n=1; // Ignore header row

}

else

{

$arrResult[] = $data;

}

fclose($handle);

// Select data subset

$arr1=array_slice($arrResult,0,$numdays);

// Determine chart scale

foreach ($arr1 as $data )

{

$min = min($data[4], $min);

$max = max($data[4], $max);

}

// Data is currently stored Newest-> Oldest.

$arr = array_reverse($arr1);

// Now plot data (date in element 0, value in element 4).

}

In processing the data, I discovered a number of additional PHP features, including

array_reverse -to reverse an array (required because the Yahoo data is returned with the most recent data first)
array_slice -to take a subset of an array
min, max - to determine the minimum and maximum values (how many times has code for this suimple function been written!)

This meant that I had the data in a format which I could then process to produce a graph (dynamically). Now the next question was to select a suitable graphing package which offered a good PHP interface. But that is another story.

The Power of the Door

2008-10-14T22:54:00.007+01:00

I have just read an interesting presentation on the rules of productivity. It presents 8 rules to determine the most productive number of hours per week (40), the type of office environment (team rooms) and how a team should be located (non-siloed - multi-disciplined). In most cases the results are possible counter-intuitive to most management but each of the results are backed up by sound evidence.

It made me think about my experiences on numerous development projects and I would tend to agree with the recommendations outlined in the presentation. I remember when I was working long hours AND studying for a post-graduate degree trying to write some very simple Eiffel code and failing to get it work after 2 hours of staring at a simple logic problem. I simply couldn't solve it at the end of a 15 hour working day. I went home and came back a few days later, refreshed. I solved the problem in 5 minutes. The lesson was clear to me then - you need sleep, not heroes. This was admirably demonstrated in an overnight session to get a demo working and after 8 hours through the night of being in a no better situation than we were when we started!.

As organisations have changed over the years, office space has become at a premium. The 'power of the door' was demonstrated when I was a young engineer doing a major retargetting exercise. I remember we had to convince our manager of the benefit of having our own server to work on this exercise - he agreed provided we could reduce the schedule by 6 months. This we did easily, not just because of having our own server but because we had a large wooden door on our small team office. If it was shut (which it was normally) people just walked past so you didn't get disturbed. I would guess that if we reverted back to small offices, the software industry would be much more productive than it is today. I wonder how many project managers have the power or insight to challenge the office environment and make the necessary changes that will increase the chance of project success and increase team morale.

The role of Project Managers in Agile Projects

2008-09-11T22:52:00.010+01:00

I recently attended Allan Kelly's presentation on 'Why and How to Become Agile', an event organised by the BCS Project Management Specialist Group. As you would expect from an experienced agile practitioner, Allan provided a good overview of what agile was and why he considered agile to be better.

However, given that the audience was mainly project managers of one sort or another, I was interested in his statements on the role of project managers in agile developments. Mainstream agile methods such as XP, SCRUM or Crystal are very silent about the role of the Project Manager. This doesn't mean that they can be dispensed with, it is just that the various agile developments concentrate on the approach to improving business value with a development rather than the associated management tasks. I know agile promotes self-organising teams (in my experience this is removing a hierarchy of developers, architects and testers) but I know of few organisations that don't allocate a project manager to a development project however small. I therefore advocate that the project manager role remains as critical as ever although the scope of some of his tasks may change.

Regardless of the development approach, all projects need to manage risk, budgets, communications and resources (physical and people). The classic management approach is for these tasks to be allocated to a (often dedicated) project manager so that the workers can get on with serious development work. I don't think there is any need for this approach to significantly change. One task which Allan indicated would change for the project manager was planning as the emphaisis on planning changes in agile developments to be much less formal. This may be appropriate for (small) purely software development projects but the vast majority of projects are multi-disciplined in which the dependencies between the various activities need to be accepted and understood by all parties regardless of the formality of capturing this information. The project manager's role still remains an important factor in the eventual success of the project and the choice of project manager is probably more important than ever. In my experience the best project managers for developments adopting some agile practices are those who are hands-on, are developers who have experienced agile first-hand, understand the project and are empowered to make decisions. It is the last point which mustn't be under-estimated; delays in decision making processes are classic signs of a development that is struggling and agile developments can't afford unnecessary delays.

I note that the latest DSDM version (Atern) now explicitly includes the role of a Project Leader (you can even get a qualification). Does this now recognise that the classic project manager now needs to more formally recognised in agile developments? I would probably say no; it is probably more a reflection on the type of organisations using DSDM who feel comfortable with an explicit role being defined rather a comment on the management of agile developments in general.

Now that agile development approaches have become more accepted as a 'normal' way of performing software developments, it is probably true to say that the role of the project manager has survived relatively intact. However the project manager now has a key role in being much more involved with the development rather than performing a purely managerial overview role; but this is what the best project managers have always done.

Agility for Complex Systems

2008-07-15T21:42:00.002+01:00

At SPA2008, there was much debate amount the use of agile practices in developing software.

A comment that I heard was 'that for all the noise that the agile community makes us believe that agile is mainstream and the most common approach in developing software today, the reality is somewhat different'. Certainly there are many good examples of where agile approaches have been successfully applied but I guess that many of these are in 'young' and 'smallish' companies and not in large, corporate companies which have many decades of history.

It is not easy to convince these large and well-established companies that there are alternative ways of developing software. One reason, is often the 'process' police. Large companies, in my experience, are very keen on following a tried and tested approach, with some limited tailoring to cater for different sizes of developments.

So what are the challenges in applying agile techniques to complex systems?

The Relentless March of the MicroChip

2008-06-20T22:22:00.007+01:00

I have just returned from the inaugural Kilburn Lecture which concluded a day celebrating 60 years since the first stored program computer (the Manchester 'baby'). The excellent lecture given by Professor Steve Furber gave a historical perspective of the major innovations which have originated from Manchester University over the last 60 years as the technology used in computing has developed together with a personal view of the developments which he has been personally involved.

Over the last 60 years, the technology has changed from the vacuum tubes used in the Manchester 'baby' and the Ferranti Mk1, the first commercial computer, through the transistor, which although invented in Bell labs in 1947 wasn't adopted for computers until the 1960's when the Atlas computer was developed, the fastest computer at the time, to the integrated circuit used in MU5 (a forerunner to the ICL 2900 series), Dataflow and Amulet systems. The Atlas computer also introduced the concept of the single level store, which is more commonly referred as virtual memory which one of the attendees at the lecture, Professor Dai Edwards, remarked that he still received payments for this invention. Each decade has seen the level of complexity increase as the number of transistors has increased which shows no sign of slowing down.

Steve also provided a personal perspective of his involvement in microprocessor design starting with the BBC Micro of 1982 whilst at Acorn. While the BBC Micro was primarily built from off the shelf components including the 6502 microprocessor, Steve designed two simple bespoke chips which helped reduce the chip count by 40. The success of the BBC Micro and its design led to Acorn developing further bespoke chips resulting in the Acorn Risc Machine (ARM) being released in 1985. This is probably one of the most significant microprocessor developments over the last 25 years as derivatives of this processor have now become a significant component in mobile phone technology. The ARM machine had a simple design, was small and had low power consumption and was really a System on a Chip (SoC). By 2008, there have been over 10 billion ARM processors delivered, making it the most numerous processor in the world. When Steve moved to Manchester University in 1990, he continued to use ARM technology in the AMULET system. Over various generations of AMULET, the size of the chips didn't change but the chips became more complex as the transistor spacing reduced from 1 micron in 1994, to 0.18 microns in 2003, which is smaller than the wavelength of visible light.

There was an interesting comparison of the changing energy requirements over the last 60 years. in 1948, the Manchester 'baby' required 3.5 KW of power in order to execute 700 instructions per second which represents 5 Joules/instruction. Contrast this with the ARM968 processor on 2008, which requires 20 mW of power and can execute 200 million instructions per second which would represents 0.0000000001 Joules/instruction. This represents a 50,000,000,000 times improvement! - there are few examples of such a dramatic improvement in energy efficiency. Steve did give a warning though as he stated that more efficient computing can lead to increasing power requirements.

No lecture on the development of processors can ignore Gordon Moore's seminal article (colloquially known as Moore's Law), and this lecture was no different. The original paper published in 1965 predicted the exponential increase in the number of transistors per chip only until 1975. However, this prediction is still true today and has become a self-professing policy and has become a key input into planning next generation microprocessor designs (see International Technology Roadmap for Semiconductors). The reduction has been achieved by shrinkage in transistor size, cheaper components and reduced power consumption. Steve cited that the current generation of microSD cards, which provide a flash memory of 12 GB contain over 50 million transistors in the size of a fingernail, demonstrates how much progress has been made since the original use of transistors in computers in such systems as Atlas. However, exponential progress cannot go on indefinitely and there are now some physical limits which will constrain progress. As components have increased in complexity, their reliability and lifetime (in many cases less than 12 months for some items) has reduced as the tolerances on such a great number of components are very fine. There is also a recognition that the cost of design to achieve advances in technology is becoming uneconomic as it is increasing at 37% per year. Steve considers that Moore's Law may survive for another 5-10 years with current technology but there is will need to be advances in alternative technology for it to continue beyond this (and there is no signs of this happening at the moment).

The current generation of microprocessors, dual core/multi-core, have tried to address the increasing constraints by selling more processors per microprocessor rather than a single faster processor. Moors' Law can also apply to multi-core systems however there is an increasing problem with the use of such processors by application software. As single processors have increased in performance, there has been little change to the way software has been developed. However, with the advance of multi-core technology, general purpose parallelism is required in order to maximise the available processing capacity. This is one of the 'holy grails' of computing and it is becoming an increasingly important problem to solve. The use of such multi-core processors also needs to be carefully considered. It would appear that is preferable to have lots of cheap (and simple) cores rather than a small number of faster (and complex) cores due to the significant power efficiency differences.

Steve finished by giving us a glimpse into the future as hew showed how microprocessor design was converging with biology. The human brain has many attributes which are similar to the requirements of a complex network of computers e.g. tolerant to component failure (e.g. loss of a neuron), adaptive, massively parallel, good connectivity and power efficient. Steve's current project SpiNNaker is trying to build a system which can perform a real-time simulation of biological transactions mapped onto a computer architecture. The project is using 1000's of ARM processors and is trying to meet one of the UKCRC's Grand Challenges in which the architecture of the mind and brain are modelled by computer.

Assessing code quality

2008-06-06T23:27:00.004+01:00

How do you assess a software modules's quality? It is a question I have been struggling with for some time as I try to perform a peer review of a large code base.

Over time, a software module evolves from its intended form to something less than beautiful as bugs are discovered (and fixed) and enhancements over the original requirements are implemented. This is particularly true for code which is developed on a multi-person project, where personnel can change and often a module gets changed by different engineers. Although I adhere to the rule, that the code structure should reflect the original author's style (and how many people change the comment at the top of the file to identify that they have been one of the author's? This assumes that this information isn't automatically added by the configuration management system.), it can become increasingly difficult to make changes.

So what is the best way to assess code quality throughout it's development?

It's good to be free

2008-05-01T22:02:00.027+01:00

I have just returned from the BCS Manchester event on The Free Software Movement in Ethics and Practice presented by Richard Stallman, the founder and president of the Free Software Foundation. Richard, speaking for over 2 hours without notes or any visual aids, captivated the audience of over 250 with his passion about establishing a world in which a user's freedom is respected.

Traditional software was compared on it's functionality, cost and practicality and never considered 'what does the software do to my freedom?'. Conventional software does not respect a user's freedom as the user is utterly helpless as he can't change it (because he hasn't the source code) and the user has to use the software in the way that the developer deems appropriate.

To set the context, Richard identified four freedoms which are required for software to be free:

Freedom 0 - Being able to run the program as you wish

Freedom 1 - Being able to study and change the source code to do what you want

Freedom 2 - Being able to help your neighbour by distributing the software

Freedom 3 - Being able to contribute to the community by sharing and distributing the modified/evolved code to neighbours.

Any software which does not meet all of these freedoms, should be considered proprietary and could be considered unethical as it isn't contributing to society.

Freedom 0 is essential so that you have control of your computing. There are many applications which restrict who (named users), where (specific computers), when (whilst valid licence held) you can run your (purchased) program. The control is often not via a license but via specific code which determines what can and can't be done.

Freedom 1 allows to determine what the application is doing to you. There are many instances of applications which spy on you; this is a growing trend particularly with the always connected society connected to the Internet where you are offered (or in some cases not) updates to your software. Failure to comply with this freedom demonstrates that the applications are actually controlling you and not working for you. Examples such as Digital Rights Management (DRM) and Microsoft Vista are tremendous examples of advances in restricting users! As there are millions of users who can never use this freedom, as they don't have the necessary skills (i.e. they aren't programmers) and so much software, Freedom 3 was defined to recognise the need to work together as part of a community.

Freedom 2 is a basic moral right which proprietary software prevents. There is much propaganda with regarding distributing copies of software by calling this piracy. Therefore helping a neighbour is equated with attacking a ship; Richard thought that pirates used guns to ply their trade rather than software, so it is a very strange use of the word pirate!

Freedom 3 is all about working together and to allow co-operation as changes evolve. This results in co-operative progress regardless of the original intent of the software.

With software conforming to the 4 freedoms, everyone gets the benefits and society reaps the rewards. The software is developed under the control if its users with no single user having all the power. Free software cannot have any malicious features; if such a feature were added, someone else will take it out (and the originator will get a bad reputation).

After extolling the virtues of the 4 freedoms, Richard moved on to explore the Business View of Free Software and how businesses can benefit. An instinctive view is that free software could be bad for business, particularly software developers. This is not true, because whilst there are a small fraction of businesses developing software who may be affected by the availability of free software, there are many more businesses who use software which is vital for their survival who can benefit hugely and take advantage of the 4 freedoms. One area where businesses can benefit is in the area of support and services. With proprietary software there is an essential monopoly with little incentive to fix problems etc (typical response - we have a new upgrade in a few months time, see if that has fixed your problem (and introduced a few more)); for free software there is the option for free market support as well as a choice of paid for support from multiple suppliers.

Free Software does not remove the need for custom software development as software development (specifically programming) is a small proportion of the overall software development cost. Free software could actually generate jobs in the adaption and extension of existing software, which may be commissioned and paid for, even though the resulting software is to be subject to the 4 freedoms.

A clear distinction was made between Free Software and Open Source Software (OSS) as there is often the perception that they are the same. OSS was introduced as a distinct phrase in 1998 to distance some software from the user's freedom. OSS has different philosophical values to FSF and does not offer anything specific to respect a user's freedom.

Richard's presentation was extremely compelling and thought provoking, and whilst noting that user's freedom isn't easy to respect, there are approaches which can lead to a better society.

When do you measure software complexity?

2008-04-21T14:09:00.006+01:00

In my day job, I often have to review a number of software artefacts including software architectures, designs and code which form part of a large system. As the system is to last for many years (it is not uncommon for some of the components of the system to last for more than 10 years), I have always been keen to ensure that the artefacts are understandable by more than the creator and the reviewer, and are capable of being maintained for many years. One measure I have considered to assist this is software complexity particularly as one of the big issues in software maintenance is that as code evolves from its original form and is modified by more engineers, the complexity of the code is likely to increase unless it is actively monitored.

While assessing the complexity and understandability of architecture and design still requires some manual effort (and based on experience), one measure I have used for measuring code is the cyclomatic complexity metric which has been around for over 30 years and is used to calculate the complexity of a program (actually it is a method or function of a program) by measuring the number of linearly independent paths through a program's source code. The advantage of this metric is that it is applicable to many languages and relatively easy to calculate. There are recommended values which should be used as the thresholds with any module over 20 being considered for refactoring.

As with any metric, the key is when the metric is applied. I see that there are several choices:

Calculate the metric continually as you code. The advantage of this is that you are immediately aware if you are exceeding the agreed thresholds and can therefore not exceed the agreed complexity measure. However, this approach becomes very invasive as code is edited.
Calculate the metric as the code is compiled with code exceeding a particular threshold failing to compile (and hence failing to build the application). This is a simple way of ensuring that the complexity is not exceeded but can become frustrating if you are trying to come up with a quick fix to solve a problem.
Calculate the metric when you check-in the code to a configuration management system with any modules which exceed the agreed threshold being rejected. This is less invasive than calculating during the coding or compiling (assuming that you don't check anything in to you configuration management system that doesn't compile) and can be integrated with a number of other software metrics which can be calculated from the configuration management system.
Calculate the metric as part of the testing phase. This is probably the most common approach but is also the least appropriate. Depending on the testing strategy, it is possible that the code will already have been 'proven' to work so there is a degree of reluctance to change 'working' code.

Once the metric has been calculated, and some transgressions are identified, the code needs to be examined to identify the cause for the complexity. In many cases this is due to a poorly abstracted method or large monolithic code but there are some instances where the natural coding style will lead to the threshold being exceeded. This is particularly true where a switch statement is used. There needs to be a (manual) judgement regarding whether the code should be refactored bearing in mind that one of the benefits of examining complexity is identifying the amount of testing that may be required. Refactoring a method with a complexity measure of 20 could easily be reduced to two packages each with a measure of 5. which will be far easier to test particularly if you have a strategy of 100% code coverage.

Getting the badge

2008-04-15T21:26:00.005+01:00

I have just come across Certified Open, an organisation which is trying to help measure and encourage competition in the provision of hardware, software and services. The aim, as I understand it, is to allow purchasers to evaluate various products and determine which ones will result in commercial lock-in, which may or may not be desirable depending on the intended use and evolution of the product

The assessment has a set of questions with which the product can be scored. Determining on the final score, the product will be awarded Gold, Silver or Bronze assuming a score of 50% or more is achieved. It would appear that each question has a number of options which will have an appropriate weighting assigned to help develop the final score. Now I have no issue with allowing a vendor performing an assessment against his product as it will no doubt assist in identifying areas of weakness which may merit further work. What I am unsure about at this stage, is if there is also an independent review of the assessment as some of the questions are clearly subjective and it is easy to look at your own product in a good light which may result in a good score. The assessment also says nothing about how suitable the product is if it is used as part of an overall system or solution and how the product may be evolved. Interoperability is also key; just conforming to a well defined standard may not be enough if there are various versions of a standard.

The approach to assessment is vary similar to that adopted by US Navy for the Open Architecture Assessment Tool (OAAT) in evaluating products as part of a naval defence system. However, this assessment covers both the technical aspects of the product together with the management and evolution of the product. Now this assessment is clearly aimed at a specific market which is looking to protect its investment in complex products.

I wonder how useful these assessments can be.

Some thoughts:

Should the assessments be performed by an independent authority?
Will the assessments encourage speculative product development if 'getting the badge' is seen as a key product discriminator in a particular market?
Are there any other examples of product assessments which have resulted in products being developed specifically to meet assessment criteria?

Any comments?

SPA2008 - some final thoughts

2008-04-02T22:45:00.005+01:00

It is now 2 weeks since the end of the SPA 2008 Conference and I have finally finished writing my notes. (Yippee!) So what are my final thoughts and what I am going to take away for exploring in the next few weeks and months?

I thought the conference was excellently run with a good mix of software practitioners attending. I was spoilt for choice in selecting the sessions to attend but there are some good notes appearing which summarise the sessions I didn't attend. There are also a number of articles appearing on various blogs. The dialogue during and between sessions was also stimulating and I hope to continue this when time permits.

Technically, I am particularly interested in following up the following topics:

Python, Django and other frameworks

Erlang, particularly as part of a multi-language development probably with C, C++ or Java

Further development of metric dashboards (which is work in progress ) now that I have got some further input to consider

Look more at DSM, DSL's and domain specific code generation

Decaying architecture and legacy code bases, and attempting to detect when to intervene

I also learnt a lot about me. The conference confirmed that generally, the work I do is pretty well in line with what the rest of the software world in the UK is currently doing (or tyring to do!). There are differences but these appeared to be related to the domains, size of developments and team dynamics.

Describing Software Architectures

2008-04-02T21:54:00.002+01:00

My last session at the SPA 2008 Conference was a workshop to explore the approach to describing a software architecture.

Nick Rozanski, Eoin Woods and Andy Longshaw asserted that any architecture requires a lot of selling with many different stakeholders and customers interested in the architecture. Communicating the architecture is obviously key and a group session explored what should be produced in an architectural description document. The varied experience of the groups produced a diverse set of content, with the more experienced architects recognising the importance of a clear baseline of requirements from which the architecture can be derived and confirming an understanding of how the delivered system is to be used. Clearly, the project size and life cycle approach has an impact on the production of the architecture; projects following a traditional waterfall life cycle (tending to be part of larger programmes) placed a increased emphasis on the requirements whereas the more agile projects placed much less emphasis on this and were keen to start producing some tangible software. This discrepancy also appeared in another SPA 2008 session that I attended.

There was also an interesting exploration in the tools and techniques used to produce the content of an architecture document. A recurring message was that the document must be in a format suitable for all of the readership, so clear preference for using a variety of standard office based tools such as MS-Word, Powerpoint or web pages on an Intranet rather than specific architecture tools e.g. modelling tools for UML which may not be available to all readers (one solution to mitigate this would be to ensure that such content is exported into one of the more accessible formats e.g. HTML). There was the recommendation that the tools used in the architecture should form part of a integrated toolset with good traceability between the requirements, architecture and design being available. The availability and use of such integrated toolsets appeared to be limited to larger programmes, particularly where the software was only a part of the eventual solution.

Once an architecture is produced, it needs to be publicised and communicated to everyone who is impacted or interested in the architecture. While there was general agreement that the architecture must be peer-reviewed before this wider communication is performed, the approach to communicating (or socialising, as Andy called it) the architecture varied from simple 1-1 chats to major presentations. What was clear was that the architecture must continue to live and that it needs to be actively maintained, remembering that the original approach of selling the architecture also applies to any subsequent changes.

Overall, I felt the workshop confirmed that the approach to software architecture that I follow continues to be appropriate and picked up some good ideas for trying on a future architectures.

A summary of the output produced by the workshop is available on the SPA 2008 Conference wiki.

Automated Testing Experience

2008-04-01T23:52:00.008+01:00

Keith Braithwaite presented his recent experience of automated testing, particularly using Test Driven Development, at SPA 2008 Conference.

The audience, a mixture of youth and experience, had already some good experience of automated testing tools such as Xunit, Selenium, Cruise Control, jMock, dbUnit and Build-o-Matic (funny how all of these tools are open source).

Keith outlined some of the problems that he saw with manual testing such as:

Takes a long time...

Not consistently applied

Not particularly effective

Requires scheduling of people

Keith advocated automated testing as a better way with a clear view that 'Checked Examples' need to be performed automatically. Before describing his approach he went back to the basis of software engineering with regards how the system requirements were specified -natural language which is often vague and ambiguous. He advocated that unless the customer talked the same language as the engineers, then there would always be problems in building systems. One of the major problems in systems are specifying the boundaries or constraints of the system to be developed; writing these rules precisely was often very difficult with multiple exceptions to the norm. However, the customer could often give examples of 'what the system should do' - these weren't the rules but were sufficient for the reference results to be manually calculated by hand. There was no guarantee that the reference results were correct but by ensuring that there were multiple contributors to the production of this data, the probability of erroneous data was reduced.

With reference data available, a test framework was developed using the FIT library which was then used to automatically compare the results from the system with the reference results. At the end there were several hundred scenarios (or examples) used, with the test data being available before the features were developed, thereby facilitating the incremental development. Keith stated that the approach using reference data and automatic comparison of the system against the reference data detected problems early and also revealed issues with the design (if it couldn't be instrumented, then the design was questionable).

Keith stated that the delivered system was defect-free with no failures being reported.

Is Defect-Free possible?

Developers make mistakes and errors, ...so

The system contains defects,... so

The user experiences failures

So, if the user doesn't experience failures, does this mean the system is without defects? Probably not, but this is hard to verify. I would agree with this by saying that it is very difficult to claim that any development is defect free, only that the defects haven't revealed themselves during the testing and normal (and presumably intended) operation of the system. It also depends on the impact that the failure has, as clearly some systems have a more severe impact than others. By adopting Keith's approach of developing reference data with the customer, the potential for misunderstanding the system before delivery is reduced.

Keith summarised by saying that the tests weren't really tests at all, merely gauges of how the system is working for the users. The reference data were 'checked examples' which were a very powerful way of ensuring the customer and developers worked together.

The Economics of Legacy Code

2008-03-31T23:21:00.002+01:00

The final keynote for the SPA 2008 Conference was given by Michael Feathers from Object Mentor on the economics of managing legacy code.

Michael set the scene by mentioning some classic references including Freakonomics with regards behavioral economics, the Social Atom with regards how people behave in groups and the Big Ball of Mud which describes a common software architecture, the haphazard or casually structured system with a challenge 'do we believe it or do we ignore it?'.

He compared a Normal distribution curve with the Power Law curve stating that a power law appears in many aspects of life where preferences can be made. He then cited that if a distribution of method size across a project is plotted, it follows a power law and not a normal distribution. Intuitively you would expect uniformity with a clean design but suggested that this might be unnatural (maybe this is boutique design?). Personally, I think that unless the development process is actively managed, there will always be differences in the approach developers, primarily based on previous experience; this is a demonstration of the creative process of development.

There are some obvious reasons why organisations have difficulties in maintaining code:

Too many ways of Solving the Same Problem

As an example of showing that there are many ways of solving the same problem, Michael cited Perl where there is little consistency between two programs as there are multiple ways of doing the same thing. This clearly leads to difficulties in maintaining other people's work.

The Organisation

Another law, Conway's Law demonstrates that the software reflects the organisational structure that produced the software; it is often easy to see that different teams have worked on the project (see later for a personal view on this). Michael posed the question 'Can organisations align with the software to be produced?' and hence produce more maintainable code. Maybe....

Public bodies have a short-term view as they control expenditure
Organisations in the 'competitive milieu' have a number of options:

Software impacts the bottom line such that some changes may make the software better and hence benefit the organisation
Hyper-competitive organisations where software becomes a mess as the organisation strives
Monopoly organisations become lethargic asking 'why does it matter?'

Within a development team with dedicated people assigned, it is highly unlikely everyone will have an equal workload. Michael suggested that development should be subject to 'collective' ownership . This is one way of recognising that there are never enough good people to go round. This led to a discussion on professionalism which stirred up some interesting views.

My comments

I believe that code should be seen by multiple developers with continuous review. Pair programming achieves this with 2, but other practices can also lead to multiple views of the code with a reduced tendency to claim code as a personal asset rather than a team asset

Ensuring code is maintainable is not normally the primary goal of any developer; it is to get the task in hand done, hopefully on time. However, if there are good development practices defined at the start of the development which are managed consistently then there is a greater chance of achieving maintainable code. Given the increasing tendency to have off-shored development teams, then the managing the consistency of the development almost certainly mandates good tooling. As another session at SPA2008 showed, if this isn't done then the architecture and design will decay until it can no longer be maintained.

Smelly Software Architectures

2008-03-29T21:48:00.006Z

At the SPA 2008 Conference, Mark Dalgarno led an interesting workshop which explored the deterioration of software architecture over time. Mark outlined a number of different conditions which could indicate that the software architecture (assuming that there was one to start with!) is decaying from the as-intended architecture. There was an interesting debate about the importance of architecture and there was a clear distinction about architecture minimalism from those practicing agile techniques, to those engaged in large and complex systems of which software is only a small part of the overall solution.

I firmly believe that a key principle is that any architecture should be resilient to change and should be an enduring artifact. It should also be flexible and scalable, although this can depend on the expected lifetime of the architecture (architectures that are only expected to last for a year or two clearly don't need to exhibit the same attributes as an architecture which is intended to last for 10 years or more). This also requires that the architecture and design are seen as distinct activities - I have seen many examples where the architecture has been assumed and the detailed design and implementation has proceeded without any thought about the architectural options or future maintenance requirements.

How an architecture can be assessed was an interesting discussion with metrics being used (e.g. dependency counts) or consideration of some potential change scenarios although the more agile developments considered this to be unnecessary. Example scenarios were offered such as a change in the operating system version or a database upgrade and the resilience of the architecture to this change. However, it is clearly unachievable to think of all such change scenarios that could occur. Some examples of changes that weren't originally anticipated but were then implemented with some significant pain included the addition of error messages in multiple languages. The impact of a project driven by time pressures (time driven coding) were also likely to lead to a less enduring solution and a less robust architecture.

The economics of architectural decay was considered with a view that 'rewriting is considered harmful' particularly for solutions which are deployed widely where there is no acceptable alternative to maintaining what is currently implemented. While I sympathise with this as a view, any maintenance regime should always consider UUD (upgrade, update and disposal) of an in-service solution and to recognise that there may be some time when multiple versions of the same solution operating on potentially different architectures may have to be maintained in parallel.

While the session didn't offer any solutions to a difficult problem, it did pose some interesting thoughts which will require some further consideration for future software architectures.

Hello World in another language

2008-03-28T23:38:00.000Z

The SPA 2008 Conference provided me with an opportunity to see the functional language Erlang for the first time. This isn't a new language (see panel session) having been in use for over 20 years in Ericson for applications such as PABX systems.

As with all programming languages, the Erlang version of 'Hello World!' was demonstrated but the more interesting examples were those which highlighted some of the features of Erlang that make it standout from the other languages. It is clearly particularly well suited to massively concurrent applications as it supports 'cheap' thread creation with data being shared by messages. A simple demonstration in calculating Fibonanci numbers showed. It also showed how crucial the implementation of a multi-threaded, parallel application is. Ensuring that the implementation was tail recursive created an application that returned a result in a few seconds; not doing this resulted in the result taking much longer.

With the move to multi-core architectures, efficient use of the cores through parallel applications becomes much more important. In the classic article, The Free Lunch is Over, Herb Sutter outlined that concurrency is the biggest change to impact software development since the adoption of OO. There is a fundamental change required in how programs are written, impacting good principles such as modularity, to ensure that the applications can take full advantage of the available processing resource. Erlang's approach has been to support parallel processing from the start and to make concurrency easy through the use of the Actor Model (contrast this with the approach taken by Haskell which uses Software Transactional Memory). Erlang's approach is extremely efficient and lightweight, and performs asynchronous processing; there is no guarantee on the ordering of messages.

Erlang clearly has a very different approach to any concurrent programming I have done in other languages such as C, C++, Ada, Java etc. In the short session I couldn't see many other benefits other than this but maybe this is just encouraging me to go away and experiment. I would be particularly interested in seeing how Erlang could be used in a mixed-language application, say Erlang , C++ and Java - I have already found an example which discussed the use of Erlang and Java.

Erlang also got me thinking. Many years ago, shortly after I graduated from university, one of my friends enrolled for a PhD in which he was to develop mathematical algorithms (specifically ordinary differential equations) to run efficiently in a multi-processor environment. His language was FORTRAN (there were other languages around at the time but FORTRAN was the dominant scientific language at the time) and he managed to get a number of the algorithms operating efficiently in multi-processor configurations. I wonder if Erlang had been around at the time, if it could have been used and would have made his job any easier?

Metrics that are useful

2008-03-27T23:07:00.008Z

As an experiment, I ran one of the BoF sessions at the SPA 2008 Conference on the topic of 'Metrics which are useful'. An interesting discussion ensued in the group consisting of academics, software practitioners and quality specialists.

The following is a summary of my notes:

Why Capture Metrics and what are you trying to achieve?

Purpose of metric capture depends on customer and business

Provide 'Bird's eye view of projects'

Used to improve quality

Used to provide evidence to support quality measure e.g. CMMI

Some thoughts on metrics (good and bad)

SLOC (source lines of code). Easy to calculate once agreement has been made with regards 'a line of code' but considered not good measure. Not particularly appropriate when system includes COTS components as part of solution. SLOC can change depending on language choice. No incentive to encourage reuse (see later), abstractions etc and can lead to excessive cut n'paste.

Coupling/Cohesion of interfaces as a mechanism for showing well-designed modules which can be reused

Use Metrics to monitor 'right first time' during integration. Key issue is how and what to measure. Can also be used as a measure of achievement by Project Manager
Monitoring reuse but difficult to measure or demonstrate. Potential measure could be number of hours saved. Reuse is dependent on expertise, team, business processes and functionality.

Measuring quality of code by examining use (or not!) of framework primitives and higher levels of abstractions.

Number of dependencies (Java) – the more a class is used, the greater the chance that bugs could have been found. Also consider number of interfaces used by component.

Number of tests passed. Not a good measure as it says nothing about requirements achievement. Better measure would be number of requirements passed – each test would have to reference the requirement(s) which are being tested (in part).

Measurement source code changes between different phases e.g. Unit test and Functional Test
Code coverage and number of tests (and completed) are not particularly useful. Code coverage for TDD is always 100%.

Measuring capabilities delivered can be useful metric particularly when adapted to meet business needs.

Key Performance Indicators (KPIs) often used for measures of performance across diverse set of projects (i.e. Not just software)

Using Metrics

What to do with the data once calculated/presented. Metrics must be presented in an easy to understood form (examples include traffic light reports with appropriate thresholds for each colour, graphs (are these always clear?)).

How frequently should the data be reviewed and corrective action instigated? Time should probably be a function of the size of the project/development and the anticipated development time. Small projects may be appropriate to measure daily (probably as part of an overnight build). Other projects it may be more appropriate to record on a weekly or monthly period depending on the likely changes between each report.

The cost of measuring/calculating the metrics should be negligible.

Metrics are always a snapshot. Need to examine the trends/dynamics rather than the absolute values and take appropriate action if trend is going in wrong direction. Doesn't matter if you have 1000 bugs this week, what matters next week is that you have less than 1000 bugs!. Any thresholds need to be appropriate to the project and reviewed and revised continuously. New developments may be able to start with a threshold of no compilation warnings when all code is new; this threshold might not be appropriate for a legacy system.

Some Recommendations

Measure the business value and not the code. Measuring something that is significant to the business is more important. Examples include how many times has the software been used?

Measurement must be understood by EVERYONE (and described in appropriate language), easy to calculate (i.e. Not subjective) and explain what it means to the business. The types of metrics to be selected depends on the type of organisation (and the business structure) and frequently change!

Metrics should be used to encourage good practice (e.g. Reuse, abstractions, frameworks) and not to use to punish offenders!

Starting a project from scratch is ideal to set good practices. However, regardless of where metrics are introduced, the key practice is to monitor the dynamic nature of the project.

What's the point of Software Architecture?

2008-03-27T20:38:00.006Z

At the SPA 2008 conference, I attended a session entitled 'Does Architecture Help?' in which Eric Nelson from Microsoft identified 9 observations on architecture that he has encountered from his work with Independent Software vendors (ISVs). He noted that there was no correlation to project success if there isn't an architecture and that there are many different ways (read architecture) of solving the same problem. Although there isn't a perfect architecture, there are clearly examples of bad architectures, which are often demonstrated by symptons such as change being very expensive, that have delivered a great service possibly through the massive advances in hardware technology being 'the get out of jail card'.

The rise of agile developments has clearly impacted the importance of architecture with perhaps the removal of the formality of this activity. Clearly the availability of technology can have significant impacts on software architecture. While there is always the tempatition to try and accomodate the latest technology in your next project, I think this is a risky strategy as architectures normally need to be based on firm foundations (there are always exceptions). With a good architecture (not a perfect one!), it is possible to keep technology choice separate from the architecture. In my experience of Model Driven Architectures (MDA), it is possible to separate the technology from the architecture through the use of separare models:

Platform Independent Models (PIM) which is independent of a the technical choice

Platform Specific Model (PSM) in which the technical choice is made

The slides for the session are available from Eric's blog.

I discovered something when I downloaded the slides, a new file extension .PPTX. My computer didn't recognise the extension (I only had Microsoft Powerpoint Viewer 2003 installed). A hunt around Microsoft's website revealed a free download for Microsoft Powerpoint Viewer 2007 which adds the capabilty to view .PPTX files, which I discovered are Powerpoint files stored in Microsoft's new XML file format.

Are we nearly there?

2008-03-26T22:43:00.000Z

The first panel at the SPA2008 Conference addressed the interesting topic 'Is Software Practice Advancing' with particular reference to the last 15 years. In summary, the panel said 'Yes, just!' after considering programming languages, software design and project management.

There hasn't been much advances in programming languages over the last 15 years (new languages such as C#, Javascript were cited) although some languages have become more popular (e.g. Haskell, Ruby) and others are now only used in maintenance tasks rather than new developments. Tooling for languages has improved with such toolsets as Eclipse, Visual Studio and IntelliJ aligned with better compilers although it is debatable if this has produced any significant change (reduction?) in development costs. There is also an increasing number of open source tools which are free which is a noticeable change for development. Programming is still unpredicatble and the risk of things going wrong hasn't changed. While languages haven't changed, there has been some significant improvements in libraries and frameworks which lead to some gains in the development life-cycle.

COMMENT: I would say that there are an increasing number of languages which are now available and supported due to the convergence on to two development platforms (Unix (and all flavours of) , and Microsoft) as the use of proprietary systems has diminished. However, the days of the coder maybe numbered as an increasing number of tools can now auto-generate significant amounts of code.

Software Design also hasn't changed significantly in the last 15 years although the debates about OO notations being finally resolved with the release of UML. Maybe what has changed is the way software is now assembled as there is now acceptance that using open-source components and 'glueing them together' is more than adequate for many applications and demonstrates good software re-use. More complex (and better?) systems are now being produced although the design skill level hasn't changed; it may actually have decreased as design becomes unfashionable as the need to 'write code and get something working' results in formal design processes being side-stepped. The production of software architecture has improved and is recognised as being of value to all developments.

COMMENT: The rise and maturity of the open-source is now having an increasing influence on software developments - the development and adoption of standards has probably helped this. The rise of the web has probably contriubuted to the loss of design fomality. There is still no substitute for expereince when designing systems. The use of tools to support the design process, particularly for large systems which are adopting model-driven developments, could lead to some changes in the perception of design and result in changes in effort profiles for software developments as the emphaisis changes to design and integration rather than coding.

The final section of interest covered project management where it was remarked that some techniques have improved, the practice is still poor. Projects have got bigger and bigger and the probability of failure has only marginally decreased over the last 15 years (still >70% chance of failure). The widespread adoption of agile and iterative life cycles over the traditional waterfall life cycle has not been as dramatic as the noise made by the agile community would have us believe.

COMMENT There is no precise definition of what Project Management is as it can cover a vast array of activities depending on the size of the development. Some approaches have been made e.g. OGC, PRINCE, DSDM to try and formalise the activities but these don't adequately address a badly specified and procured system which are often root causes of many project failures.

How DSM can improve productivity

2008-03-25T22:11:00.008Z

At the SPA2008 conference, I attended a session on Domain Specific Modelling given by Juha-Pekka Tolvannen from Metacase. I increasingly believe that the use of full code generation can lead to significant improvements in productivity, and if appropriately managed, code quality. I was therefore interested in Juha-Pekka's assertion that productivity increase is achieved not by using particular high-level languages such as C#, Java or Python or design approaches such as UML, but by increasing the level of abstraction. There have been some attempts through the use of frameworks, patterns and libraries which can help the abstraction level provided that the artefacts are used appropriately.

However Domain Specific Modelling (DSM), which is creating a language for specific purposes with automatic code generation, can lead to bigger gains in productivity when compared with normal development practices (claims of up to 30 times the productivity) with the code generation resulting in 50% less bugs than manually written code. The productivity gains need to be considered with the cost of the creation of the specific DSM and the number of times the DSM is to be used.

The cost of the DSM development (typically one or two developers together with a number of domain experts) and the number of applications over time can then determine whether a DSM will deliver the required productivity gains.

The examples which were cited clearly demonstrated the power that DSM has in producing production quality code quickly and efficiently. It is clear that for some large development projects, there would need to be many different Domain specific models created to support the development, and that it is highly unlikely that a whole development could be completed entirely using DSM. Clearly some further research is required to determine the categories of applications for which DSM should be considered - off to visit the DSM Forum.

Learning another language beginning with P

2008-03-24T22:47:00.006Z

I arrived early at the SPA2008 conference so that I could attend a fast-paced (not my words!) workshop given by Nick Efford on Python and how it could be used to develop web applications. Having experience in using the other two P languages for the web (Perl and PHP), I was keen to see what made Python a better choice than the other two.

Nick started by covering the fundamentals of the language, explaining some of specific features of the language that were different to other languages including dynamic typing and the importance of layout as a way of implying structure (e.g. no } brackets for if statements, just ensure that the if body is indented). Clearly one of the strengths of Python is the ability to efficiently handle and operate on sets of data as tuples or lists.

After a explanation of OO-Python, a demonstration of the neat way that documentation can be created for classes and methods (embedding a method/class description in """) and a quick overview of the Standard Python libraries provided; Nick moved on to the various implementations of Python that are available including Jython (a Java version which currently lags behind the 'native' Python), IronPython (for .NET), CherryPy and Django.

Nick is obviously a fan of Django and demonstrated very quickly how a web-based application could be created quickly using the Django toolkit although I couldn't get it working on my Vista Laptop. One neat feature of Django that particularly impressed me, was the Admin interface which is automatically generated when you create a project. I also like the approach to documenting Django through the publication of both a paper and web-based book. Clearly I need a project to try and experience the power and efficiency of developing web-based applications using Django for real.

In summary, I felt that Nick had done a very good job at highlighting the advantages of Python and left me keen to try and experiment further, particularly with Django. At the end of the session, I left knowing a lot more about Python than before and I feel I will soon be able to add another language beginning with P to my kitbag knowing that there will be some applications where Python will make an excellent implementation language. It is also clear that within the SPA community, Python is a must-know language as there where many sessions in the conference were Python was cited as the language of choice to implement certain tasks.

SPA 2008

2008-03-23T21:24:00.003Z

I have just returned from attending the BCS Software Practice Advancement annual conference (SPA2008). It was my first visit to this conference and found it a very interesting and stimulating few days. As with many conference programmes, there is always too much to try and attend, so I tried to stick with the sessions about architecture or languages I knew nothing about (in this case there were sessions on Python and Erlang). I even got the chance to try my hand at running a BoF session on Metrics which were useful , which resulted in an interesting discussion on a number of metrics which weren't very useful! After each session, there is always valuable discussions with fellow attendees; in many cases there was further opportunities to explore the topic further or to perform some further personal research.

Over the next few days, I will post my summaries of the various sessions and keynotes that I attended together with some further thoughts.