tag:blogger.com,1999:blog-18462731045538803522024-03-13T15:25:53.284+00:00Notes from a Software Architect, on a small islandPoints of note or issues which are or appear to be importantMehttp://www.blogger.com/profile/09171556704204860197noreply@blogger.comBlogger42125tag:blogger.com,1999:blog-1846273104553880352.post-67823345276555524792013-11-27T23:41:00.001+00:002013-11-28T21:33:40.114+00:00Programming a million core machine<div align="JUSTIFY" style="margin-bottom: 0cm;">
<a href="http://www.bcs.org/custom/bcs/img/redesign2013/bcsLogoTop.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="http://www.bcs.org/custom/bcs/img/redesign2013/bcsLogoTop.png" /></a><span style="font-family: Arial, sans-serif;">I have just attended an
excellent talk by<a href="http://www.manchester.ac.uk/research/steve.furber"> Steve Furber</a>, Professor of Computer Engineering at
University of Manchester on the challenges on programming a million
core machine as part of the SpiNNaker project. </span></div>
<div align="JUSTIFY" style="margin-bottom: 0cm;">
<span style="font-family: Arial, sans-serif;"><br /></span></div>
<div align="JUSTIFY" style="margin-bottom: 0cm;">
<span style="font-family: Arial, sans-serif;"><br /></span></div>
<div align="JUSTIFY" style="margin-bottom: 0cm;">
<span style="font-family: Arial, sans-serif;"><br /></span></div>
<div align="JUSTIFY" style="margin-bottom: 0cm;">
<span style="font-family: Arial, sans-serif;"><br /></span></div>
<div align="JUSTIFY" style="margin-bottom: 0cm;">
<span style="font-family: Arial, sans-serif;"><br /></span></div>
<div align="JUSTIFY" style="margin-bottom: 0cm;">
<span style="font-family: Arial, sans-serif;">The
<a href="http://apt.cs.man.ac.uk/projects/SpiNNaker/">SpiNNaker project</a> has been in existence for around 15 years and has
been attempting to answer two fundamental questions:</span></div>
<ul>
<li><div align="JUSTIFY" style="margin-bottom: 0cm;">
<span style="font-family: Arial, sans-serif;">How
does the brain do what it does? Can massively parallel computing
accelerate our understanding of the brain?</span></div>
<div align="JUSTIFY" style="margin-bottom: 0cm;">
</div>
</li>
<li><div align="JUSTIFY" style="margin-bottom: 0cm;">
<span style="font-family: Arial, sans-serif;">How
can our (increasing) understanding of the brain help us create more
efficient, parallel and fault-tolerant computation?</span></div>
</li>
</ul>
<div align="JUSTIFY" style="margin-bottom: 0cm;">
<span style="font-family: Arial, sans-serif;">The
comparison of a parallel computer with a brain is not accidental since brains share many of the required attributes being massively
parallel, have lots of interconnections, provide excellent
power-efficiency, require low speed communications, is
adaptable/fault-tolerant (of failure) and capable of learning autonomously. </span><span style="font-family: Arial, sans-serif;">The
challenges for computing as Moore's law progresses is that there will
eventually come a time when further increases in speed will not be
possible and as processing speed has increased, energy efficiency has become
an increasingly important characteristic to address. The future is
therefore parallel but the approach to handling this is far from clear.
The SpiNNaker project has been established to attempt to model a
brain (around 1% of a human brain) using approximately 1 million
mobile phone chips with efficient asynchronous interconnections whilst also examining the approach to developing efficient parallel applications.</span></div>
<div align="JUSTIFY" style="margin-bottom: 0cm;">
<br /></div>
<div align="JUSTIFY" style="margin-bottom: 0cm;">
<span style="font-family: Arial, sans-serif;">The
project is built on 3 core principles:</span></div>
<ul>
<li><div align="JUSTIFY" style="margin-bottom: 0cm;">
<span style="font-family: Arial, sans-serif;">The
topology is virtualised and is as generic as possible. The physical
and logical connectivity are decoupled.</span></div>
<div align="JUSTIFY" style="margin-bottom: 0cm;">
</div>
</li>
<li><div align="JUSTIFY" style="margin-bottom: 0cm;">
<span style="font-family: Arial, sans-serif;">There
is no global synchronisation between the processing elements.</span></div>
<div align="JUSTIFY" style="margin-bottom: 0cm;">
</div>
</li>
<li><div align="JUSTIFY" style="margin-bottom: 0cm;">
<span style="font-family: Arial, sans-serif;">Energy
frugality such that that cost of a processor is zero (removing the
need for load balancing) and the energy usage of each processor is
minimised.</span></div>
</li>
</ul>
<div align="JUSTIFY" style="margin-bottom: 0cm;">
<span style="font-family: Arial, sans-serif;">[As
an aside, energy efficient computing is a growing interest such that
when a program is constructed, how much energy is required to
complete the computation is now the key factor in many systems (in
terms of operational cost)]</span></div>
<div align="JUSTIFY" style="margin-bottom: 0cm;">
<br /></div>
<div align="JUSTIFY" style="margin-bottom: 0cm;">
<span style="font-family: Arial, sans-serif;">The
SpiNNaker project has designed a node which contains two chips; one
chip is used for processing and consists of 18 ARM processors (1
hosts the operating system, 16 are used for application execution and
1 is spare) and the other chip is for memory (SDRAM). The nodes are
connected in a 2D-mesh due to simplicity and cost. 48 nodes are
assembled onto a PCB such that 864 processors are available per
board. The processor only supports integer computation. The major
innovation in the design is the interconnectivity within a node and
between nodes on a board, A simple packet switched network is used to
send very small packets around; each node has a router which is used
to efficiently send the packets either within the node or to a
neighbouring node. Ultimately, 24 PCBs are housed within a single 19”
rack which are then housed (5) within a cabinet such that each
cabinet has 120 PCBs which equates to 5760 nodes or 103680
processors. 10 cabinets would therefore result in over 1 million
processors and would require around 10KW. A host machine (running
Linux) is connected via Ethernet to the cabinet (and optionally each
board).</span></div>
<div align="JUSTIFY" style="margin-bottom: 0cm;">
<br /></div>
<div align="JUSTIFY" style="margin-bottom: 0cm;">
<span style="font-family: Arial, sans-serif;">Networking
(and is efficiency) is the key challenge to emulate neurons. The
approach by Spinnaker is to capture a simple spike (representing a
neuron communication) within a small packet (40 bits) and then
multicast this data around (each neuron is allocated a unique
identifier, there is a theoretical limit of 4 billion neurons which
can be modelled). By the use of a 3-stage associative memory holding
some simple routing information, the destination of each event can be
determined. If the table does not contain an entry, the packet is
simply passed through to the next router. This approach is ideally
suited to a static network or a (very) slowly changing network. It
struck me that this simple approach could be very useful in efficient
communication across the internet and maybe useful for meeting
the challenge of the 'Internet of Things'.</span></div>
<div align="JUSTIFY" style="margin-bottom: 0cm;">
<br /></div>
<div align="JUSTIFY" style="margin-bottom: 0cm;">
<span style="font-family: Arial, sans-serif;">Developing
applications for SpiNNaker requires that the problem is split into
two parts; one part handles the connectivity graph between nodes; the
other part handles the conventional computing cycle with compile/link
and deploy. Whilst the performance in terms of throughput is
impressive (250 Gbps for 1024 links), it is the throughput which is
exceptional at over 10 Billion packets/second.</span></div>
<div align="JUSTIFY" style="margin-bottom: 0cm;">
<br /></div>
<div align="JUSTIFY" style="margin-bottom: 0cm;">
<span style="font-family: Arial, sans-serif;">The
programming approach is to use an event-driven programming paradigm
which discourages single-threaded execution. Each node runs a single
application with the applications (written in C) communicating via an
API to SARK (the SpiNNaker Application Runtime Kernel) which is
hosted on the processor. The event model effectively maps to
interrupt handlers on the processor with 3 key events handled by each
application:</span></div>
<ul>
<li><div align="JUSTIFY" style="margin-bottom: 0cm;">
<span style="font-family: Arial, sans-serif;">A
new packet (highest priority)</span></div>
<div align="JUSTIFY" style="margin-bottom: 0cm;">
</div>
</li>
<li><div align="JUSTIFY" style="margin-bottom: 0cm;">
<span style="font-family: Arial, sans-serif;">A
(DMA) memory transfer</span></div>
<div align="JUSTIFY" style="margin-bottom: 0cm;">
</div>
</li>
<li><div align="JUSTIFY" style="margin-bottom: 0cm;">
<span style="font-family: Arial, sans-serif;">A
timer event (typically 1 millisecond)</span></div>
</li>
</ul>
<div align="JUSTIFY" style="margin-bottom: 0cm;">
<span style="font-family: Arial, sans-serif;">As
most applications for Spinnaker have been to model the brain, most of
the applications have been written in <a href="http://neuralensemble.org/PyNN/">PyNN</a> (a python neural network)
which is then translated into code which can be hosted by SpiNNaker.
The efficiency of the interconnections mean that brain simulations
can now be executed in real-time, a significant improvement over
conventional supercomputing.</span></div>
<div align="JUSTIFY" style="margin-bottom: 0cm;">
</div>
<br />
<div align="JUSTIFY" style="margin-bottom: 0cm;">
<span style="font-family: Arial, sans-serif;">In
concluding, it is clear that whilst the focus has been on addressing
the 'science' challenges, the are clearly insights into future
computing in terms of improved inter-processor connectivity, improved
energy utilization and a flexible platform. Whilst commercial
exploitation has not been a major driving force for this project, I
am confident that some of the approaches and ideas will find a way
into main-stream computing in much the same way that 50 years ago,
Manchester developed the paging algorithm which is now commonplace in
all computing platforms. </span>
<br />
<span style="font-family: Arial, sans-serif;"><br /></span>
<span style="font-family: Arial, sans-serif;">The slides are available <a href="http://manchester.bcs.org/wp-content/uploads/BCSnov13.pdf">here</a>.</span></div>
Mehttp://www.blogger.com/profile/09171556704204860197noreply@blogger.comtag:blogger.com,1999:blog-1846273104553880352.post-84429663853224152732013-04-07T22:38:00.001+01:002013-04-07T22:38:53.897+01:00A Byte of PI<br />
<h2>
<i style="font-family: Arial, serif; font-size: large;">A
national science week activity to introduce primary pupils to
programming</i></h2>
<div align="JUSTIFY" class="western" style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;">In
common with many of my colleagues, I bought a </span><a href="http://www.raspberrypi.org/" style="font-family: Arial, serif;">Raspberry PI</a><span style="font-family: Arial, serif;"> last
summer to find out what all the fuss was about. What could a £25
computer really do? I was most impressed as it was a real computer
running a real operating system which could do the same things that
computers costing 10 times or more could do. I had these crazy ideas for
projects which could use a Raspberry PI but somehow these never came
to anything and the PI was left in a drawer, which appears to be what
has happened to many of the PIs after that initial burst of
enthusiasm. Which is a real shame because the Raspberry PI could just
be the catalyst to get our children programming again. I often hark
back to the heady days of the early 1980's when with the advent of
the <a href="http://en.wikipedia.org/wiki/BBC_Micro">BBC Micro</a>, <a href="http://en.wikipedia.org/wiki/ZX_Spectrum">ZX Spectrum</a> and other machines connected to the family
television, there was lots of interest which resulted in a huge
increase in children (and adults) learning to program and creating
some great games and applications. I am sure these first steps into
computing were the stimulus for many to consider a career in
computing and may be just the reason we had the Dot-COM boom of the
late 1990's.</span></div>
<div align="JUSTIFY" class="western" style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><br /></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://4.bp.blogspot.com/-O-p0agaKS1Q/UWHi2zMD1ZI/AAAAAAAACfI/1Y4Acmi7ahA/s1600/RaspberryPI+March13+004.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="240" src="http://4.bp.blogspot.com/-O-p0agaKS1Q/UWHi2zMD1ZI/AAAAAAAACfI/1Y4Acmi7ahA/s320/RaspberryPI+March13+004.JPG" width="320" /></a></div>
<br />
<div align="JUSTIFY" class="western" style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;">Then
something magical occurred. <a href="http://www.mosi.org.uk/">MOSI</a> in Manchester organised an event for
<a href="http://www.stemnet.org.uk/">STEM ambassadors</a> working in the computing industry in mid-January. I
have been a STEM Ambassador for around 18 months and have always felt
that there weren't that many ambassadors with a strong computing
background. However at this event, there were over 30 like-minded
enthusiastic ambassadors who all felt that the Raspberry PI was
special and was ideal to kick start programming again in schools and
help move the curriculum forward from ICT (using computers) and on to
computing (making computers do something useful).</span></div>
<div align="JUSTIFY" class="western" style="margin-bottom: 0cm;">
<br />
</div>
<div align="JUSTIFY" class="western" style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;">And
then it struck me – if we have our own PIs, could we pool these
together to create an event which could go round schools to try and
stimulate interest in programming? Clearly one PI per school wasn't
going to be enough, we needed lots of PIs so that we could immerse
lots of pupils at the same time. And so a Byte of PI was born. I met
up with another STEM ambassador who I had done an event with 12
months ago for Year 5 pupils. Although we were both computing
professionals, our previous STEM activity for Year 5 was more general
science (these ambassadors have such versatility!) and we were
initially going to repeat the activity this year. However, we both
agreed that it was worth a go at trying to create a Raspberry PI
themed science event. </span>
</div>
<div align="JUSTIFY" class="western" style="margin-bottom: 0cm;">
<br />
</div>
<div align="JUSTIFY" class="western" style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;">But
where do we start? It was clear that the activity needed to be
something which was engaging, stimulating and fun but it also needed
to make sure that it had a clear goal such that at the end of the
session our Year 5 pupils could say that 'I have learnt something
today – I can program a computer'. </span><span style="color: #00000a;"><span style="font-family: Arial, serif;"><span style="font-size: small;"><span lang="en-GB">The
initial thoughts was to have 10 PIs with 2 pupils per PI each trying
to create a different program probably using <a href="http://scratch.mit%2Cedu/">Scratch</a>, a brilliant
visual programming language. We also thought about having some
control experiments e.g. simple traffic light controller. As the
weeks progressed (we had around 6 weeks to make the event a reality),
it dawned on me that I would have to first write a program on the PI,
then de-construct it in such a way that a clear set of instructions
could be created suitable for a Year 5 audience and then tested.</span></span></span></span></div>
<div align="JUSTIFY" class="western" style="margin-bottom: 0cm;">
<br />
</div>
<div align="JUSTIFY" class="western" style="margin-bottom: 0cm;">
<span style="color: #00000a;"><span style="font-family: Arial, serif;"><span style="font-size: small;"><span lang="en-GB">Several
iterations later, I had a program and a set of instructions ready for
testing. My youngest son, William, was keen to help. He had some
experience of Scratch which had been useful as he had discovered some
new features for me to use. Testing demonstrated that the quiz would
take between 30 and 40 minutes to complete which was ideal for a 1
hour session. I also shared with my fellow STEM ambassadors so that
they were comfortable helping at the session.</span></span></span></span></div>
<div align="JUSTIFY" class="western" style="margin-bottom: 0cm;">
<br />
</div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://1.bp.blogspot.com/-oVnCzs_svz4/UWHjPQXaNgI/AAAAAAAACfQ/gvuQ34egCjk/s1600/RaspberryPI+AGSB+group.JPG" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="240" src="http://1.bp.blogspot.com/-oVnCzs_svz4/UWHjPQXaNgI/AAAAAAAACfQ/gvuQ34egCjk/s320/RaspberryPI+AGSB+group.JPG" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><span style="font-family: Arial, Helvetica, sans-serif;">The Byte of PI team ready for Action...</span></td></tr>
</tbody></table>
<div align="JUSTIFY" class="western" style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;">It
was 'launched' at a <a href="http://www.computingatschool.org.uk/">Computing at Schools</a> NW conference in March where
it was explained how Scratch could be used as a catalyst for getting
more pupils into programming </span>
</div>
<div align="JUSTIFY" class="western" style="margin-bottom: 0cm;">
<br />
</div>
<div align="JUSTIFY" class="western" style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;">So
what is a Byte of PI? A byte of PI session is a </span><span style="color: #00000a;"><span style="font-family: Arial, serif;"><span style="font-size: small;"><span lang="en-GB">1
hour session consisting of 4 parts</span></span></span></span></div>
<ul>
<li><div align="JUSTIFY" class="western" style="margin-bottom: 0cm;">
<span style="color: #00000a;"><span style="font-family: Arial, serif;"><span style="font-size: small;"><span lang="en-GB">A
brief introduction to introduce what a program is and why we need
programs (remember computers are stupid; they need people to tell
them precisely what to do!)</span></span></span></span></div>
</li>
</ul>
<ul>
<li><div align="JUSTIFY" class="western" style="margin-bottom: 0cm;">
<span style="color: #00000a;"><span style="font-family: Arial, serif;"><span style="font-size: small;"><span lang="en-GB">A
hands-on session developing an application (a maths quiz). There are
a number extensions available if the initial application is
completed quickly</span></span></span></span></div>
</li>
</ul>
<ul>
<li><div align="JUSTIFY" class="western" style="margin-bottom: 0cm;">
<span style="color: #00000a;"><span style="font-family: Arial, serif;"><span style="font-size: small;"><span lang="en-GB">An
example demonstration of what can be achieved with Scratch including
animation, multiple sprites and sound. (We used a recent homework
assignment from Thomas, my eldest son for this)</span></span></span></span></div>
</li>
</ul>
<ul>
<li><div align="JUSTIFY" class="western" style="margin-bottom: 0cm;">
<span style="color: #00000a;"><span style="font-family: Arial, serif;"><span style="font-size: small;"><span lang="en-GB">A
brief round-up reviewing what was learnt and a video promoting
computer science and why it is cool to code.</span></span></span></span></div>
</li>
</ul>
<div align="JUSTIFY" class="western" style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;">It
was clearly a success and hopefully the event can be repeated
elsewhere.</span></div>
<div align="JUSTIFY" class="western" style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;"><br /></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://3.bp.blogspot.com/-0ODF373sUEc/UUZF3ow5wyI/AAAAAAAACcs/olDytHpuJn0/s1600/ByteofPI+007.JPG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="240" src="http://3.bp.blogspot.com/-0ODF373sUEc/UUZF3ow5wyI/AAAAAAAACcs/olDytHpuJn0/s320/ByteofPI+007.JPG" width="320" /></a></div>
<div align="JUSTIFY" class="western" style="margin-bottom: 0cm;">
<br />
</div>
<div align="JUSTIFY" class="western" style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;">So
what have I learnt:</span></div>
<ul>
<li><div align="JUSTIFY" class="western" style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;">It
is always great fun working with primary children (but I think most
STEM Ambassadors would say that!)</span></div>
</li>
</ul>
<ul>
<li><div align="JUSTIFY" class="western" style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;">The
IT infrastructure in many schools isn't readily PI compatible but
industry can help this by loaning out DVI or HDMI monitors.</span></div>
</li>
</ul>
<ul>
<li><div align="JUSTIFY" class="western" style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;">Scratch
is great language to get children developing applications quickly
but contains many of the features which 'professional' languages
contain. This means the basics in good programming techniques can be
learnt before moving onto more advanced languages.</span></div>
</li>
</ul>
<ul>
<li><div align="JUSTIFY" class="western" style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;">An
event like this needs lots of hands-on support, not necessarily from
ambassadors with a computing background, because there will be lots
of questions.</span></div>
</li>
</ul>
<div align="JUSTIFY" class="western" style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;">The
next steps:</span></div>
<ul>
<li><div align="JUSTIFY" class="western" style="margin-bottom: 0cm;">
<span style="color: #00000a;"><span style="font-family: Arial, serif;"><span style="font-size: small;"><span lang="en-GB">Develop
more Byte of PI exercises, introducing more concepts and possibly
with different languages. The aim is to demonstrate that everyone
can programme and then get engaged with activities such as <a href="http://www.codeclub.org.uk/">CodeClub</a>
and <a href="http://www.stemclubs.net/">STEMclubs</a>.</span></span></span></span></div>
</li>
</ul>
<ul>
<li><div align="JUSTIFY" class="western" style="margin-bottom: 0cm;">
<span style="color: #00000a;"><span style="font-family: Arial, serif;"><span style="font-size: small;"><span lang="en-GB">Develop
a 'Slice of PI' which would be aimed at teachers with a view to
trying to provide some more background behind the activities and in
particular giving them the confidence to teach it within their
schools.</span></span></span></span></div>
</li>
</ul>
<div align="JUSTIFY" class="western" style="margin-bottom: 0cm;">
<span style="font-family: Arial, serif;">Special
thanks must go the Donna Johnson and Daniel O’Donnell at MOSI for
helping us (particularly for providing extra PIs and monitor cables) and ensuring our
enthusiasm never waned; Karen Crowther at AGSB for providing us with
the excuse to run the event (and to have so much fun!), <a href="http://www.nmi.org.uk/">NMI</a> for
providing us with 6 PIs provided that we did something with them to
get children coding (I think we have achieved that), my fellow STEM
Ambassadors Anthony, Lisa, Sam and Amin, and finally my two sons,
Thomas (aged 12) and William (aged 10) for testing my program,
teaching me some of the finer details of Scratch and being excellent
tutors during the workshop.</span></div>
Mehttp://www.blogger.com/profile/09171556704204860197noreply@blogger.comtag:blogger.com,1999:blog-1846273104553880352.post-64017884709811646292013-02-19T23:43:00.003+00:002013-02-19T23:47:28.104+00:00Technologists are good for business<a href="http://www.bcs.org/custom/bcs/img/bcsLogoTop.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="73" src="http://www.bcs.org/custom/bcs/img/bcsLogoTop.png" width="320" /></a>This evening's <a href="http://www.bcs.org/category/17401">BCS/IET Turing Lecture</a> given by <a href="http://www.bcs.org/category/17402">Suranga Chandratillake</a>, founder and former chief strategy officer of Blinkx, at Manchester University was an interesting talk linking the technical excellence of an engineer with the needs of an entrepreneur. His premise was that his undergraduate course in Computer Science at Cambridge University had provided him with many of the skills he needed to have a successful business career - it was just that he wasn't aware he had the skills.<br />
<br />
Suranga first compared the stages that an inventor and entrepreneur went through with the evolution of an idea. The inventor would go from a position where he felt he wanted to challenge the world to the point where he had a flash of inspiration and onto the stage where the invention was now tangible Compare this to an entrepreneur who starts by thinking 'I need money' (because ideas are not enough) to the stage where the product or service is now a salable item up to the point where he is now making a profit. The UK is very good at educating and nurturing many great technologists to create and innovate; unfortunately it is not always good at exploiting these ideas mainly because many of the skills to allow a entrepreneur to exploit technical ideas are not well developed.<br />
<br />
<a href="http://cdn.blinkx.com/images/blinkx4/images/blinkx_logo_grey.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="http://cdn.blinkx.com/images/blinkx4/images/blinkx_logo_grey.png" /></a>He described how he was offered the opportunity to be the founder CEO of <a href="http://www.blinkx.com/">Blinkx</a>, a startup spun out from Autonomy. He was reluctant (very!) at taking on this role because he felt that he didn't have the necessary skills to fulfill the role as he was essentially a technologist. He struck a deal with Mike Lynch, CEO of Autonomy, that said that if he needed help with some of the business functions such as finance, HR, Sales and marketing that Mike would help him out. What amazed me was that the skills he needed for finance, marketing and sales were all taught on his undergraduate course, it was just that they weren't expressed in business manner. For example, for marketing to determine the most effective approach to use (e.g. PR, web-page banner ads or search adverts), requires the application of some simple probabilistic modelling, a 2nd year course. I felt he stretched the analogy a bit far when he compare a HR organisation to that of a system architecture; however, I think many aspects of HR (particularly recruitment) can be covered in parts of undergraduate courses particularly with the increasing amounts of team-working forming part of the curriculum.<br />
<br />
Suringa summarised that the attributes of a technologist of being qualitative, rigorous and analytical had actually prepared him perfectly for business in a technical organisation. He stated that it is a fallacy that technologists do not understand business, it is just that they assume that they don't have the skills. This is a mental block rather than a a lack of ability.<br />
<br />
I found the talk provided much food for thought. Clearly the business environment that Suranga operated in was not typical of many companies but it was illuminating to see he was able to relate back to his undergraduate course. The opportunity to work in a small company with a unique technology (as blinkx) is clearly not going to be available to everyone. However, provided the opportunities are available I am sure many more technologists should feel empowered to exploit technology to create viable and thriving businesses.<br />
<br />
<!-- Start of IET.TV webcast -->
<br />
<table>
<tbody>
<tr>
<td style="vertical-align: top; width: 80px;"><img alt="" src="http://scpro.streamuk.com/webcasts/18/16066/images/c068ecd5-4e3a-4c09-982b-60d9b55fdcc4_source.jpg" style="border-right: 3px solid #0c517a;" width="75" /></td>
<td style="font-family: Arial,Helvetica,sans-serif; font-size: 11px; line-height: 1.6em; padding-left: 10px; vertical-align: top;"><div style="color: #0c517a; font-family: Arial,Helvetica,sans-serif; font-size: 11px; font-weight: bold; line-height: 1.6em; margin-bottom: 0px; margin-top: 0px; width: 300px;">
The BCS/IET Turing Lecture 2013: The IET Turing Lecture 2013: What they didn't teach me: building a technology company and taking it to market</div>
<div style="font-family: Arial,Helvetica,sans-serif; font-size: 11px; line-height: 1.6em; margin-bottom: 0px; margin-top: 0px;">
Suranga Chandratillake</div>
<div style="font-family: Arial,Helvetica,sans-serif; font-size: 11px; line-height: 1.6em; margin-bottom: 0px; margin-top: 0px;">
<a href="http://tv.theiet.org/technology/infopro/16066.cfm">Play webcast</a></div>
The IET Prestige Lecture Series 2013, Turing Lecture, Savoy Place, London, 18 February 2013
<br />
<br /></td>
</tr>
</tbody></table>
<!-- End of IET.TV webcast -->
Mehttp://www.blogger.com/profile/09171556704204860197noreply@blogger.comtag:blogger.com,1999:blog-1846273104553880352.post-27175139103294208592012-02-16T22:51:00.000+00:002012-02-16T22:54:04.714+00:00Employing OSS as part of successful productI recently attended a webinar given by <a href="http://www.olliancegroup.com/">Oliance Group</a> on the success factors for use of OSS. Whilst the webinar concentrated on one particular organisations' experience, the following are my 10 points of note which should apply to any product based organisation using OSS.<br />
<br />
<ol>
<li>Innovation with OSS traditionally comes from the community or vendors (where commerical success can be gained). However there is increasing innovation emerging from customers, partners, academics when OSS is used as there is an increase in collaboration as the wider benefits of OSS are now being recognised.</li>
<li>OSS is increasingly being used in non-differentiating aspects of products. A good example is the <a href="http://www.genivi.org/">GENIVI</a> alliance which provides an open source in-vehicle infotainment toolkit where there is non-differentiation in manufacturers products.</li>
<li>Most product organisations have recognised that it is futile to prevent OSS being used within its products and have now focused on how to best harness the benefits and opportunities that OSS offers. It is essential that the consequences of redistribution is understood when OSS is included as a component within your product - this requires a good understanding of the licences with a preference to use components released under one the permissive licences (Apache 2.0, MIT, BSD) over the copyleft (GPL family) licences. The use of some OSS components has helped its customers, partners etc in adopting and developing new products.</li>
<li>Use of OSS components/products should often result in enhancing and fixing bugs as well as before contributing back into the community. Internal developments may benefit from being released into the community if it is a non-discriminatory development as it will re-energise the component with a better product emerging.</li>
<li>Use of OSS must not diminish customers needs for relaible, quality and secure products. Whilst OSS may offer benefits in terms of reduced development timescales, not all OSS is good and careful selection is required before it can become a key component within a product. The selection process is key to future success and in addition to assessing the licence requirements and component maturity, there needs to be an assessment of the functional fit to ensure that the OSS component is compatible with the overall product architecture and its intended use (e.g. embedded, linked, modified).</li>
<li>Governance policies are required with regards the use of and the contribution to OSS components. All stakeholders must fully understand the approach (there are too many misconceptions about use of OSS by senior management that this needs to be carefully managed). Product owners and architects must be educated and informed about all OSS usage. A knowledge base of approved porducts (and versions) should be actively maintained to avoid a proliferation of different versions (of the same product) and similar products (providing equivalent functionality).</li>
<li>Synchronising product release cycles with that of OSS products can be problematic particularly if the OSS components are released frequently to address bug fixes/security fixes. It is recommended that product release plans are aligned with OSS components (knowledge of the roadmap for each OSS component is therefore essential for this). As products often have long term support requirements, there also needs to be some guarantee that the OSS components are compatible with the same support requirements.</li>
<li>Product standards may need to be harmonized across multiple components, particularly if a UI is involved. Also a consistent approach to security should be adopted (e.g. particularly if SSO is used/required) across both OSS and internal developments. OSS components must be actively monitored for vulnerabilities and fixes applied appropriately.</li>
<li>Support for each OSS component is important, particularly when long term support is considered. Options include do nothing, but this is only of the OSS component is very mature and stable; develop skills internally; establish a maintenance activity by active engagement within the OSS community; or employ a 3rd party support service</li>
<li>Use of an OSS component within a commerical product must ensure that the organisations's intellectual property is protected and that market discriminators remain.</li>
</ol>Mehttp://www.blogger.com/profile/09171556704204860197noreply@blogger.comtag:blogger.com,1999:blog-1846273104553880352.post-69102571335357335782012-02-05T22:17:00.001+00:002012-02-05T22:17:29.472+00:00Windows 8 - Keep taking the tablets<br />
<span style="font-family: Arial, Helvetica, sans-serif;">So should I try the <a href="http://blogs.msdn.com/b/b8/">Windows 8 beta</a> when it become available next month? That was the question following a very honest presentation given by <a href="http://www.thelongclimb.com/">Mike Halsey</a> about the forthcoming Windows 8 operating system to the <a href="http://manchester.bcs.org/">Manchester branch of the BCS</a>. After having a few days to think about it, I can't see many reasons for </span><span style="font-family: Arial, Helvetica, sans-serif;">upgrading from my current Windows 7 setup (and various Linux distributions running as virtual </span><span style="font-family: Arial, Helvetica, sans-serif;">machines and an Android smartphone). This is very different to when Windows 7 </span><span style="font-family: Arial, Helvetica, sans-serif;">came out, as it was a significant improvement from Windows Vista - I used </span><span style="font-family: Arial, Helvetica, sans-serif;">the Beta version on my main machine until it was officially released when I </span><span style="font-family: Arial, Helvetica, sans-serif;">then upgraded all of my machines to Windows 7.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span><br />
<a href="http://res2.windows.microsoft.com/resbox/en/Windows%207/main/30639003-9b14-48c9-9b56-4b9c70089203_12.png" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" src="http://res2.windows.microsoft.com/resbox/en/Windows%207/main/30639003-9b14-48c9-9b56-4b9c70089203_12.png" /></a><span style="font-family: Arial, Helvetica, sans-serif;">Whilst there are probably many significant improvements behind the scenes, including support for new and emerging technologies such as <a href="http://en.wikipedia.org/wiki/USB_3.0">USB3</a> and <a href="http://en.wikipedia.org/wiki/Thunderbolt_(interface)">Thunderbolt</a>, Windows 8's main evolution appears to be with the user interface, and the introduction of the new Metro interface. Mike had a pre-beta version of Windows 8 loaded onto a tablet and having a quick play on the tablet (start up time from cold was most impressive), the user interface was very similar to both Android and IoS. However, one significant change was that the application icons (or tiles as Microsoft would call them) could be different sizes and could also be live (so that they could show the weather or a stock price) without having to launch the application. This was first launched on the Windows Phone but </span><span style="font-family: Arial, Helvetica, sans-serif;">I still think that this was a neat idea (although there are now similar products available</span><span style="font-family: Arial, Helvetica, sans-serif;"> for both Android and IoS).</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span><br />
<span style="font-family: Arial, Helvetica, sans-serif;">2012 is clearly going to be interesting year for operating systems for tablets with Google's Ice-Cream sandwich (aka Android 4), Apples's latest IoS and Microsoft's entry in tablet space (Windows 8). There is clearly room for all of them but it is now very clear that operating systems that cater for tablets must also work seamlessly with smartphones and other devices. With Microsoft being the last to release a tablet based OS, they are clearly playing catch-up and their success will clearly depend on the quality and takeup of the Apps in the recently <a href="http://blogs.msdn.com/b/windowsstore/archive/2011/12/06/announcing-the-new-windows-store.aspx">announced Windows Store</a>. </span><span style="font-family: Arial, Helvetica, sans-serif;">Windows 8 is </span><span style="font-family: Arial, Helvetica, sans-serif;">blatantly</span><span style="font-family: Arial, Helvetica, sans-serif;"> aimed at the consumer market, and is clearly trying to </span><span style="font-family: Arial, Helvetica, sans-serif;">be a common platform across a variety of different platform types (desktops/laptops, smartphones, </span><span style="font-family: Arial, Helvetica, sans-serif;">tablets and games consoles). This is is a bold strategy which no one has yet </span><span style="font-family: Arial, Helvetica, sans-serif;">mastered. It is also promoting connected 'experiences' (with the cloud being a key part of the strategy) and clearly expects </span><span style="font-family: Arial, Helvetica, sans-serif;">a 'touch' interface to become increasingly the primary form of interaction. Having said that, </span><span style="font-family: Arial, Helvetica, sans-serif;">I understand it is still possible to get to the good old DOS window so that </span><span style="font-family: Arial, Helvetica, sans-serif;">traditional</span><span style="font-family: Arial, Helvetica, sans-serif;"> user </span><span style="font-family: Arial, Helvetica, sans-serif;">interface (command line) can still be experienced.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span><br />
<span style="font-family: Arial, Helvetica, sans-serif;">Will Windows 8 be a success? I don't know, but I think Windows 9 (scheduled for late 2015) might be the better bet as it will have the benefit of seeing how the integrated desktop/smartphone/tablet/games console world works. Microsoft are clearly betting on trying to develop a platform which can be common across a range of platform types, a laudable aim which will certainly deliver benefits in terms of product management (assuming it works!). However, I can't see any attraction for large corporates, many of which have yet to migrate from Windows XP. A big problem is in the application space in which applications developed specifically for Windows 8 cannot be run on Windows 7 (or predecessors). It was not clear to me if existing Windows 7 (or earlier) applications could run on Windows 8; if not this will be a huge mistake unless Windows 8 apps becomes priced at typical app prices (i.e. free or typically less than £1) rather than the several £100's that Microsoft applications typically cost.</span><br />
<span style="font-family: Arial, Helvetica, sans-serif;"><br /></span><br />
<span style="font-family: Arial, Helvetica, sans-serif;">Will I download the Windows 8 beta when it is available later this month? Maybe but only out of curiosity and it will be running on some old equipment as it don't see it as replacement OS for my primary machine.</span><br />Mehttp://www.blogger.com/profile/09171556704204860197noreply@blogger.comtag:blogger.com,1999:blog-1846273104553880352.post-35915031629821243182012-01-01T18:31:00.000+00:002012-01-01T18:31:39.384+00:00TDD for embedded development<br />
I recently attended an interesting talk on test driven development (TDD)<br />
for embedded development given by <a href="http://www.zuehlke.com/en/the-company/company/locations/unitedkingdom.html">Zulke UK</a> for the <a href="http://www.bcs-spa.org/">BCS SPA specialist group</a>.<br />
<br />
<table cellpadding="0" cellspacing="0" class="tr-caption-container" style="float: left; margin-right: 1em; text-align: left;"><tbody>
<tr><td style="text-align: center;"><a href="http://www.robotshop.com/Images/xbig/en/dfrobotshop-rover-lights-B.jpg" imageanchor="1" style="clear: left; margin-bottom: 1em; margin-left: auto; margin-right: auto;"><img border="0" height="320" src="http://www.robotshop.com/Images/xbig/en/dfrobotshop-rover-lights-B.jpg" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;">Robotshop Rover</td></tr>
</tbody></table>
The talk was the result of some experiments using an <a href="http://www.arduino.cc/">Arduino</a> board, a bluetooth interface and an android tablet. The embedded platform was a <a href="http://www.robotshop.com/eu/productinfo.aspx?pc=RB-Rbo-33&lang=en-US">Robotshop Rover</a> with a number of sensors and controllers. The sensors were used to guide the robot along a track indicated by a solid black line; the motors were used to control the direction of the robot and also its speed.<br />
<br />
Although the standard <a href="http://arduino.cc/en/Guide/Environment">arduino development environment</a> isn't a full IDE and is limited in its functionality, does come with some good code<br />
examples. An alternative environment is the <a href="http://eclipse.org/cdt/">Eclipse CDT</a> with the <a href="http://www.blogger.com/goog_1506602558">AVR</a> <a href="http://avr-eclipse.sourceforge.net/wiki/index.php/The_AVR_Eclipse_Plugin">plugin</a> added to handle the download of the arduino image to the target platform. To support the development using TDD, <a href="http://www.cpputest.org/">CPPUTEST</a> was used as the test framework. CPPUTEST is recommended as a framework suitable for embedded development (see <a href="http://pragprog.com/book/jgade/test-driven-development-for-embedded-c">James Greening book on TDD for Embedded Development</a>) and appeared to the presenters to be more effective than <a href="http://sourceforge.net/apps/mediawiki/cppunit/index.php?title=Main_Page">CPPUNIT</a>. It was noted by members of the audience that there are few tools which have good integration with the continuous integration platforms such as Jenkins.<br />
<br />
An overview of TDD was given, and its application for an embedded target environment<br />
<br />
1/<span class="Apple-tab-span" style="white-space: pre;"> </span>TDD needs to be able to test on both the development and target environments. This requires that two projects are created, one with arduino as the target environment and an other targeted at an x86 environment<br />
<br />
2/<span class="Apple-tab-span" style="white-space: pre;"> </span>The cycle of test->code->refactor needs to be followed with the tests being chosen from a backlog. Code shouldn't be written unless it is to satisfy an existing test.<br />
<br />
3/<span class="Apple-tab-span" style="white-space: pre;"> </span>The cycle should be choose test->write test->run-test->fail! If it doesn't fail (and it could be due to compiler or link failure), then it normally indicates that the test has been incorrectly written.<br />
<br />
4/<span class="Apple-tab-span" style="white-space: pre;"> </span>Limited design is required although normal good software engineering practice should still be adopted (no monolithic functions etc).<br />
<br />
5/<span class="Apple-tab-span" style="white-space: pre;"> </span><a href="http://en.wikipedia.org/wiki/Mock_object">Mock interfaces</a> should be used to unit test sensor interfaces. This allows the logic to be tested and debugged first before loading on to the target.<br />
<br />
6/<span class="Apple-tab-span" style="white-space: pre;"> </span>If a state machine is required, some design is essential before any test cases can be identified and written.<br />
<br />
The exercise in TDD with the arduino target resulted in no logic errors once the code was exercised on the target. However, the behaviour of the robot, in particular the speed of the motors required some further development and enhancement to the codebase. Given the fact that arduino boards are typically focused on the school market, I was surprised that C++ was the chosen development language. However, the<a href="http://www.blogger.com/goog_1506602574"> C++ used on</a> <a href="http://www.arduino.cc/en/Reference/HomePage">the arduino</a> is a cut down version which removes many of the complexities of the full C++ language.<br />
<br />
Whilst the associated Android development (on a <a href="http://www.motorola.com/Consumers/US-EN/Consumer-Product-and-Services/Tablets/MOTOROLA-XOOM-with-WiFi-US-EN">Motorola XOOM tablet</a>) appeared to be successful in terms of developing a simple user interface to send commends via Bluetooth to control the robot (e.g. stop, start,...), the development of the application revealed some shortcomings. Although the <a href="http://www.eclipse.org/resources/resource.php?id=539">Android development kit works very well with Eclipse</a> (the code is a mixture of Java and xml) and allows on target debugging, the use of TDD is less appropriate given the extensive use of callbacks (e.g.onclick, ....). The testing of GUI applications cannot be adequately tested within a development environment; fortunately the android debugger is excellent at pinpointing issues (typically null pointer exceptions). Android emulators can help to a limited extent but are not a sufficient replacement to an actual device. Android development for tablets is just evolving as the platform moves from a phone, with relatively simple applications, to potentially far more complex applications. The launch of <a href="http://developer.android.com/sdk/android-4.0.html">Android 4.0 development kit</a> (aka Ice-Cream sandwich) will clearly accelerate the development of more complex applications which will necessitate the employment of sound software engineering principles in delivering quality products.<br />
<div>
<br /></div>
<div>
In summary, the session demonstrated successfully that TDD could be applied in an embedded environment and that through the use of appropriate open source tools, software development for the expanding android market can follow tried and tested techniques.</div>Mehttp://www.blogger.com/profile/09171556704204860197noreply@blogger.comtag:blogger.com,1999:blog-1846273104553880352.post-72745572967250041092011-11-17T23:25:00.004+00:002011-11-17T23:44:52.339+00:00Open Source Software - the legal implications<a href="http://www.bcs.org/custom/bcs/img/bcsLogoTop.png" onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}"><img style="float:left; margin:0 10px 10px 0;cursor:pointer; cursor:hand;width: 375px; height: 88px;" src="http://www.bcs.org/custom/bcs/img/bcsLogoTop.png" border="0" alt="" /></a><br /><div>I attended the recent <a href="http://manchester.bcs.org/">BCS Manchester</a>/<a href="http://www.scl.org/">Society for Computers and Law</a> event on the use of Open Source Software and the legal implications. It was given by Dai Davis, an information technology lawyer who is also a chartered engineer, a very unusual combination but he clearly knew his material.</div><div><br /></div><div>Dai started by explaining the basic aspects of copyright law and the implications that this</div><div>had on software. It was clear that some of the original purposes of copyright law (to prevent</div><div>copying) were applicable to software but that the period that copyright lasts (70 years after</div><div>the authors death) clearly made little sense for software. However a number of points caught my eye:</div><div><ul><li>Copyright protects the manifestation and not the subject matter. This means that look and feel is not normally subject to copyright although fonts are.</li><li>Copyright infringement also includes translating the material. Translation in the software case includes compiling source code as well as rewriting the source code into another language.</li><li>Copyright protects copying some or all of the material. The amount does not normally matter.</li><li>Moral rights do not extend to software but do apply to documentation</li><li>Copyright infringement is both a civil and criminal offence with a maximum of 10 years imprisonment and an unlimited fine.</li></ul></div><div>Dai then explained that the first owner (or creator) of the material owns the copyright. Misunderstanding this is the major cause of disputes about copyright. Clearly there are exceptions if the material is created in the course of employment (copyright rests with the employer) or if the contract under which the material is being created 'assigns' the copyright to the purchaser.</div><div><br /></div><div>All software licences grants the purchaser permission to use the software otherwise the purchaser would be in breach of the copyright. Licences can be restrictive e.g. by time, number of concurrent users and all licences are transferable according to EU law.</div><div><br /></div><div>Copyright of <a href="http://opensource.org/">Open Source Software</a> is no different to normal copyright of software but the approach to licencing is very different:</div><div><ul><li>Nearly all OSS do not require payment to acquire</li><li>Free relates to restrictions on use (non-OSS can place restrictions)</li><li>Open access to source and usage is required (not normally available with non-OSS)</li></ul></div><div>However, the licences are very difficult to enforce mainly because there has been no loss in terms of monetary value. There has never been a successful prosecution in the UK although there are a number of examples in <a href="http://www.gpl-violations.org/">Germany</a> (where litigation is cheaper than the UK) and an example in the US (<a href="http://en.wikipedia.org/wiki/Jacobsen_v._Katzer">Jacobson v Katzer</a> in 2009) where a 'token' settlement of $100000 was awarded.</div><div><br /></div><div>Whilst there may be little prospect of getting sued for use of Open Source Software the biggest issue often comes when businesses are sold and OSS is found within a product - this often affects the eventual purchase price of the company. Many businesses don't know where Open Source Software is being used and included within its own products because it is very difficult to police and manage.</div><div><br /></div><div>A video of the session was filmed by students from <a href="http://www.artdes.mmu.ac.uk/filmmedia/course/">Manchester Metropolitan University</a> the resulting video being made available via the <a href="http://manchester.bcs.org/">BCS Manchester website</a>.</div>Mehttp://www.blogger.com/profile/09171556704204860197noreply@blogger.comtag:blogger.com,1999:blog-1846273104553880352.post-73966934337507107842011-07-27T23:03:00.003+01:002011-07-27T23:13:26.322+01:00How good are your requirements?I read an interesting <a href="http://marekblotny.com/2011/07/how-detailed-the-requirements-should-be/">post</a> which was trying to determine how much detail a requirement should contain. As with any question like this, it all depends on a number of factors and it is not possible to give a rule which can be religiously followed. Experience over time will determine what works for you - there is always a cost if the balance isn't right. Too few requirements and there is likely to be a very difficult verification phase; too many requirements and it will prolong the development and testing phases.<div><br /></div><div><div>The first rule of requirements is to ask your customer 'why?'. If they can't explain simply why the requirement is needed then it isn't essential and it should be discounted. If the response is acceptable, the next question is 'How am I going to validate that I have satisfied the requirement'? If you can't agree on how the requirement is to validated, then the requirement clearly needs clarifying, maybe with some constraints explicitly included in the requirement wording. The language of requirements is very important and the differences between shall, should, will, may are VERY important. Any requirement with phrases such as 'such as', 'for example' should always be rejected as they are open-ended and can never be fully satisfied.</div><div><br /></div><div>As an example, I encountered a requirement recently which included the phrase 'any printer shall be supported'. After discussions, and any customer that doesn't engage in discussions is clearly an indication of a very difficult customer!, it became very clear why the requirement had be written in the way - a third party would be supplying the as yet unspecified printer. However, the discussion did enable the requirement wording to became slightly more achievable by rephrasing as 'a network connected printer supporting postscript'. At least I would have a chance of testing this (and have constrained it to exclude printers with parallel or usb interfaces).</div><div><br /></div><div>The hardest part of requirements is not the accepting the requirement, it is the validation at the end. How often have you encountered 'This isn't what I wanted'. The only way to avoid this is to remain in constant contact with your customer to validate any assumptions throughout the development so that there aren't any surprises at the end.</div></div><div><br /></div><div>Other than experience, I have yet to find a reliable and objective way of assessing the 'goodness' of a requirement set. It would clearly be possible to perform some analysis of the language used, looking for ambiguous phrases for example, but would this be a sufficient measure? </div>Mehttp://www.blogger.com/profile/09171556704204860197noreply@blogger.comtag:blogger.com,1999:blog-1846273104553880352.post-9963530367134456352011-06-04T22:57:00.006+01:002011-06-05T21:48:59.291+01:00Computer Forensics<div>BCS Manchester recently hosted an interesting evening on the growing importance of computer forensics. The session was led by <a href="http://www.raincock.co.uk/">Sam Raincock</a>, an experienced expert in performing forensic analyses. Whilst the session did not reveal the secrets behind a successful analysis (or give hints of how to make an analysis more difficult), it did explore some of the approaches (in general) that can be used in establishing evidence. Whilst a typical forensic investigation does include a significant amount of technical information this only accounts for about 20% of an analysis as the remaining time is concerned with communication with lawyers and report writing. As in all legal cases, it is crucial to review and present all the evidence to piece together a coherent case rather than circumstantancial evidence.<div><br /><div>While Computer forensics is primarily the examination of any device that permanently stores data (the range of devices is ever-expanding from the traditional hard disc drives, CD-ROMS and USB memory sticks to mobile phones and cameras), it also includes reviewing system functionality in its goal to try to establish what happened and who did it. It is used in a variety of cases include criminal, civil and fraud cases.</div><br /><div> It was stated that 'Every contact leaves a trace' by <a href="http://en.wikipedia.org/wiki/Edmond_Locard">Edmond Locard</a>, an early pioneer of forensic science. This is very true with all computer usage as every time a file is created, every web page that is browsed, every document that is printed is recorded somewhere although computer usage is unique to everyone.</div><br /><div>Some key points that I took away from the session included:<ol><li>Never assume anything</li><li>All humans are unpredictable, and different</li><li>Personnel cause more damage than they discover</li><li>Do not assume that common sense prevails</li><li>The IT department are not forensically trained and don't necessarily understand the value of every piece of data</li><li>Forensics is not about data recovery</li><li>Ownership of data must be established</li></ol>A forensic examination is looking at where the offence was allegedly committed, how the event occurred and who performedthe activity. A typical examination can normally be performed on a single device (once a forensic image has been taken) by an appropriate expert and does not normally need to consult with outside agencies (e.g. internet service providers) to obtain specific information. The examination will review such data as cookies, the various system logs, network connections (IP addresses, type of connection particularly whether it was local, remote, fixed, wireless etc). The usage patterns of a computer will reveal a significant amount as every human has particular behaviour traits. The use of the various system logs that reside on a computer or within a network can reveal significant and valuable data; these logs should be actively monitored as they can often be the first sign that something unusual is being performed that may merit investigation. The sooner something is detected, the greater the chance of limiting the damage (or increasing the evidence in order to establisha conviction). In the case of an incident being detected within a business, the primary aim is to return a business to normal as quickly as possible. This is where policies are vitally important; it is equally important that they are actively used, policed and maintained.</div><div><br />Whilst there is no formal qualification required to become a forensic expert (an inquiring mind would probably be useful), it is clearly a growing and important aspect of computing. There are clearly many challenges with the continually evolving usage of computers; the growing importance of the cloud will clearly require different techniques to those employed when examining a physical item such as a laptop. The session left me wondering what traits my computer usage would reveal about me but also wanting to find out more about what is being recorded without me having any knowledge.</div></div></div>Mehttp://www.blogger.com/profile/09171556704204860197noreply@blogger.comtag:blogger.com,1999:blog-1846273104553880352.post-4639112328246359652010-10-11T23:34:00.004+01:002010-10-12T00:03:29.821+01:00Open Source Document and Content Management<p>BCS Manchester recently hosted a meeting on open source content management and document management in the Public Sector. The speaker. <a href="http://www.grahamoakes.co.uk/">Graham Oakes</a>, explained that the catalyst for this had been an article in the <a href="http://www.guardian.co.uk/technology/2009/aug/07/local-government-open-source-birmingham-website-costs">Guardian</a> about the use of commercial software by Birmingham City Council and its attempt at building websites. They had already spent £2.8M and the question was asked ‘Why not use Open Source Software instead?’ Graham stated that there is an awful lot of mis-information around, in particular that the use of OSS does not mean it costs nothing! This led the <a href="http://ossg.bcs.org/">BCS Open Source Software Group</a> to run a <a href="http://ossg.bcs.org/2010/01/07/">conference</a> in January 2010 looking at the use and adoption of OSS in the public sector.</p> <p>After briefly outlining what OSS is (source code owned by the community which can derive new works), Graham outlined a typical software stack found in many organisations. At every level, there are good open source solutions available. Content Management normally fits in at the application layer below portals and webservers. Content Management has a number of very strong options including <a href="http://plone.org/">Plone</a>, <a href="http://www.onehippo.com/">Hippo</a>, <a href="http://typo3.com/">TYPO3</a>, <a href="http://drupal.org/">Drupal</a>, <a href="http://www.joomla.org/">Joomla</a>, <a href="http://ez.no/">eZ</a> and <a href="http://umbraco.org/">Umbraco</a>. Many of these have already been adopted by public sector in developing websites, for example a number of Police forces.Document Management is not as well developed as Content Management and there are fewer options. </p><p>Gartner and Forrester enterprise software reviews both report that OSS should be adopted and it is becoming more amenable to use in the UK but it is still necessary to consider the full life-cycle costs. OSS should be considered equivalent to proprietary – public sector should now consider and event contribute to OSS projects. However UK is some way behind other european nations (specifically Netherlands, Germany, Italy and Denmark) with the <a href="http://www.ososs.nl/">OSOSS project</a> in Netherlands urging public administration to use more OSS and open standards.</p> <p><span class="Apple-style-span" style="font-weight: bold; ">Advantages</span></p> <p>The key advantages of adopting OSS within the public sector were identified as </p> <ul> <li>The reason for adoption is low up-front costs. Low costs of initial ownership, but the organisation needs to consider normal software selection processes and consider the risks, requirements for the software etc. OSS should be considered no differently to commercial software. It is still necessary to look at the total cost of ownership (TCO) and an organisation may need to still involve a system integrator in order to deploy effectively.</li><li>OSS applications do not constrain the design. The public sector can can use it to start small, think big.</li> <li>Many OSS is easier to work with because access to source code (only useful if you have skills to use it!) is always available. This can provide an additional option to documentation. OSS is also increasingly important with the cloud as proprietary license models don’t adapt readily.</li> <li>Good to help the public sector to demonstrate openness (committed to visibility and open to the public)</li> </ul> <p>Apart from the last one, the advantages are not particularly specific for public sector adoption.</p> <h4>Risks</h4> <p>Of course, there is always a downside, commonly called risks in procurement circles. The risks identified included</p> <ul> <li>Getting over the perception that everything is free. This misunderstands the true costs of OSS. Most costs in using software are not in the licenses – it is actually in the content created/managed by the software. Most project costs are less than 10% on technology/license and the migration costs must always be considered as these can often be many more times more expensive than the base software costs.</li> <li>Misperception around content management and reusability of OSS. Just because some OSS can be used securely, does not mean that all OSS is secure.</li> <li>Public sector are use to working with large organisations. OSS needs a different approach which the public sector may not readily adopt due to the mismatch of scale. OSS developers often move at a far faster pace than the public sector does (or can). This can be difficult at the procurement stage as the existing procurement model (level playing field) is broken. The procurement approach needs to recognise that OSS is no more less secure than proprietary solutions</li> <li>Unreasoned decisions still dictate the major procurement decisions. OSS might not be a perfect match to the requirements but may provide a suitable solution.</li> </ul> <h4>Conclusions</h4> <p>Graham presented a set of conclusions, which if I am honest are no different to proprietary software.</p> <ul> <li>Remember not all OSS are not the same, different quality levels and capabilities</li> <li>Always chose carefully – consider usages scenarios</li> <li>Look beyond licensing costs, user adoption, change management, migration</li> <li>A team still creates the success over the technology. Choose the right team!</li> <li>OSS supports evolutionary delivery, try before buy, which encourages innovation and supports agile and lean practices. However, this is good practice for all software.</li> <li>License fees for software bring costs forward and commit project for the duration (unless trial licenses are available). OSS does not have this commitment and it is (relatively) easy to change OSS software without excessive upfront costs.</li> </ul> <p>So would OSS have solved Birmingham’s problem. No. The problem was not a cost of licences issue; it was not understanding the issue well enough. OSS would have helped to examine the problem in the small before the initial financial commitment was identified which might have produced a more realistic budget.</p>Mehttp://www.blogger.com/profile/09171556704204860197noreply@blogger.comtag:blogger.com,1999:blog-1846273104553880352.post-1801677526888694382010-07-20T20:04:00.003+01:002010-07-21T19:21:06.978+01:00Lessons in measurement and data analysis<p>Recently I attended a very interesting and entertaining lecture by Peter Comer, from <a title="Go to website" href="http://www.abelliosolutions.co.uk/">Abellio Solutions</a>, to the <a title="Go to website" href="http://www.bcs.org/server.php?show=conWebDoc.1240">BCS Quality Management Specialist Interest Group (NW)</a> on lessons learnt in measurement and data analysis following a recent quality audit of an organisation’s quality management system (QMS).</p> <p>The talk started by highlighting the requirements for measurement in <a href="http://www.bsigroup.com/en/Assessment-and-certification-services/management-systems/Standards-and-Schemes/ISO-9001/">ISO9001</a> (section 8). Key aspects that were highlighted included</p> <ul> <li>Measure process within QMS to show conformity with and effectiveness of QMS</li> <li>Monitoring and measurement of processes, products and customer satisfaction with QMS</li> <li>Handle and control defects with products and services</li> <li>Analyse data to determine suitability and effectiveness of QMS</li> <li>Continual improvements through corrective and preventative actions</li> </ul> <p>It was noted that everyone has a <a href="http://en.wikipedia.org/wiki/Key_performance_indicator">KPI (Key Performance Indicator)</a> to measure the effectiveness of products and services although every organisation will use the KPIs slightly differently.</p> <p>Peter outlined the context of the audit, which was an internal audit in preparation for a forthcoming external audit. The audit was for a medium sized organisation with small software group working in transport domain. A number of minor non-conformances which were relatively straightforward to correct. However, after the audit an interesting discussion ensued regarding the software development process which stated that they were finding more bugs in bespoke software development than anticipated and a lot harder to fix. Initial suggestions included:</p> <ul> <li>Look at risk management practices. However, the organisation had already done this by reviewing a old (2002) downloaded paper looking at risk characteristics. </li> <li>Look at alternative approaches to software development.</li> </ul> <p>It was the approach to risk which intrigued Peter. The quality of the paper was immediately considered. What was the quality of the paper? Has it been peer-reviewed? Is it still current and relevant?</p> <p>Peter then critiqued the paper. The paper proposed a number of characteristics supplemented by designators; it was quickly observed that there was considerable overlap between the designators. The analysis of the data was across a number of different sources although no indication of what the counting rules are (and no indication if they were rigorous and consistent). The designators were not representative of all risk factors that may affect a development and said nothing about their relevance to the size of development. The characteristics focused on cultural issues rather than technical issues - risk characteristics should cover both. Just counting risk occurrences does not demonstrate the impact that the risk could have on the project.</p> <p>Turning to the conclusions, Peter considered if the conclusions were valid. What would happen if you analysed the data in a different way, would the conclusions be different? Can we be assured that the data has been analysed by someone with prior experience in software development? It was observed that designators were shaped to criteria which is appropriate, but one size doesn’t fit all. Only by analysing the data in a number of different ways can the significance of the data can be established. It can also show if the data is not balanced which can in turn lead to skewed results. In the paper under review, it was clear that qualitative data was being used quantatively.</p> <p>Peter concluded by stating that by ignoring simple objective measures can lead to the wrong corrective approach which might not be appropriate to their process and product. This is because ‘you don’t know what you don’t know’. It is essential to formally define what to count (this is a metric) with an aim to make the data to be objective. Whatever the method for collection, it must be stated to ensure that it is consistent. </p> <p>The talk was very informative and left much food for thought. I have always aimed to try and automate the collection process to try and make this consistent. However this does nothing if the data is interpreted incorrectly or inconsistently. It is also difficult to know if you are collecting the right data but that is what experience is for!</p>Mehttp://www.blogger.com/profile/09171556704204860197noreply@blogger.comtag:blogger.com,1999:blog-1846273104553880352.post-13908287888136423842010-03-17T22:50:00.007+00:002010-03-17T23:40:21.538+00:00Creating Intelligent Machines<a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://www.bcs.org/cache/thumb-70-upload-img-chris-bishop.jpg?4afab182="><br /></a><br /><a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://www.bcs.org/cache/thumb-70-upload-img-chris-bishop.jpg?4afab182="><img style="float:left; margin:0 10px 10px 0;cursor:pointer; cursor:hand;width: 70px; height: 85px;" src="http://www.bcs.org/cache/thumb-70-upload-img-chris-bishop.jpg?4afab182=" border="0" alt="" /></a>I have just attended the excellent <a href="http://www.bcs.org/server.php?show=nav.11481">IET/BCS 2010 Turing Lecture 'Embracing Uncertainty: The New Machine Intelligence'</a> at the University of Manchester which was given this year by <a href="http://research.microsoft.com/en-us/um/people/cmbishop/index.htm">Professor Chris Bishop</a> who is the Chief Research Scientist at Microsoft Research in Cambridge and also Chair of Computer Science at the University of Edinburgh. The lecture allowed Chris to share his undoubted passion for machine learning, and although there were a number of mathematical aspects mentioned during the talk, Chris managed to ensure everyone was able to understand the key concepts being described.<br /><br />Chris started by explaining that his interest is in building a framework for building intelligence into computers, something which has been a goal for many researchers for many years. This is now becoming increasingly important due to the vast amounts of data which is now available for analysis. With the amount of data doubling every 18 months, there is an increasing need to move away from purely algorithmic ways of reviewing the data to solutions which are based on learning from the data. This has traditionally been the goal for machine (or artificial) intelligence and despite what <a href="http://web.media.mit.edu/~minsky/">Marvin Minsky</a> wrote in 1967 in <a href="http://portal.acm.org/citation.cfm?id=SERIES11430.1095587">'Computation: Finite and Infinite Machines'</a> that "within a generation ... the problem of creating 'artificial intelligence' will substantially be solved", the problem still does not have a satisfactory solution for many classes of problem.<br /><br />A quick summary of the history of artificial intelligence showed that expert systems, which were good at certain applications but required significant investment in capturing and defining the rules, and neural networks which provide a statistical learning approach but have difficulty in capturing the necessary domain knowledge within the model, were not adequate for today's class of problems. An alternative approach which was able to integrate domain knowledge with statistical learning was required and Chris's approach was to use a combination of approaches:<br /><ol><li>Bayesian Learning which uses probability distributions to quantify the uncertainty of the data. The distributions are amended once 'real data' is applied to the model which results in a reduction in the uncertainty.</li><li>Probabilistic Graphical Models which enables domain knowledge to be captured in directed graphs with each node having a probability distribution.</li><li>Efficient inference which ensures efficiency in computation</li></ol>To explain the approach, Chris sensibly used real-life case studies to demonstrate the application of the theory in three very diverse applications.<br /><br />His first example was of Bayesian Ranking system to be used in producing a global ranking from noisy partial rankings. The conventional approaches is to use the <a href="http://en.wikipedia.org/wiki/Elo_rating_system">Elo rating system</a> which is a method for calculating the relative skill levels of players in two-player games. The Elo system could not handle team games or more than 2 players. As part of the launch of the Xbox 360 Live online playing solution, Microsoft developed the TrueSkill algorithm to match opponents of similar skill levels. The TrueSkill algorithm converges far faster than Elo by managing the uncertainty in a more efficient way; it also operates quickly so that users can find suitable opponents in a few seconds out of a user population of many million. Further details on TrueSkill(TM) are available at <a href="http://research.microsoft.com/en-us/projects/trueskill/">http://research.microsoft.com/en-us/projects/trueskill/</a><br /><br />The next example was for a website serving adverts and how to determine which advert to show based on the probability of being clicked and the value of click. The proposed approach was to use gausian probability in order to assign a weight to a number of features which is used to determine the ranking. However it is important to ensure that the system continually learns in order to re-evaluate the ranking to ensure that the solution accurately reflects the dynamics of the adverts. If this was not the case, it would be very difficult for a new advert to be be served.<br /><br />The final example was the <a href="http://www.maas.org.uk/">Manchester Asthma and Allergy Study</a> which is working with a comprehensive data set acquired over 11 years. The data set is continually being augmented with new types of data (recently genetic data has been added) and the study has been successful at establishing the important variables and features and their relationships. By defining a highly structured model of the domain knowledge, it has been possible to assign each variable a probability distribution. By placing the data at the heart of the study and applying some machine learning techniques, a number of key observations are now being reported which might not have been apparent if more traditional statistical techniques had been used.<br /><br />As a closing remark, Chris promoted a product from Microsoft Research (<a href="http://research.microsoft.com/infernet">Infer.net</a>) which provides a framework for further experimentation in developing Bayesian models for a variety of machine learning problems.<br /><br />As is now traditional with the Turing Lecture, it is presented at several locations around the country. A webcast of the version presented at the IET in London is available on the <a href="http://tv.theiet.org/">IET TV channel</a>.Mehttp://www.blogger.com/profile/09171556704204860197noreply@blogger.comtag:blogger.com,1999:blog-1846273104553880352.post-37170225632748975552009-11-27T22:53:00.005+00:002009-11-27T23:20:25.716+00:00Performance Management for a Complex World<img style="float:right; margin:0 0 10px 10px;cursor:pointer; cursor:hand;width: 134px; height: 96px;" src="http://www.theiet.org/staticfiles/images/template09/iet-logo.gif" border="0" alt="" /><br />A thought provoking presentation was recently hosted by the <a href="http://www.theiet.org/local/uk/northwest/manchester/index.cfm">Manchester branch of the IET </a>on Performance Management in a Fast Moving World presented by<a href="http://www.manchester.ac.uk/research/therese.lawlor-wright/"> Dr Therese Lawlor-Wright</a> (University of Manchester), Elizabeth Jovanovic (EEF) and Andrew Wright from <a href="http://www.dynamic-technologies.co.uk/">Dynamic Technologies Ltd.</a><br /><br />After a brief introduction in to what performance measurement was (determining the quality of execution, competence and effectiveness of a actor or operation compared against a standard) and how it could be applied to both people (typically in the form of appraisals) and processes and systems (typically in the form of project reviews), the presentation made a few key observations about how performance measurements are currently performed.<br /><ul><li>Organisations define performance measurements at the strategic level to see how it can achieve goals, often stated in terms of cost, quality, time and reported against a number of key performance indicators (KPIs). These can then be used to communicate and confirm (or adjust) progress.</li><li>Any KPIs used must be balanced, fair and transparent</li><li>The performance measurements must include indicators of the effectiveness of the organisation (the extent at which the strategic objectives are being met) and the efficiency of the organisation (how economically the resources of the organisation are being used to provide the level of performance).</li></ul>Whilst the measurements can be useful, there was a word of caution in the unwanted effects of performance management systems (cited by <a href="http://www.emeraldinsight.com/Insight/ViewContentServlet?contentType=Article&Filename=/published/emeraldfulltextarticle/pdf/0420220602.pdf">De Bruijn (2002)</a>)<br /><ul><li>The measurements will take too long to accumulate and will soon be out of date (data lag)</li><li>Competition between teams or business units can result in a tendency to not share valuable data</li><li>The measurements can stifle innovation at the expense of efficiency</li><li>There will be 'game' playing to maximise the 'score'</li></ul>These effects will dominate in the long term. To counter this, the measurements must be refreshed regularly. This should ideally be every 2 to 3 years, as this gives sufficient time for the data to be used for benchmarking/comparison purposes but short enough to counter the complacency that can arise.<br /><br /><b>Managing Performance</b><br /><br />Having established the measurements to be made, the presentation moved on to the application of the measures in the form of performance management. Performance management needs to be both process-oriented and people-oriented and must be a continuous process (and not just an annual activity which is often the case). By being continuous it helps to clarify expectations, standards and targets and allows for any corrective actions to be addressed as soon as they arise. The most common approach for people-oriented performance management, the appraisal, should link individual targets to organisation targets but must be an opportunity to praise and develop. As expected, the audience was reminded that all objectives must be <a href="http://en.wikipedia.org/wiki/SMART_criteria">SMART</a>. However, an alternative method of specifying a target was proposed - 'Positive - Personal - Present' which can be used to change behaviour. It can improve staff morale if done correctly, as it apparently tricks your subconscious mind into acting positively (the targets should be written starting 'I ...'). There was a strong suggestion that contrary to many organisations, performance management must not be linked to remuneration since it results in a warped approach in order to fit in with the inevitable budgetary constraints.<br /><br /><b>The Complex World</b><br /><br />The complex world was defined as fast moving (continual change, increasing hierarchies of complexity) together with increasing challenges (timescales, budgets, mergers, multiple dependencies). These require that performance management is aligned with the business strategy. However, the classical approach to strategic management with a top-down controlling hierarchy was considered unsuitable. For complex systems,a more holistic approach is required with a thoughts being contributed from the bottom upwards.<br /><br />With a fast changing environment, long-term plans quickly lose touch with reality with inflexible KPIs driving behaviours that fail to respond to the real-world challenges. Inconsistency between the strategy, real-world reality, the KPIs and the objectives inevitably quickly leads to poor performance. Clearly there needs to be an alternative approach to performance measurement which is both flexible and efficient.<br /><br />Taking some of the ideas from the <a href="http://agilemanifesto.org/principles.html">Agile Manifesto</a>, a more lively and dynamic approach developed by consensus which adapts to the changing environment was proposed. This approach addressed many of the unwanted effects with the traditional performance measurement schemes by being much more efficient and flexible with an empowered organisation with a shared vision. Use of such techniques such as the <a href="http://en.wikipedia.org/wiki/Balanced_scorecard">Balanced Scorecard</a> and the<a href="http://ww1.efqm.org/en/"> EFQM Excellence Model </a>can clearly help in communicating a comprehensive view of an organisation.<br /><br /><b>Key Conclusions</b><br /><ul><li>Strategy must become change oriented with a dynamic response in a controlled manner. The route to the strategic vision may change.</li><li>Long-term plans leave companies without direction</li><li>KPIs and objectives must respond to the changing needs</li><li>The measurement process must be flexible and efficient</li><li>Good performance must be encouraged by reducing uncertainty</li><li>Organisations need to move from optimising the 'simple' status quo to optimising 'complex' continuous change</li></ul>Mehttp://www.blogger.com/profile/09171556704204860197noreply@blogger.comtag:blogger.com,1999:blog-1846273104553880352.post-23882236035161945212009-09-20T21:09:00.010+01:002009-09-20T21:57:09.104+01:00Open Source Certification<a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://www.bcs.org/custom/bcs/img/bcsLogoTop.png?4a9403f3="><img style="float:left; margin:0 10px 10px 0;cursor:pointer; cursor:hand;width: 375px; height: 88px;" src="http://www.bcs.org/custom/bcs/img/bcsLogoTop.png?4a9403f3=" border="0" alt="" /></a><br /><br /><br /><br /><br /><br />I have just been browsing the relaunched website of the <a href="http://www.bcs.org/">British Computer Society</a> and came across an interesting article on <a href="http://www.bcs.org/img/server.php?show=conWebDoc.28321">Open Source Certification</a>. Now there are some pretty important and successful open source applications out there, but there is limited experience of 'certification' in the same way that you can be become, for example, Microsoft certified. Red Hat does offer some courses for you to become a <a href="http://www.redhat.com/certification/rhce/">Red Hat Certified Engineer (RHCE)</a> but this is an exception for open source applications. <div><br /></div><div>The big question is does it matter? It all depends on your point of view regarding certification. Does the fact that a product is 'certified' make it a better product? Does the fact that an engineer is 'certified' make him a better engineer compared to one who isn't? As in all cases it depends. A certified engineer should certainly have independently demonstrated a degree of competence in using or configuring a product. However certification without experience to back up the qualification is no use to anyone. Similarly a certified product might demonstrate that the product has become too large and cumbersome that it really needs to be entrusted to a select band of engineers who have demonstrated that they understand the product better than those who have just learned to tame the product to met their specific requirements. A certified engineer should also probably be aware of a few tricks and tips which are not widely known.<div><br /></div><div>So should all open source products offer a certification programme? In my view, no. However there is clearly a point at which certification becomes necessary or expected by the customer community. I would suggest that this can occur in a number of cases:</div><div><ul><li>When the product is becoming widely accepted as one of the market leaders across multiple platforms. </li><li>When the product is now developed on 'commercial' lines with a funding line.</li></ul></div><div>In either case, a professional certification programme should be promoted and managed, but recognizing that significant experience of a product should be automatically rewarded (on request) with certification, particularly if the experience has been gained through the formative years of the product.</div><div><br /></div><div>A similar approach was adopted a few years ago by the BCS when it launched the Chartered IT Professional (CITP) qualification. To date, this has yet to become a widely accepted, recognized (and demanded) qualification for key roles within the IT industry. Until recognized qualifications or certifications within the IT industry become a pre-requisite for certain roles, the certifications people achieve will be little more than another the certificate to put on the wall or in the drawer. Until this is the case open source certification will become little more than a commercial exercise in raising funds for future product developments.</div></div>Mehttp://www.blogger.com/profile/09171556704204860197noreply@blogger.comtag:blogger.com,1999:blog-1846273104553880352.post-13788788192726361592009-09-06T14:02:00.005+01:002009-09-06T14:17:44.084+01:00Project Management blogging<a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://www.bcs.org/img/bcs/bcslogo.gif"><img style="float:left; margin:0 10px 10px 0;cursor:pointer; cursor:hand;width: 125px; height: 60px;" src="http://www.bcs.org/img/bcs/bcslogo.gif" border="0" alt="" /></a>From time to time, I anonymously contribute to the <a href="http://www.bcs.org/server.php?show=ConBlog.17">BCS ProjectEye Blog</a> on a variety of subjects related to Project Management. Some of my contributions include:<div><ul><li><a href="http://www.bcs.org/server.php?show=ConBlogEntry.1229">Open Source Project Management</a></li><li><a href="http://www.bcs.org/server.php?show=ConBlogEntry.1075">Skill Deficits</a></li><li><a href="http://www.bcs.org/server.php?show=ConBlogEntry.944">The Manchester City Project</a></li><li><a href="http://www.bcs.org/server.php?show=ConBlogEntry.917">Agile Project Management</a></li><li><a href="http://www.bcs.org/server.php?show=ConBlogEntry.843">Festive Thoughts</a></li><li><a href="http://www.bcs.org/server.php?show=ConBlogEntry.747">Ensuring Timely Success</a></li><li><a href="http://www.bcs.org/server.php?show=ConBlogEntry.736">The Sorcerer's Apprentice</a></li></ul></div>Mehttp://www.blogger.com/profile/09171556704204860197noreply@blogger.comtag:blogger.com,1999:blog-1846273104553880352.post-1085065907407908062009-06-23T23:47:00.008+01:002009-07-26T23:25:07.114+01:00Writing poor code requires skill<div>Recently I read an interesting post on<a href="http://blog.decayingcode.com/2009/06/improving-code-quality-2-ways-to-go.html"> decaying code</a> and the various ways that could be adopted in order to improve code quality. It briefly outlined three approaches to improving code (presumably on a code base which has been evolved due to enhancements and bug fixes):</div><div><ul><li>Detect the bad code and fix it. But this is too expensive...</li><li>Don't write it in the first place. But this requires you (or a tool) to be able to spot bad code (consistently) in the first place.</li><li>Formal Training. This is fine, but how do you ensure that that 'training' is put into practice correctly? And it is all too easy to fall back into bad habits which won't get spotted.</li></ul><div>This got me thinking. Are there any other approaches to prevent bad code (or code smells)? Well I reckon there are. In fact it should take skill to write poor code given the amount of help that there is at constructing software now.</div><div><br /></div><div><b>Increase use of automatic code generation</b></div><div><br /></div><div>When I started programming last century, there were two languages I could choose to write my software in. One was a high-level language and the other was assembly language. While all but the most time critical code was written in the high-level language, I sometimes found it necessary to look at the generated assembly code to understand why my program was failing or to optimise the code. Now, I never look at the object code as I never question the quality of the object code generated by the compilers. However badly the code is written, the compiler would normally ensure that the generated code is efficient; this often means that engineers can get away with sloppy or bad code as it is often automatically refactored in the background </div><div>by the compiler to something more optimal. It also encourages, in my opinion, the lazy programmer who understands that the compiler does all the hard work in terms of 'writing' good code. </div><div><br /></div><div>The increasing use of model driven development (MDD) as part of a <a href="http://www.omg.org/mda/">model driven architecture (MDA)</a> as a way of improving software productivity is moving the goalposts again. In its purest form, the development uses visual tools and the generated code (in a high level language) and subsequent object code is never seen (by a human). However, I have yet to see any real world MDD which doesn't involve some algorithmic code still being written by hand in a conventional high-level language. Over time, the increasing use of MDD should result in less 'algorithmic' code being written which will, by implication, reduce the potential for less code to decay.</div><div><br /></div><div>MDD offers the opportunity for engineers to focus on the design rather than the implementation which should result in more maintainable systems which can readily be adapted to include enhancements and changes throughout their life. As the generated code is not the primary artifact which engineers work with, the quality of the generated code becomes less important. However, a key question to consider is does the use of MDD lead to the potential of (over time) of creating a decaying model and if so, how do you prevent it? And can you perform 'bad' MDD? </div><div><br /></div><div><span class="Apple-style-span" style="font-weight: bold; ">Reuse (of code snippets)</span></div></div><div><div><br /></div><div>There are an increasing number of source code repositories (for example see <a href="http://www.smashingmagazine.com/2009/07/21/45-excellent-code-snippet-resources-and-repositories/">here</a>) now available on the Internet offering a variety of resources from simple algorithms to reusable components. All of these repositories help in providing a (hopefully!) proven way of solving a particular problem, and should also be written in such a way to be readable in case the snippet needs to be tweaked in the future. Although I cannot advocate that the quality of all snippets will meet the criteria of not being 'bad' code, it is reasonable to expect that the many will be examples of 'good' code. If the code snippets can be used 'as is' without modification the code should remain maintainable; if the code is modified, the style of the original code should normally be preserved so that the code remains 'good'.</div><div><br /></div><div>Although the repositories do not require that the code passes any quality checks with regards maintainability etc, it should become obvious that the better code will be downloaded more frequently.</div><div><br /></div><div><span class="Apple-style-span" style="font-weight: bold; ">Use Multiple Compilers</span></div><div><br /></div><div>I have always advocated compiling code with two different compilers as a way of improving code quality. No two compilers are ever the same as each one has different strengths and weaknesses. I have also always promoted 'clean' compilation i.e. ensuring that all code compiles without warnings once the set of compile options have been defined. If the code compiles cleanly with two different compilers, there is an increased probability that the code is well-structured, which IMO implies that the code will be more maintainable. It should also help testing as a number of latent faults can often be removed prior to run-time.</div><div><br /></div><div>Clearly if the multiple compiler approach is adopted, this must be used at all subsequent code evolutions; if not the code will clearly decay albeit more slowly than if a single compiler has been used.</div><div><br /></div><div><b>Conclusion</b></div><div><b><br /></b></div><div>Writing maintainable software requires skill. With a bit of thought (and resisting the temptation to code the first thing that comes into your head), quality code can be produced using one or more of the techniques outlined which will support future product evolutions in a cost effective manner. </div><div><br /></div></div>Mehttp://www.blogger.com/profile/09171556704204860197noreply@blogger.comtag:blogger.com,1999:blog-1846273104553880352.post-86921538690783265202009-04-17T21:04:00.008+01:002009-04-26T19:24:25.576+01:00Some thoughts on Kanban in Software DevelopmentI recently watched a presentation given by <a href="http://www.infoq.com/presentations/kanban-for-software">David Anderson at QCon 2008</a> on his experience and observations of a kanban system applied in a software engineering environment. I found some of the observations very interesting, particularly the approach that optimising the system for lowest cost didn't actually result in the most efficient system in terms of optimal throughput. As with many great ideas, kanban appears to be a very simple concept; however clearly there is more to it than just a simple approach at managing a flow of post-it notes on a whiteboard.<div><br /></div><div>I was keen to understand more about kanban and how it might be applied in various project scenarios. At the <a href="http://www.spaconference.org/spa2009">SPA2009 conference</a>, there was a session presented by Karl Scotland on his take on kanban (<a href="http://availagility.wordpress.com/2008/10/28/kanban-flow-and-cadence/">Kanban, Flow and Cadence</a>) together with a very interactive BoF session on Lean and Kanban. Whilst the comparison of lean, based on a model, originally promoted by Toyota, of optimising production, against kanban, which limits the work in progress was a simple and very understandable comparison of the two approaches, it was when the debate moved to seeing how both lean and kanban could be applied to software developments that the debate got really interesting. </div><div><br /></div><div>Clearly software developments come in all shapes and sizes and the audience clearly represented a good cross section of developments and associated practices. Both approaches are clearly well suited to an agile approach in which features are developed to provide a flow of (increasing) value to the end user. This works particularly well in an iterative development where frequent delivery is encouraged, especially where the features are being evolved based on user feedback. This also requires a tolerant customer who can accommodate some failures. An interesting statement of 'get it right second time' was promoted as acceptable provided you learn from mistakes. I find that hard to accept as the 'norm' because it is completely ignoring any recognition of embedding quality in a delivered product. It might be acceptable in a prototype development but not in a production environment. Clearly there was something missing. There was! The integration step. Now I consider integration to be a very important (and potentially very expensive) stage, which is often squeezed in terms of time (how do you know you have finished?), and can't be omitted.</div><div><br /></div><div>So could kanban work in a more traditional waterfall development approach? Maybe. In a traditional approach, the requirements are analysed before design, implementation and testing are performed. Using a kanban system, a flow of features could be pushed through the separate stages of design, implementation and testing before handing over to integration, with limits being imposed at each stage. This would probably require that the 3 life cycle stages are performed by different team members, a not unreasonable expectation in a traditional development. Provided the requirements are relatively stable and the ordering and independence of features is organised to support a sensible integration approach in order to deliver increasing value (or functionality) then this could be an interesting approach. Managing the queues at each stage of the life cycle needs to be carefully managed since the costs of managing each queue must be proportional to the size of the queue otherwise the process becomes very inefficient. </div><div><br /></div><div>Now for the challenge. How likely is it that each feature will be take approximately the same time through the life cycle? - probably not. In my experience, it is highly unlikely that each requirement (or feature) will be equivalent in terms of effort expended, particularly when the software forms part of a complex system. This will result in some initial inefficiency as it would take some time for the queues to become populated with tasks. There is the temptation to process easy requirements first as a way of seeding the work queues quickly. However, from a project management perspective this is probably not what is required as it leaves the difficult tasks (and hence risks) to the end (and most difficult tasks will probably impinge on the easier tasks in some form resulting in some additional unplanned work).</div><div><br /></div><div>Looking back at the David Anderson presentation, it is clear why a kanban approach worked well in one of the case studies since the approach was only applied to bug fixing rather than in a new development. I believe that looking at kanban in a multiple life cycle development needs a different approach for development with the queues being placed at different points in the system life cycle (as distinct from the software life cycle) to avoid bottlenecks building up. I would advocate that the same developer should be responsible for design, implementation and testing of each feature so that the queue limits are placed on the exit from analysis (or requirements definition) and entry into integration rather than within the different stages of software development. In a lean or agile development this approach is normal since the developers are performing all stages of the life cycle; in waterfall developments, and particularly large systems, this is not the norm. </div><div><br /></div><div>Adopting kanban in waterfall certainly isn't new (see <a href="http://www.infoq.com/articles/hiranabe-lean-agile-kanban">Kenji Hiranbe's article)</a> but there is little evidence currently to demonstrate if it can deliver tangible benefits for all types of project approaches. Are there any case studies to prove or disprove my hypothesis?</div>Mehttp://www.blogger.com/profile/09171556704204860197noreply@blogger.comtag:blogger.com,1999:blog-1846273104553880352.post-74650958149222644112009-04-13T23:21:00.004+01:002009-04-13T23:58:12.173+01:00Web Security - an eyeopenerAn impromptu Birds of a Feather (BoF) session given by <a href="http://www.linkedin.com/in/misell">Dave Misell </a>at the <a href="http://www.spaconference/spa2009/">SPA Conference</a> admirally demonstrated what happens behind your back when you visit a website. Key to the demonstration was showing (very simply) how much data is passed between sites using cookies and how much data is retained (for considerable length of time in some cases) between site visits.<br /><br />The session used the <a href="http://www.parosproxy.com/">Paros</a> proxy server with Firefox which easily showed the web transactions when entering a simple URL (I won't disclose the URLs used in the demonstration, suffice to say that the sites were well-known). Using Paros, it is easy to see how much information is passed via cookies. Stopping cookies results in the same information being passed via URL (although this is more obvious since the data appears in some form in the address bar). In many cases the same data is passed to a number if websites regardless of whether the data is appropriate or useful to the receiving website.<br /><br />Although it is possible to see the information being sent to a website, it is not possible to determine what is done with the data by the receiving site. In most cases the data is used to trace a journey through website (so that the 'user experience' can be improved) which is totally transparent to the user since these are performed server side.<br /><br />So is it possible to stop the information being transferred? Not easily, but choose your sites carefully. There is some legislation in the EU which has tightened up the exchange of information through websites, particulalry to third parties, without the express permission of the user. Unfortunatley this legislation is dependent on where the website is hosted, which isn't always obvious from a simple URL. There is always 'education' which applies to both the end user and also to the developers of the site, and there are some good courses now which can help (e.g. <a href="http://www.foundstone.com/">ethical hacking</a> and <a href="http://www.isg.rhul.ac.uk/">MSc in Information Security</a> at Royal Holloway College in London) increase awareness.Mehttp://www.blogger.com/profile/09171556704204860197noreply@blogger.comtag:blogger.com,1999:blog-1846273104553880352.post-11652127458856488182009-04-09T23:45:00.006+01:002009-04-13T22:55:49.236+01:00Can you scrum on your own?An interesting question posed by <a href="http://www.pbell.com/">Peter Bell</a> at this year's <a href="http://www.spaconference.org/spa2009">SPA conference</a>. His premise was that some of the lean and agile approaches can be equally applicable when you are THE team. He called his approach<a href="http://www.pbell.com/index.cfm/2007/6/17/Solo-Scrums"> Solo Scrum.</a><br /><br />Peter identified a couple of scenarios where he felt that 'Solo Scrum' could be appropriate:<br /><ul><li>A full time developer doing open source development part-time</li><li>A solo consultant developing applications for non-technical clients</li><li>A 'work at home' contractor managing multiple clients</li></ul>Clearly working on your own brings a number of challenges (note that I didn't say problems!) which need to be considered when working on your own. I think that probably one of the biggest challenges to overcome, (and this is based on personal experience), is to try and avoid too many distractions within the 'home' work area. The second challenge is that of loneliness! In the many (professional) development projects that I have worked, talking to colleagues has been a great way of confirming your thoughts for a solution or helping to find a way through a problem. When you are on your own, this isn't possible, or is it? However the use of technology can certainly help (think of IM, email, VOIP (with video)) and it can certainly be seen as 'virtual' pair programming. This isn't a new idea (see <a href="https://c2.com/cgi/wiki?VirtualPairProgramming">here</a>); however, there is still the problem of finding a suitable 'pair' to link up with.<br /><br />The traditional problems of planning don't go away when working on your own (as I said at the session 'Fail to plan, plan to fail'). Working on your own necessitates an efficient process for planning and managing the plan (a bit more than the 'back of an envelope' note). There are a number of Eclipse plugins (e.g. <a href="http://www.eclipse.org/mylyn/">mylyn</a>) which offer the right balance in terms of monitoring the work activity without distracting from the task-at-hand.<br /><br />Some interesting ideas were discussed, including the observation that many team collaboration software packages often includes a free version for a small team (where small can be up to 5). This can overcome a often heard comment which was that I don't have the time/experience/etc to set up a 'professional' development environment (the minimum being a source control system, an IDE and a bug tracking system) from scratch.<br /><br />Peter has now started a <a href="http://groups.google.co.uk/group/solo-scrum">Google group</a> to further the debate. I will watch with interest.Mehttp://www.blogger.com/profile/09171556704204860197noreply@blogger.comtag:blogger.com,1999:blog-1846273104553880352.post-76157931990010339402009-04-09T23:29:00.004+01:002009-06-18T23:30:46.807+01:00SPA2009<a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://4.bp.blogspot.com/_WgfjXTvj12I/Sd52-7XY-UI/AAAAAAAAACg/KHbd46YYYq4/s1600-h/spa2009.gif"><img style="float:left; margin:0 10px 10px 0;cursor:pointer; cursor:hand;width: 118px; height: 80px;" src="http://4.bp.blogspot.com/_WgfjXTvj12I/Sd52-7XY-UI/AAAAAAAAACg/KHbd46YYYq4/s400/spa2009.gif" border="0" alt="" id="BLOGGER_PHOTO_ID_5322822633058269506" /></a>Having attended <a href="http://iamasoftie.blogspot.com/2008/03/spa-2008.html">last year's conference</a> and thoroughly enjoyed it, I was disappointed that my work commitments meant that I could only attend a single day of this <a href="http://www.spaconference.org/spa2009">year's conference</a>. Attending a single day meant that I was keen to catch as much of the buzz as possible as well as catching up with some friends. I managed to attend 2 sessions, on AJAX web testing using Selenium and Solo Scrum, together with 2 BoF sessions on Web security (or lack of it) and Lean/Kanban in a stimulating day. I will summarise my notes over the next few days.Mehttp://www.blogger.com/profile/09171556704204860197noreply@blogger.comtag:blogger.com,1999:blog-1846273104553880352.post-26915680788545214742009-03-20T21:12:00.004+00:002009-03-20T21:36:24.367+00:00Case Studies in Enterprise Architecture<p style="margin-bottom: 0cm">A few nights ago, I attended a fascinating evening with <a href="http://blogs.sun.com/eclectic">Wayne Horkan</a>, the CTO of Sun Microsystems for UK and Ireland, who presented his views on Enterprise Architecture. The event was jointly organised by <a href="http://www.iet.org/">IET, </a>the <a href="http://www.bcsmanchester.org.uk/">BCS Manchester branch </a>with a pitch for the recently formed <a href="http://www.ea.bcs.org/">BCS Enterprise Architecture Specialist Group</a> forming the warm act before Wayne's presentation.</p> <p style="margin-bottom: 0cm">Wayne presented his experience in 3 (anonymous) organisations who experienced difficulties in their approach to delivering an enterprise architecture. The problems could easily be addressed by recognising the key and important role of enterprise architects who maintain an understanding of the business context, provides domain knowledge and ensure that the architecture and supporting IT remains aligned with the business. </p><p style="margin-bottom: 0cm">In assessing any programme, a simple 4 stage approach is followed:<br /></p> <p style="margin-bottom: 0cm"></p><ol><li>Gather the facts by a series of interviews and information gathering. This includes understanding the current 'as is' enterprise architecture, the current IT/IS setup and the 'vision' for the future.<br /></li><li>Analyse the data<br /></li><li>Determine the key conclusions/recommendations and the way foward. This includes establishing the order of change to ensure that the changes deliver value to the organisation without destablising the exsiting organisation.<br /></li><li>Assess the project against the five key areas of likely project failure:<br /></li></ol><p></p> <p style="margin-bottom: 0cm"></p><ul><li>Strategy. Is it really needed? Why is it needed now? Will it deliver a realistic ROI?<br /></li><li>Contractual. Is the contract, including the financial elements, appropriate defined and achievable?<br /></li><li>Process/procedures. Are we in control or out of control?<br /></li><li>Politics and personnel. Is there buy-in from all concerned/affected?<br /></li><li>Technical and architectural. The least likely cause of project failure (apparently) but need to ensure that the right resources are available in order to deliver the right solution.<br /></li></ul><p></p> <p style="margin-bottom: 0cm">Wayne concluded with a number of best practices based on his experiences with these (and other) organisations:<br /></p> <ol><li><p style="margin-bottom: 0cm">Ignore the religious arguments about which framework to use. Just pick a single framework and stick with it. Most organisations use a blend of <a href="http://www.zachmaninternational.com/index.php/the-zachman-framework">Zachman</a> (artefact focused) and <a href="http://www.opengroup.org/togaf/">TOGAF</a> (procedurally focused). None of the frameworks provide a magic bullet with which success is guaranteed. </p> <p style="margin-bottom: 0cm"></p> </li><li><p style="margin-bottom: 0cm">Obtain sponsorship from key stakeholders. The stakeholders must have a strategy for the business, which may change over time, that is communicated to the EA team.</p> <p style="margin-bottom: 0cm"></p> </li><li><p style="margin-bottom: 0cm">The EA team must continually:</p> </li></ol> <ol> <p style="margin-bottom: 0cm"></p> </ol> <ul> <li><p style="margin-bottom: 0cm">Remain delivery focused, providing value to the organisation</p> <p style="margin-bottom: 0cm"></p> </li><li><p style="margin-bottom: 0cm">Involve the whole team (consisting of Enterprise, Infrastructure and Application architects) in all decisions, ensuring that cross-team communication is integral to the team (no silos)</p> <p style="margin-bottom: 0cm"></p> </li><li><p style="margin-bottom: 0cm">Ensure that there is the right mix of technical skills to deliver the right architecture</p> <p style="margin-bottom: 0cm"></p> </li><li><p style="margin-bottom: 0cm">Obtain and foster Sponsorship, Sponsorship, Sponsorship without which a programme is likely to fail</p> <p style="margin-bottom: 0cm"></p> </li><li><p style="margin-bottom: 0cm">Keep aligned (and checking) with the business strategy</p> <p style="margin-bottom: 0cm"></p> </li></ul> <p style="margin-bottom: 0cm">As a reality check, Wayne highlighted that the current economic climate now requires that the investment in enterprise architecture must be seen to deliver value far more quickly (i.e. in the matter of months rather than years) to the business than has previously been tolerated.</p><p style="margin-bottom: 0cm">The session was well attended and left the audience with much to consider in future projects.</p> <p style="margin-bottom: 0cm">The slides, and Wayne's personal views on the session are contained <a href="http://blogs.sun.com/eclectic/entry/yesterdays_ea_case_studies_presentation">here</a>.<br /></p> <p style="margin-bottom: 0cm"><br /></p>Mehttp://www.blogger.com/profile/09171556704204860197noreply@blogger.com0tag:blogger.com,1999:blog-1846273104553880352.post-57404441343489828592009-03-01T16:53:00.013+00:002009-03-01T20:39:54.803+00:00Going forward with PHPI run a number of websites which provide a daily update of a shareprice together with the value of the shareprice converted to a loical currency (e.g. £, $, Aus$). When I started, I selected Perl as my scripting language which provided me the necessary facilities (the LWP package) to retrieve a shareprice from the Yahoo finance site. I used the URL<div><span class="Apple-style-span" style="font-family:'courier new';"><blockquote><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;">http://finance.yahoo.com/d/quotes.csv?f=l1d2c1p2&s='.$stock_symbol</span></span></blockquote></span></div>as my data page which returned the share's closing price and other useful information.<br /><br /><div>I also generated a graph which showed the historical share price, relative to one of the major indices (e.g. FTSE, CAC40).</div><br /><div>As I couldn't host my scripts on my web site, I was required to run a number of Perl scripts to generate updated files and then upload the files to the webhost. I wrapped the scripts in a DOS batch file and set up a scheduled task (under Windows) to run the scripts every weekday after the markets had closed.</div><br /><div>This worked very well, most of the time. However it suffered from a number of problems. If I wasn't around, and didn't turn my PC on, then the scripts didn't run and the shareprice wasn't updated. I also suffered from some problems with my web connection which meant that the scripts didn't run as expected and I obtained a zero shareprice.</div><br /><div>I had been wondering how to make the system better and less dependent on my PC being turned on. Most ISPs that I have experienced don't allow the LWP package to be installed as part of the standard Perl installation, presumably based on security concerns. I therefore considered alternative scripting languages. I had already seen <a href="http://iamasoftie.blogspot.com/2008/03/learning-another-language-beginning.html">Python at SPA2008</a> and was keen to try and see Python was a suitable replacement. However, my web host didn't offer Python in its standard configuration, but did offer the other P language, PHP. </div><br /><div>PHP treats a URL as a filename and does not appear (in the standard configurations installed on webhosts) to suffer from the security constaints that Perl installations suffer. A simple function retrieves the values for a stock symbol returning the parameters in an array.</div><blockquote><div><div><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;">function get_share_price($stock_symbol)</span></span></div><div><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;">{</span></span></div><div><span class="Apple-tab-span" style="white-space:pre"><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;"> </span></span></span><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;">$url="http://finance.yahoo.com/d/quotes.csv?f=l1t1d1c1p&s=".$stock_symbol;</span></span></div><div><span class="Apple-tab-span" style="white-space:pre"><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;"> </span></span></span><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;">$filesize = 2000;</span></span></div><div><span class="Apple-tab-span" style="white-space:pre"><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;"> </span></span></span><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;">$handle = fopen($url, "r");</span></span></div><div><span class="Apple-tab-span" style="white-space:pre"><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;"> </span></span></span><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;">$arr = fgetcsv($handle , $filesize , ',');<br /></span></span></div><div><span class="Apple-tab-span" style="white-space:pre"><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;"> </span></span></span><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;">fclose($handle);</span></span></div><div><span class="Apple-tab-span" style="white-space:pre"><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;"> </span></span></span><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;">return $arr;</span></span></div><div><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;">}</span></span></div></div><div></div></blockquote><div>This resolved the problem of updating a daily share price, and also offered the opportunity for offering live share price updates (subject to the market delays which free services such as Yahoo suffer - at least 15 minutes delay).</div><br /><div>To resolve the historic share price required PHP again but this time with a different Yahoo feed which I had only recently discovered, the ichart interface which returns the data as a stream of CSV separated values. </div><br /><div><div><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-tab-span" style="white-space:pre"> </span>function plot($numdays, $stock_symbol)</span></span></div><div><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-tab-span" style="white-space:pre"> </span>{</span></span></div><div><span class="Apple-tab-span" style="white-space:pre"><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-tab-span" style="white-space:pre"> </span> </span></span></span><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;">// Get current date</span></span></div><div><span class="Apple-tab-span" style="white-space:pre"><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-tab-span" style="white-space:pre"> </span> </span></span></span><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;">list($e,$d,$f)=split('-',date("j-n-Y"));</span></span></div><div><span class="Apple-tab-span" style="white-space:pre"><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-style-span" style="color: rgb(0, 0, 0); font-size: 16px; white-space: normal; "><span class="Apple-tab-span" style="white-space: pre; "><span class="Apple-style-span" style="color: rgb(255, 0, 0); "><span class="Apple-style-span" style="font-size: 10px; "> </span></span></span><span class="Apple-style-span" style="color: rgb(255, 0, 0); "><span class="Apple-style-span" style="font-size: 10px; ">$d--;<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-style-span" style="color: rgb(0, 0, 0); font-size: 16px; "><span class="Apple-tab-span" style="white-space:pre"><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: 10px; "> </span></span></span><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: 10px; ">// Months are 0-11</span></span></span></span></span></span></span></span></span></div><div><span class="Apple-tab-span" style="white-space:pre"><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-tab-span" style="white-space:pre"> </span></span></span></span></div><div><span class="Apple-tab-span" style="white-space:pre"><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;"> <span class="Apple-tab-span" style="white-space:pre"> </span></span></span></span><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;">$yahoostr="http://ichart.yahoo.com/table.csv?s=".$stock_symbol."&a=0&b=1&c=2003&d=".$d."&e=".$e."&f=".$f."&g=d&ignore=.csv";</span></span></div><div><span class="Apple-tab-span" style="white-space:pre"><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: 10px; "><span class="Apple-tab-span" style="white-space:pre"> </span> </span></span></span><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: 10px; ">$arrResult = array();</span></span><br /></div><div><span class="Apple-tab-span" style="white-space:pre"><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: 10px; "><span class="Apple-tab-span" style="white-space:pre"> </span> </span></span></span><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: 10px; ">$min=1000;</span></span><br /></div><div><span class="Apple-tab-span" style="white-space:pre"><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;"> <span class="Apple-tab-span" style="white-space:pre"> </span></span></span></span><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;">$max=0;</span></span></div><div><span class="Apple-style-span" style="color: rgb(255, 0, 0); font-size: 10px;"><span class="Apple-style-span" style="color: rgb(0, 0, 0); font-size: 16px; "><span class="Apple-tab-span" style="white-space: pre; "><span class="Apple-style-span" style="color: rgb(255, 0, 0); "><span class="Apple-style-span" style="font-size: 10px; "><span class="Apple-tab-span" style="white-space: pre; "> </span> </span></span></span><span class="Apple-style-span" style="color: rgb(255, 0, 0); "><span class="Apple-style-span" style="font-size: 10px; ">$handle = fopen($yahoostr, "r");</span></span></span><br /></span></div><div><span class="Apple-tab-span" style="white-space:pre"><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-tab-span" style="white-space:pre"> </span> </span></span></span><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;">if( $handle ) </span></span></div><div><span class="Apple-tab-span" style="white-space:pre"><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-tab-span" style="white-space:pre"> </span> </span></span></span><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;">{</span></span></div><div><span class="Apple-tab-span" style="white-space:pre"><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-tab-span" style="white-space:pre"> </span> </span></span></span><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;">$n=0;</span></span></div><div><span class="Apple-tab-span" style="white-space:pre"><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;"> <span class="Apple-tab-span" style="white-space:pre"> </span> </span></span></span><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;">while (($data = fgetcsv($handle, 1000, ",")) !== FALSE) </span></span></div><div><span class="Apple-tab-span" style="white-space:pre"><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-tab-span" style="white-space:pre"> </span> </span></span></span><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;">{</span></span></div><div><span class="Apple-tab-span" style="white-space:pre"><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-tab-span" style="white-space:pre"> </span> </span></span></span><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;">if ($n == 0)</span></span></div><div><span class="Apple-tab-span" style="white-space:pre"><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-tab-span" style="white-space:pre"> </span> </span></span></span><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;">{</span></span><span class="Apple-tab-span" style="white-space:pre"><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;"> </span></span></span></div><div><span class="Apple-tab-span" style="white-space:pre"><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;"> <span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-style-span" style="white-space: normal; ">$n=1;<span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-style-span" style="color: rgb(0, 0, 0); font-size: 16px; "><span class="Apple-tab-span" style="white-space:pre"><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: 10px; "> </span></span></span><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: 10px; ">// Ignore header row</span></span></span></span></span></span></span></div><div><span class="Apple-tab-span" style="white-space:pre"><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;"> <span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-style-span" style="color: rgb(0, 0, 0); font-size: 16px; white-space: normal; "><span class="Apple-tab-span" style="white-space:pre"><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: 10px; "> </span></span></span><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: 10px; ">}</span></span></span></span></span></span></div><div><span class="Apple-tab-span" style="white-space:pre"><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;"> <span class="Apple-tab-span" style="white-space:pre"> </span> </span></span></span><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;">else</span></span></div><div><span class="Apple-tab-span" style="white-space:pre"><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;"> <span class="Apple-tab-span" style="white-space:pre"> </span> </span></span></span><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;">{</span></span></div><div><span class="Apple-tab-span" style="white-space:pre"><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;"> <span class="Apple-tab-span" style="white-space:pre"> </span> </span></span></span><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;">$arrResult[] = $data;</span></span></div><div><span class="Apple-tab-span" style="white-space:pre"><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;"> <span class="Apple-tab-span" style="white-space:pre"> </span> </span></span></span><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;">}</span></span></div><div><span class="Apple-tab-span" style="white-space:pre"><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-tab-span" style="white-space:pre"> </span> </span></span></span><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;">}</span></span></div><div><span class="Apple-tab-span" style="white-space:pre"><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;"> <span class="Apple-tab-span" style="white-space:pre"> </span> <span class="Apple-tab-span" style="white-space:pre"> </span></span></span></span><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;">fclose($handle);</span></span></div><div><span class="Apple-tab-span" style="white-space:pre"><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: 10px; "><span class="Apple-tab-span" style="white-space:pre"> </span> </span></span></span><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: 10px; ">// Select data subset</span></span><br /></div><div><span class="Apple-tab-span" style="white-space:pre"><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-tab-span" style="white-space:pre"> </span> </span></span></span><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;">$arr1=array_slice($arrResult,0,$numdays);</span></span></div><div><span class="Apple-tab-span" style="white-space:pre"><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;"> <span class="Apple-tab-span" style="white-space:pre"> </span></span></span></span><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;">// Determine chart scale</span></span></div><div><span class="Apple-tab-span" style="white-space:pre"><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-tab-span" style="white-space:pre"> </span> </span></span></span><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;">foreach ($arr1 as $data ) </span></span></div><div><span class="Apple-tab-span" style="white-space:pre"><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-tab-span" style="white-space:pre"> </span> </span></span></span><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;">{ </span></span></div><div><span class="Apple-tab-span" style="white-space:pre"><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;"> <span class="Apple-tab-span" style="white-space:pre"> </span></span></span></span><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;">$min = min($data[4], $min); </span></span></div><div><span class="Apple-tab-span" style="white-space:pre"><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-tab-span" style="white-space:pre"> </span></span></span></span><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;">$max = max($data[4], $max); </span></span></div><div><span class="Apple-tab-span" style="white-space:pre"><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-tab-span" style="white-space:pre"> </span> </span></span></span><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;">} </span></span></div><div><div><span class="Apple-tab-span" style="white-space:pre"><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-tab-span" style="white-space:pre"> </span> </span></span></span><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;">// Data is currently stored Newest-> Oldest.</span></span></div><div><span class="Apple-tab-span" style="white-space:pre"><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-tab-span" style="white-space:pre"> </span> </span></span></span><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;">$arr = array_reverse($arr1);</span></span></div></div></div><div><span class="Apple-tab-span" style="white-space:pre"><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;"><span class="Apple-tab-span" style="white-space:pre"> </span> </span></span></span><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;">// Now plot data (date in element 0, value in element 4).<br /></span></span></div><div><span class="Apple-style-span" style="color: rgb(255, 0, 0); font-size: 10px;"><span class="Apple-tab-span" style="white-space:pre"> </span>}<br /></span></div><div><span class="Apple-tab-span" style="white-space:pre"><span class="Apple-style-span" style="color: rgb(255, 0, 0);"><span class="Apple-style-span" style="font-size: x-small;"> <span class="Apple-style-span" style="white-space: normal; "><span class="Apple-tab-span" style="white-space:pre"> </span>}</span></span></span></span></div><div><br /></div><div>In processing the data, I discovered a number of additional PHP features, including</div><div><ul><li><span class="Apple-style-span" style="font-weight: bold;">array_reverse</span> -to reverse an array (required because the Yahoo data is returned with the most recent data first)</li><li><span class="Apple-style-span" style="font-weight: bold;">array_slice</span> -to take a subset of an array</li><li><span class="Apple-style-span" style="font-weight: bold;">min, max</span> - to determine the minimum and maximum values (how many times has code for this suimple function been written!)<br /></li></ul></div>This meant that I had the data in a format which I could then process to produce a graph (dynamically). Now the next question was to select a suitable graphing package which offered a good PHP interface. But that is another story.Mehttp://www.blogger.com/profile/09171556704204860197noreply@blogger.com0tag:blogger.com,1999:blog-1846273104553880352.post-15794367450555615472008-10-14T22:54:00.007+01:002008-10-14T23:29:15.842+01:00The Power of the DoorI have just read an interesting presentation on the <a href="http://lostgarden.com/Rules%20of%20Productivity.pdf">rules of productivity</a>. It presents 8 rules to determine the most productive number of hours per week (40), the type of office environment (team rooms) and how a team should be located (non-siloed - multi-disciplined). In most cases the results are possible counter-intuitive to most management but each of the results are backed up by sound evidence.<br /><br />It made me think about my experiences on numerous development projects and I would tend to agree with the recommendations outlined in the presentation. I remember when I was working long hours AND studying for a post-graduate degree trying to write some very simple <a href="http://www.ecma-international.org/publications/standards/Ecma-367.htm">Eiffel</a> code and failing to get it work after 2 hours of staring at a simple logic problem. I simply couldn't solve it at the end of a 15 hour working day. I went home and came back a few days later, refreshed. I solved the problem in 5 minutes. The lesson was clear to me then - you need sleep, not heroes. This was admirably demonstrated in an overnight session to get a demo working and after 8 hours through the night of being in a no better situation than we were when we started!.<br /><br />As organisations have changed over the years, office space has become at a premium. The <span style="font-style:italic;"><span class="Apple-style-span" style="font-weight: bold;">'power of the door'</span><span style="font-weight:bold;"></span></span> was demonstrated when I was a young engineer doing a major retargetting exercise. I remember we had to convince our manager of the benefit of having our own server to work on this exercise - he agreed provided we could reduce the schedule by 6 months. This we did easily, not just because of having our own server but because we had a large wooden door on our small team office. If it was shut (which it was normally) people just walked past so you didn't get disturbed. I would guess that if we reverted back to small offices, the software industry would be much more productive than it is today. I wonder how many project managers have the power or insight to challenge the office environment and make the necessary changes that will increase the chance of project success and increase team morale. Mehttp://www.blogger.com/profile/09171556704204860197noreply@blogger.com0tag:blogger.com,1999:blog-1846273104553880352.post-38649508166611761492008-09-11T22:52:00.010+01:002008-09-16T22:12:12.063+01:00The role of Project Managers in Agile Projects<a onblur="try {parent.deselectBloggerImageGracefully();} catch(e) {}" href="http://www.proms-g.bcs.org"><img style="float:left; margin:0 10px 10px 0;cursor:pointer; cursor:hand;" src="http://1.bp.blogspot.com/_WgfjXTvj12I/SMmilgY3F_I/AAAAAAAAACA/uYzerxFABi4/s400/promsg.PNG" border="0" alt="PROMS-G Logo" id="BLOGGER_PHOTO_ID_5244902006282721266" width="120px"/></a><br />I recently attended <a href="http://www.allankelly.net">Allan Kelly's</a> presentation on <a href="http://www.proms-g.bcs.org/eventbooking/showevent.php?eventid=psg0801">'Why and How to Become Agile'</a>, an event organised by the <a href="http://www.proms-g.bcs.org">BCS Project Management Specialist Group</a>. As you would expect from an experienced agile practitioner, Allan provided a good overview of what agile was and why he considered agile to be better. <br /><br />However, given that the audience was mainly project managers of one sort or another, I was interested in his statements on the role of project managers in agile developments. Mainstream agile methods such as <a href="http://www.xprogramming.com/xpmag/whatisxp.htm">XP</a>, <a href="http://www.controlchaos.com/about/">SCRUM</a> or <a href="http://alistair.cockburn.us/index.php/Crystal_light_methods">Crystal</a> are very silent about the role of the Project Manager. This doesn't mean that they can be dispensed with, it is just that the various agile developments concentrate on the approach to improving business value with a development rather than the associated management tasks. I know agile promotes self-organising teams (in my experience this is removing a hierarchy of developers, architects and testers) but I know of few organisations that don't allocate a project manager to a development project however small. I therefore advocate that the project manager role remains as critical as ever although the scope of some of his tasks may change. <br /><br />Regardless of the development approach, all projects need to manage risk, budgets, communications and resources (physical and people). The classic management approach is for these tasks to be allocated to a (often dedicated) project manager so that the workers can get on with serious development work. I don't think there is any need for this approach to significantly change. One task which Allan indicated would change for the project manager was planning as the emphaisis on planning changes in agile developments to be much less formal. This may be appropriate for (small) purely software development projects but the vast majority of projects are multi-disciplined in which the dependencies between the various activities need to be accepted and understood by all parties regardless of the formality of capturing this information. The project manager's role still remains an important factor in the eventual success of the project and the choice of project manager is probably more important than ever. In my experience the best project managers for developments adopting some agile practices are those who are hands-on, are developers who have experienced agile first-hand, understand the project and are empowered to make decisions. It is the last point which mustn't be under-estimated; delays in decision making processes are classic signs of a development that is struggling and agile developments can't afford unnecessary delays.<br /><br />I note that the latest <a href="http://www.dsdm.org">DSDM version (Atern)</a> now explicitly includes the role of a Project Leader (you can even get a <a href="http://www.dsdm.org/certification/apl_practitioner.asp">qualification</a>). Does this now recognise that the classic project manager now needs to more formally recognised in agile developments? I would probably say no; it is probably more a reflection on the type of organisations using DSDM who feel comfortable with an explicit role being defined rather a comment on the management of agile developments in general.<br /><br />Now that agile development approaches have become more accepted as a 'normal' way of performing software developments, it is probably true to say that the role of the project manager has survived relatively intact. However the project manager now has a key role in being much more involved with the development rather than performing a purely managerial overview role; but this is what the best project managers have always done.Mehttp://www.blogger.com/profile/09171556704204860197noreply@blogger.com0tag:blogger.com,1999:blog-1846273104553880352.post-36372629979388512432008-07-15T21:42:00.002+01:002008-07-15T22:22:15.221+01:00Agility for Complex SystemsAt <a href="http://www.spaconference.org/spa2008/">SPA2008</a>, there was much debate amount the use of agile practices in developing software.<br /><br />A comment that I heard was 'that for all the noise that the agile community makes us believe that agile is mainstream and the most common approach in developing software today, the reality is somewhat different'. Certainly there are many good examples of where agile approaches have been successfully applied but I guess that many of these are in 'young' and 'smallish' companies and not in large, corporate companies which have many decades of history.<br /><br />It is not easy to convince these large and well-established companies that there are alternative ways of developing software. One reason, is often the 'process' police. Large companies, in my experience, are very keen on following a tried and tested approach, with some limited tailoring to cater for different sizes of developments.<br /><br />So what are the challenges in applying agile techniques to complex systems?Mehttp://www.blogger.com/profile/09171556704204860197noreply@blogger.com0