Improving Agile team performance

I have often been asked my opinions on teams that are not performing. Typically this is prompted by an observation of going through the motions on a Scrum ceremony that they have witnessed. I have seen the same situation a few times now so thought it worth sharing.

So consider this team:

They appear listless, little passion or focus. They have the necessary process and artefacts but more from a sense of obligation than because they are actually deriving any value from them. They probably get together as a whole team to discuss stuff right at the beginning of the sprint and then drift off into smaller groups at the start of sprint ending with a minor panic in the last couple of days leaving sprints incomplete. Standups are largely pointless, mumbled status updates to the Scrum Master. They will often respond with excuses of interdependencies or lack of understanding. Their “Definition of Done” is a little hazy and regularly compromised to get things though at the end of a sprint and for the same reason, smaller pieces of low priority work often get put into sprints to enable the team to take some easy wins – to keep THEM happy.

It is easy to point the finger at the team, kick them a little bit, maybe even shout and mention things like performance reviews, objectives and the like, but for a team to have gotten into this mess there must be something systemically wrong and pep talks will have short term benefits at best.

 

So the Scrum Master is at fault, well maybe. Typically I find the fault is in the system the team is in and either the Scrum Master is part of that system or has given up fighting. Watch the Scrum Master. If they are busy directing the team flow like a policeman at a busy junction, having quiet chats with the Product Owner and separately with Architects and Project Managers then yes, the Scrum Master is part of the system that is killing the team. If however – as I have often mentioned, the Scrum Master appears like the rest of the team, quiet, reluctant, weary and at a loss, then you probably have a respectable Scrum Master that isn’t sufficiently powerful to break the system and therefore is as beaten as the team.

 

A first starting point is to look at team size, most of the situations I have seen have been compounded by over sized teams. I know many people in the Agile Coaching industry that would argue that you can have teams up to 11 and yes you can get things to work with larger teams, but you need the system to be working well first; and many of the issues that affect smaller teams have a more significant impact on larger ones.

Agile is fundamentally about people, it is light on process by design, and process enables people to be directed forcibly. The Agile approach depends on self direction and self direction thrives on motivation. Fail to motivate the team and they will become despondent and without the structure and direction in a waterfall model, they will split, drift and performance will be a fraction of what is possible.

Motivation in an Agile context typically stems from empowerment and an appreciation of what they are working on, not just realisation of the final product but also a direct understanding of their element of it.

 

Given this there are some demotivational factors to look out for:

Component teams – delivery teams that are working on technological slices of the application have a much harder time to appreciate the point of their work, and therefore their impact on the value. Because it is harder to see the final solution it is also much harder for the team to drive out an MVP, component teams often gold plate because they struggle to be able to identify which features are the most critical.

 

Specialists – A team of specialists that are able to completely break up the work will find themselves operating in increased isolation. The handoffs between team members will start to become increasingly formal in an attempt to ensure responsibility. What this means is each team member looks out for themselves to ensure that when something goes wrong they are able to point the finger elsewhere. The team will start working when they act as a team, and that can be helped though cross functional delivery.

 

Knowledge of the users – have the team, not a management representative of the team, but the team themselves, ever actually met and talked to the actual users of the system? People will probably have spoken to the team about the users, may even have involved the team in creating personas but there is something very powerful about actually meeting them. Seeing the whites of the eyes of the people you are building things for. It is about consequence, about responsibility. It is harder to cut corners and deliver substandard product for someone you have actually met and have a relationship with.

 

Culture of Management – This is the most subtle and pervasive of the issues, and also the one that typically gets worse as performance issues grow creating a vicious circle. A poor team typically attracts more management attention, who will act to direct and control the flow and definition of work. What the team needs is empowerment, freedom to own their own process.  Did they write their own definition of done (and not have it “rephrased” by someone senior)? Have they chosen their own tracking and reporting templates?

A really important piece will be the decision makers. These decision makers need to be in the team and making decisions with the team – in the presence of their team members. This makes their decisions team decisions, as opposed to decisions made on behalf of the team. It is an important but subtle distinction. Project Managers and Architects are the two roles that this usually applies to.

 

In short if you have a team that appears to be drifting and disinterested then:

  • Reduce the team size as much as possible – facilitate the team to split maybe
  • Empower the team – stop making decisions outside the team on their behalf
  • Connect the team with the users
  • Enable the team to own their own process
  • Give the team a slice of the system where they can own the definition, delivery and deployment of something of value

 

Digital Transformation – No, just having a website doesn’t count!

Recently a colleague asked me about Digital transformation, and how best to express it; and more importantly, what practical steps to take to start to achieve it.

As I understand it, Digital is about using information to shape your product line and sales strategies to the ever changing market, using the technological capabilities of the modern age to tailor your product offering and operate at lower cost.

It is not achieved by simple having a website!

Moving to Digital is a business strategy that leans on IT’s capabilities, it is not an IT strategy. Digital is moving from selling products that consumers use to fulfil their needs to selling the actual use – servicing the NEED – your tangible product may be part of that but the engagement is based around the customer not around you.

To illustrate this, consider this evolution – a hard working peasant farmer with his cart of cabbages some time in the middle ages. One man – some cabbages, selling them to people that a) find him, b) want a cabbage for whatever purpose. Now fast forward a few hundred years and his cart has become a shop, another hundred years and a catalogue is published for mail order and then last week he launches cabbage.com. Massive progress, but hold on, still one enterprise, some cabbages, selling them to people that a) find him, b) want a cabbage for whatever purpose. Finding him is now easier due to the internet but fundamentally this is still the same concept.

This is not a digital transformation; this is channel shift to new channels as they become pervasive. There has been more than one company disappear from the market place that had full channel coverage but hadn’t innovated their product line.

Mr Cabbage man is still fundamentally selling the same product and expecting the same approach from the customer. The customer must still have a need that they identify with your product, approach you to purchase it and then use the product to address their need.

Going digital is more like the next step from the data driven product placements that have been so effectively used by the supermarkets. Through extensive data analysis of purchases, footfall, and buying patterns the supermarkets have been able to change their layout and merchandising to better tap into the underlying need of the consumer. Originally the shop would sell ingredients, it was the consumer’s need for a meal that drove that purchase but it was still down to the consumer to identify which ingredients were needed and later to assemble them into a meal. The introduction of ready meals in the 1980s was a product shift based on addressing user needs. Digital is the next step along that path.

Going to Digital means the ability to have a comprehensive understanding of your customer, and then offering services (based on known experience and products) tailored to the exact customer. Technology now gives the possibility of providing specific services to the individual and then automatically matching them to the best service offering. This is can often be done with minimal human decision involvement enabling much lower cost to operate.

Many IT buzzwords start to crop up at this point: Big data, CRM, analytics. These concepts are what really lies behind Digital, it isn’t really the IT stack or web screens – that is just what it needs.

Moving to Digital is considered a challenging cultural change because the change is not in the IT department – it is a change in the business, in the product offering and your understanding of their customers and your USP, IT will help you deliver the new world, but cannot lead it.

Many companies are now employing a Chief Digital Officer (CDO), this is a very difficult role because they are responsible for breaking out of IT, you could argue (if looking for sensationalism) that their job is to destroy the IT department – to bring technology into the heart of the business to be something we all do, not something “they” do.

It somewhat misses the point if the CDO sits under the IT director and is basically manager of online product offerings, although they would be in good company. I would imagine that the relationship between CIO and CDO would be at times tense, and to work they must be equals.

But practically speaking what are good steps to take to a digital transition:

  • Understand who your customers are, and more subtly why your customers are. What is it about you that makes them come to you – or “come back” to you.
  • Next ensure that the data that you are currently capturing is accessible – and cross referenceable, and not just along the existing product hierarchies. I remember back when I worked in retail that we could give really detailed sales information along product lines and store lines, so general merchandise, men’s shirts, that range, that size, in blue. But if you wanted aggregated info you were limited to that hierarchy, so you could find sales for all shirts in that range, but what was needed was sales for ALL BLUE clothes… but that basically meant you had to know the code for every blue item and run a query asking for each item in a list.
  • Then add some analytics on existing product lines.
  • Then get someone to actually look at this stuff, someone with a CRM background – there is too much critical work to just add this to someone’s existing work.

All these steps will help your organisation understand what it is selling, how and to who and why. From there they can try to understand what your customers are really wanting (of which your existing product is a part), and then you can start to consider what service you can offer to address that.

Don’t rush to solutionising a software build to a problem you haven’t validated and you can’t just point to a picture of your product on a web screen and declare digital success.

Empower your Scrum Masters

It is all very well people talking about Agile and how great it is, but giving good advice on how to take steps to get there, without being there, is difficult – and there is a good reason, change is a hugely personal, context dependent issue, therefore advice without an intimate understanding of your situation can be damaging. However I have recently aligned a few thoughts in my head following personal experiences, discussions with others and some powerful presentations I have attended, and believe that there are a few generic steps that can help.

There are three concepts that I think neatly reinforce each other: 

Firstly there is an acceptance that an empowered team is a positive force; Dan Pink’s work gives credence to what many Agile deliveries have empirically shown in recent years, but how to achieve empowerment? You can think of power as points, you can only redistribute them, not change the quantity as it is the ratios of power that are important. So giving more points to the team means that they are coming from somewhere, and that is usually the first hurdle – people don’t like ceding power.

Secondly, related to this, is the quandary that traditionally minded organisations have with how to promote a Scum Master. People have to move up, there are levels… and typically each level dictates to a greater or lesser degree to the level below. That DICTATION implies a lack of empowerment at the level below. Some organisations attempt to navigate round this by promoting Scrum Masters to Agile coaches; whilst this is a possible career path I think this unsustainable and it requires great patience and abstraction to be a good coach – so as to not become an “Agile” dictator, that imposes lots of good processes but fundamentally misses the point, disempowering the teams under their CONTROL.

A better answer to the Scrum Master promotion question, is one that starts to challenge the traditional hierarchy, instead of actively moving the person up, instead, remove the person above them. Now the Scrum Master retains their team awareness but can bring the responsibility and authority of the higher level to the team, it is like a vote of trust. It resonates with the concept that you promote a Scrum Master to a better Scrum Master.

The final point is one of MANAGEMENT. Management as a concept is really born out of Taylorist Scientific Management from the 1910s – 1920s. Fundamentally it separated the thinking from the doing. It aligns with McGregor’s 1960’s Theory X and Y premise, where X type people dislike work, prefer to be directed and often have to be forced.   When applied this made sense, a large majority of the workforce was uneducated and most work was functional rather than creative.  The auditors of Theory X can be labelled middle management, a swathe of most organisations that are responsible for “managing” or directing and coercing the work that others are doing.  

Many people (far better and more respected than I) have suggested that (at least here in Western Europe and the US) that the workforce and nature of work have progressed to a point where this approach is no longer appropriate and that the associated trappings such as strict hierarchies, work allocation, and most project controls are a legacy hangover. This is the age of Theory Y, where work is natural, people thrive in self direction,will actively seek responsibility and where achievement is its own reward.

I recently worked on a very successful project as a Scrum Master without a Project Manager, now there were lots of things that needed doing – many of them would fall into the project manager role description but a key thing that wasn’t missing was Management, we could have done with help on project timesheeting, project account keeping, project reporting, project risk and issue logging, all of these collectively could be called project administration, but not project management. The team was self managing, and it would have been detrimental to have imposed external management onto them.

So where am I going with this… I would suggest that if we accept that empowered teams are desired, and that Scrum Masters need some sort of career path and that middle management is a legacy of Taylorist ideology then… when approaching an Agile adoption, empower your teams by removing as much middle management as possible from around them, and critically from above them. However to cushion the transition and keep the rest of the organisation comfortable then supporting your teams with a project administrator may be a sensible move.

Own your process – or it will own you!

Process is a highly emotive topic, there are one set of extremists that believe that everything needs a process and even if you claim there isn’t one – actually if you look hard enough there is, and one the other side there are those that suggest that all process is evil and a waste of time.

Typically in these frothing debates the truth lies somewhere in the middle.

When you get down to the real ethos about Agile delivery it is a bunch of people getting down to do what is necessary to make a genuine improvement to a situation, reassessing their situation and then repeating.

Stripped bare and free of politics there are no requirements, no roles, no reporting, just people doing stuff. This works brilliantly in the teeny start up world, can you imagine documenting your first requirement when starting your own company, or allocating roles between 3 people as to who will do what? no of course not, everyone will do what is necessary, as well as they can, asking help when they need it.

So why doesn’t this scale? Even a single-team Scrum implementation brings a role, (Scrum Master) and definitions like User Story and Sprint and Velocity. What has happened?

Well critically the people doing the work are typically one step away from the passion of delivery, the real success criteria – the PURPOSE. Now we need to consistently communicate to a wide audience what is going on.

In a start up – the consequence of failure is so great that you’ll do whatever it takes to get over the line and don’t need to justify your self to anyone; at the other end of the spectrum with a heavily controlled PRINCE2 type environment it is entirely possible for an individual to do everything they are supposed to and deliver no value on a failing project.

As things continue to scale more process starts to creep in, this is usually as a consequence of lack of relationships, lack of a true, trusted connection. Process is a consequence of a problem – and not always the right solution. We need to acknowledge that the process is an inferior replacement to a trusted, close relationship and should be introduced only to the point where the people concerned are able to operate almost as well as they would in the context where it wasn’t needed. The process is not to be aspired to, it is the necessary evil. It is what you do when you can’t get people to work together due to location or contract or politics or possibly just personality

Process is a pollutant to your system – only introduce that which you truly need.

So how much do you need?

There really is only one way to find out, start with none and introduce process until you find things start to work and then ceaselessly attempt to remove it as people work together for longer. Generally each step you take will make the situation better and this lean (both in process and headcount) approach will be cost effective.

The alternative is the PRINCE2 or SAFe approach which starts with a HUGE pile of process, roles, terminology, gates, documentation etc and suggests that within it are all the pieces you need. If you are unsure exactly which bits you need in your entirely unique situation both in terms of context and people, then just apply everything. Once you have everything then you can start to remove bits.

But that is the difficult part, which bit to remove?  The consequence of change when operating this way is the reverse, change the wrong bit and you move from a ponderous expensive working system to a broken one. Also the people likely to be tasked with deciding which process to remove are part of the process, and one thing I have learnt is that it is rare for someone to suggest that they are the problem.

Removing process is especially difficult to justify in a client supplier relationship. Process is expensive because it requires a lot people to manage all the people that are managing all the process artefacts that manages the work that the remaining individuals are actually doing. But once you have set the precedent to the client that that approach is necessary, being able to later remove people and artefacts with no impact on performance is awkward and embarrassing – it also negatively impacts revenue – easier to just continue on with the client losing but being ignorant of doing so. Savvy clients will challenge the initial setup but there are many that are not, and it is in the best interest of the big consultancies to keep it that way.

Identifying the processes and tools you need should be something mutually agreed by the team – the whole team, and then continually reassessed. If the team has requested something then they are far more likely to abide by / use it. Externally pushing something on the team will be resented and resisted – which typically results in the introduction of someone to enforce it and you can see the bureaucracy mount… you will be owned by your process!

This is why we have retrospectives, to assess the approach and change it, where retros are not productive is when you have lots of externally enforced processes that can’t be changed. What is the point about discussing something you can’t change – if you can’t adapt why bother inspecting? Before long you are back in the world of a disempowered, disinterested team burdened by bureaucracy.

So, free yourselves, cast off your process and then look at all the little pieces on the floor. Now pick up the bare minimum and see how you get on.

Agile Manifesto – The Dark Side – Part 4 of 4

Finally reached the last of my 4 posts looking at the other side of Agile. The last line of the manifesto:

Responding to change over following a plan

This is slightly different to the other 3 in that it feels obvious, common sense, but so does the right hand side when considered in isolation – following a plan – well Duh yeah, who doesn’t, no plan no clue!

So making a case for having a plan is slightly easier, but then what is the problem, what do the agile community have to say about planning that warrants a direct attack in the manifesto.

When explaining Agile in training events or discussion groups I usually enhance this line with an extra word – makes things a little more colloquial but really stresses the point:

Responding to change over BLINDLY following a plan.

Now the issue is clearer. Having a good understanding of where you are, what is going on, the consequences of dependencies and whether we are likely to deliver what we thought we would when we thought we would, is a good thing. The problems occur in three places:

  • When the plan attempts to predict beyond your delivery horizon
  • When there is a lack of acceptance that the current position isn’t the planned one
  • When achievement to the plan is more important than delivering Value

I often joke about plans that if when you start a project you have a plan that says you’ll deliver in 18 months time, on the 14th of March for example, then can we pay upfront for a massive launch party on the 15th March….? Of course not – so why so much effort, scrutiny and confidence in something you don’t truly believe in?

The best plans enable change, those that don’t are either ignored, undergo endless expensive change control or drift off into fantasy in comparison to reality and then suffer catastrophic failure.

The ability to change a plan isn’t about changing the right answer, it is about avoiding the wrong one. Don Reinertsen explains this wonderfully with the following: If you were offered a chance to guess a three digit number for three dollars and if correct you win $3000, would you take it? Statistically you are break even. Now if I give the option to pay $1 for the first digit and then be told if correct before paying $1 for the second digit, and the same for the third. Which has the biggest payoff? The later costs the same to play, wins the same but is 96% better value, and it isn’t because the player is more likely to get the right answer, or because they pass less or win more. It is because they can identify a losing situation and can change their plans.

So within the wonderful world of planning, which elements should be encouraged? Engaging in a truly Agile delivery does bring some challenges to the business that are required to consume a constantly changing / enhanced technology landscape; the training material approach will need to adjust, communications need to be aligned for example. Having a high level plan of what features are in the pipeline will keep these departments together and then a low level detailed plan of what is being worked on NOW will enable the associated organisational change management activities to keep up.

A roadmap, of the type Pichler espouses for example:

http://www.romanpichler.com/tools/product-roadmap/

is a great plan in that it indicates direction and purpose but doesn’t give details that can instruct delivery teams blindly.

At a sprint level – when in a scaled environment, ensuring that after sprint planning all teams are aware of all dependencies is also an important activity – how this is done – central google doc, notecard for reference in Scrum of Scrums is contextual.

Eisenhower once said that “Plans are worthless, but planning is everything”. What he meant by this is that through planning you get a clear understanding of what is happening now, an awareness of all the moving parts and how they interact – and that is vital because it enables you to respond. The reasons behind why the plan being worthless can be answered with another great quote from Moltke the Elder (19th Century Prussian military strategist), “No plan survives first contact with the enemy”. There are too many variables at play to expect any long term plan to be expected to remain static.

In my experience the benefit of planning over following a plan is that it forces acceptance of the realities of the situation you are in, free from the illusion of where you’d like to be.

Agile Manifesto – the Dark Side – Part 3 of 4

It has been rather too long between this post and the last – consequence of client commitments, apologies.

Last post I looked at the second line of the Agile Manifesto, considering what is suitable documentation standards in Agile Delivery, this time I’ll look at the third line.

Customer collaboration over contract negotiation

So just how much contract negotiation do we need, and taking a step back what really is a contract?

I suggest that a contract is an agreement that specifies the details of the consequences for failure.

Agreements are essential for group delivery, it is what enables one individual or group to operate independently from another. There is usually some element of collective benefit, either both sides are working towards a common goal or there is a transaction attached.

Good agreements run on trust, each party has respect and trust in the other to deliver as expected, and therefore what is really at stake for failure is that trust, which is often the most important powerful thing we have. Agreeing to your spouse to be home for 19:00 is a good example. Within Agile delivery you can express Sprint commitments , WIP limits and acceptance criteria as agreements.

Agreement is based on trust and respect, a contract goes further than these honour based agreements, extending this to provide consequence to failing to comply with the agreement.

Firstly contract can mean different things to different people in different contexts. I currently find myself in a supplier /client type relationship where the contracts are legal documents and stipulate costs and disclaimers and liabilities and the like, at other times I have been working in house in which case the contracts are more intimate and are reflections of commitments between departments or even people.

There are many articles on commercial Agile contract structure / writing and I am not going to tackle that subject here – save it for another day. In summary I’d say keep it T&M and keep your contracts as short and easy to renew as possible.

It isn’t contracts that the Manifesto makes reference to, but the negotiation. Negotiation is an attempt to reach an agreement that is favourable to you. I’d argue that the best (more realistic) contracts are those that you’d be willing to sign up to either side of.

Agile delivery is all about safe to fail experiments, having the freedom and flexibility to try something and then take the learning, we wont be right all the time but the really important part is the consequence of failure is not catastrophic.

Excessive negotiation implies that you are looking to tip the balance of in your favour – it is less of a partnership now and the trust is being eroded. In a one sided contract either then likelihood or consequence of failure on one side is greater than the other – and probably known to be so. Risk is not equally shared. This is not limited to legal contracts, a senior manager in house can make it clear the consequences of failure to deliver a sprint in terms of performance appraisals, future projects or weekend working.

The premise of the manifesto statement is to put the interests of the project delivery ahead of those relating to an individual or a supplier; an unbalanced contract will encourage behaviours of self-protection, as opposed to collaborative, delivery facing behaviour. Teams will start to look to identify where they can cut quality without being noticed, or the scope details of feature items will be expanded – in addition to other nefarious activities.

The sad truth is that in both scenarios, everyone loses. Either the project fails or the client has over paid and is unlikely to offer repeat business.

So we value contract negotiation, in that it gives us a framework to deliver, it makes clear the agreements and a fair set of consequences, negotiation to a point where you’d no longer sign the other half of the contract… now then I believe it to be counter productive to delivery.

The ultimate contract is a partnership, and the ultimate consequence is loss of trust.

Agile Manifesto – the Dark Side – Part 2 of 4

In my last post I looked at the Processes and Tools side of the first line of the Agile Manifesto. This time I’d like to look at the second line – Working Software over Comprehensive Documentation.

In many projects I’ve been on just getting any focus on Working Software was a challenge, and by “Working” I mean actually delivered production capability, there is a reasonable argument to change the manifesto to “Working software over everything else” and leave it at that!

But assuming that we are in a situation where we are delivering production quality every sprint and at least on a fairly frequent basis (2-4 sprints) that work is actually being deployed to production, then we can turn to the documentation and ask, so how much is needed and why is there a perception that Documentation is bad?

Firstly I’d like to split all documentation into two camps:

  • Documentation which is an end in its own right
  • Documentation which is a means to an end

Practically speaking any document that you are going to need after the project is in the first camp, and everything else falls in the second. I’d like to make the general sweeping statement that the first camp is useful and should be perceived as project deliverables, the latter should be minimised. As an author of something, ask yourself whether anyone will ever read it after you go live – if not, then you are in the second camp.

Examples of documentation which is a means to an end would be user manuals, API service catalogues and training guides. These typically provide explanation as to how the application or its components function so that once the project development team have moved off others can understand how to use it, reuse parts of it or change it.

Some of these can be expressed in terms of a User story: As an application support user I need an explanation of the system architecture so I know where to investigate should an issue arise. The problem with expressing them like this is obselence, once you have delivered the story, and then the application changes – the document is obsolete. Typically these needs should be identified up front and then worked into the Definition of Done, so if you know there is a need to have a user manual – then updating the master document for each delivered User Story during the sprint will ensure that you always have correct documentation and don’t get into the horrible situation of having to balance the needs of delivering functions or enabling the users to understand the ones they already have.

Now most project documentation doesn’t fall into this group – most of this group would be considered optional (aspirational even) and often is outsourced to a separate group under Change management, desperately trying to catch up. Most project documentation I would argue is about gaining understanding of what we are doing now or how we have done – and really much of it is a waste of time. Here we have Requirements, roadmaps, “Strategies” of all types such as Test Strategy or Comms Strategy and progress reports. Each of these hold no value in their own right, they are only a means to an end, typically successful delivery of the project.

Approach these from the perspective of their purpose rather than their form, templates for example are there to make your life easier to ensure nothing important gets forgotten – more often though they are treated as a ruleset and create additional work. Create the document purely against it’s function, so a feature requirement is usually to enable the team to understand what is needed and have something referenceable for acceptance criteria during development – so really a few notes scribbled on the back of a credit card bill pinned to a wall may achieve that. A Strategy in this context is usually just a set of ways of working, that should be flexible anyway – so a few “agreed principles” posted to your wall should suffice.

There is a big difference in documentation between Waterfall and Agile and it relates to the level of collaboration between work definition and work delivery. Under a waterfall framework the people defining the work may do so far in advance of those delivering it, and may not ever meet – under these situations the “Purpose” of the document is similar but it also has to stand alone, unsupported by the day to day working relationship between the author and the developer, and as such needs to be a lot more thorough and structured. An Agile approach enables the level of detail and structure to be reduced to only what is necessary to get the developer to the end of the sprint when the author is sitting only a couple of metres away.

Also Waterfall derisks through total understanding of what we intend to do and how far we are through it, Agile derisks through delivery, so progress reports are only of interest for the current (small delivery) and the risk they look to manage is similarly much smaller.

I often hear that organisations require their user stories to be fully comprehensive, because those Stories are used as long standing artefacts for application support to reference, or for the training dept to use to create their guides. This is a very messy and inefficient way of operating for a few reasons. The User Stories are not easily referenced, they are usually organised against sprint not current application feature and it is common that no single user story defines any one page or process in the application due to the number of iterations that it has undergone. User stories are written to enable the team to be able to deliver them, they aren’t written with an abstracted Application Support user in mind. This results in over specified stories wasting team time and still leaves the “other” consumers with a headache in trying to find the information they need. It often occurs if the business processes around implementation haven’t been considered at the outset – so the project is probably still clinging to waterfall practices for implementation. It is better that the team just creates what is actually NEEDED as they go along – and keep their requirements for themselves and suitably lean.

In Summary:

  • Try to build useful long standing documents into your Definition of Done
  • Documents that are internal to delivery should be written for their purpose only and time spent polishing them is a waste. Remember, only the USE of the document has any value – intrinsically it is worthless.

Agile Manifesto – the Dark Side – Part 1 of 4

The Agile Manifesto has been around a while now, although looking at some Agile governed projects you’d think nobody had read it. However there are plenty of fallacies in the industry stemming from misapplication of the manifesto.

The Manifesto has statements that recommend focus on one side of four paradigms, however it does not state that the “other end” of the scale is not necessary or intrinsically bad; the term used is “ while there is value in X…” Too often that has been misinterpreted as “Agile doesn’t include X”.

I’d like to take each one in turn and discuss just how much of the Dark Side is appropriate. So starting at the top:

Individuals and interactions over processes and tools

Just how much process and tooling is suitable?

This is a very hot topic on the forums currently as Agile becomes increasingly mainstream and more companies want to get the benefits. Like an onion, the outer layer of terminology, governance and processes are the most visible, and hence can be the easiest to adopt, like wearing a costume. That typically leads to a focus on processes and tools over individuals and interactions and is the single most common failing in Agile projects. However – assuming you have avoided this pitfall, what processes and tools are sensible? (I am only going to refer to those that can be used instead of human interactions as per the manifesto, so Development tools / frameworks, most of which are very beneficial in the right context, are not discussed)

It is fairly well agreed that for effective Agile delivery then you are going to need some form of continuous deployment and that is only sustainable with some manner of supporting automated test suite. Beyond that I’d argue that all tooling should be brought in deliberately due to a defined need, rather than an assumed starting tool kit.

So do we need one of those shiny Agile tracking tools, like JIRA or Version One etc? Instead of assuming you do, try assuming you don’t and then establish why you do, and then specifically what elements of the tool you really need, rather than adopting it blindly wholesale. Distributed teams are usually one of the main drivers – also one of the biggest challenges to team interactions.

Scrum, and even Kanban, come with their own processes, do we need them? I’d argue that Scrum without underlying XP practices is likely to be a lot less successful than the other way round but the original concept in Scrum, for example, three roles, four meetings is a sensible starting point.

Scrum (and Kanban) is a framework, so customising it to your needs is expected. Many teams will also have extra meetings, backlog refinement is a good case, or their own way to run retrospectives, or organise sprint planning. These are not to be shot down in flames with the cry of “Processes are Evil”, because in the majority of cases these are ways of working that the team has consciously adopted to simplify their activities, they might have even made them up. There is a significant difference between an externally defined process applied to a team and an internally customised one adopted from within. Rules (and by extension processes) enable action without full understanding, and consequently stifle innovation. Encourage teams to regularly challenge their processes to ensure that they still support activities rather than define them. If you are going to have a process, then make sure the team owns it, and therefore is free to change it.

Agreeing on some visual representation of the current state of work is nearly always a necessary step – so a task board or a burn down, some manner of backlog etc. It is often helpful to agree some mutual standards if working in a multi team environment – but each team should be free to customise their ways of working independently to a degree.

Processes are often the consequence of a lack of relationships, as the personal relationships between people become strained then processes are often brought in to compensate. The relationships can be strained either through failing trust or often due to scale. The growth pain organisations feel is usually a result of the shift from relationship and trust based operations to one of processes. With good relationships within your team and with people around them, then the need for processes will be reduced, so to identify where process is necessary I’d approach from the other side and ask where and why the relationships are poor?

Unless the whole organisation is Agile minded and relatively small, you are going to need to apply processes at the interfaces into poor relationships. That could be some structured status report into a nameless faceless (at least from the perspective of a lowly team developer) steering committee, or maybe an agreed set of steps into an inflexible established deployment team.

A consequence of the association between relationships and processes is that those team members that are not fully embedded in teams usually have more processes around their work than those in teams. So Architects might have review sessions, UX might need their own schedule of work and SLAs to deliver against etc. Again these aren’t intrinsically bad, but need to be agreed as a compensation for the lack of relationship rather than an opportunity to reinforce it.

In summary:

  • Assume no processes and then adopt them as and when the team needs them, not by default
  • Own your processes, rather than have them applied by central command and control
  • A basic framework to show work and to harmonise group time is usually necessary
  • Accept that for teams short on skills or experience, some guideline rules are going to be beneficial, but encourage the teams to challenge them frequently as their skills and experience improve
  • Consider all the relationships inside and outside the team. Wherever there are issues – either due to building trust or maintain good communications then consider adding a process
  • Processes compensate for lack of relationship, only adopt to fill the gap and resist going further as you may reinforce the lack of relationship

Next up – just how much Documentation is sensible in Agile delivery….

Comments welcome

#philagiledesign

Agile needs to be Business Strategy not IT Strategy

In my last posting I wrote about my frustrations with large projects dressed up to look Agile but without delivering WORKING SOFTWARE, so just compiling risk.

It is very easy to point and throw stones, but what can we actually do about this – and why do we often see Scrum governance at the build phase working within a waterfall implementation phase? Why do we get Water-Scrum-fall?

A place to start would be to observe where this behaviour is most likely to manifest. From my own person experience – and I’d love to get comments from others with differing experience – I see this most frequently in large companies whose main business has IT as an enabler, not as a main product or sales channel. Or basically, IT is not central to business strategy.

So I would suggest to address the issue of failing Agile initiatives / poor adoption of Agile, is to delve deeper into that business strategy, to understand how Agile principles can be incorporated into business leadership.

A common theme at Agile conferences at the moment is corporate culture and examples of Agile principles being applied outside of software engineering, and both of these are tapping away for the same reason, successful Agile adoption requires organisational cultural change and adoption of the principles throughout all departments – not just IT.

I would suggest that the first step would be to express the corporate Target Operating Model in Agile terms – explain how their roadmap can be iteratively achieved considering all elements, people, process and technology. Instead of feature driven end state targets, we should switch to Goal Orientated roadmaps * showing incremental value not a fixation on solutions we can’t be confident in.

* see http://www.romanpichler.com/tools/product-roadmap/ for details

People and Process will need to come before technology in any cultural change. People that value small steps, empowered teams and are customer centric. Processes that enable a safe to fail environment – and don’t mandate stage gate delivery. These will be seismic changes to many organisations – and would require exec level support to take root.

But how is this going to happen:

I would suggest that the Agile movement stops preaching to the choir and starts talking to the real power breakers. Getting project managers, architects and developers together is reinforcing something which is widely supported, but where it is really needed is at the executive level – and there is far too little genuine understanding of the cultural (rather than process) side of Agile behaviours with that group. We should start inviting these people to our arena and tailing our content accordingly. Produce collateral and case studies for IT departments to submit to their senior leadership.

The other route, probably more realistic, would be to focus not on the existing knowledge leaders in the Agile movement, but to those that already have the networks and trust with the necessary audience – the top management consultancies. I am pleased to see that this message is starting to get through as Delotte, McKinseys and the Boston Consulting Group have all set up Digital agency wings, mostly relatively recently, but I can’t help thinking that slightly misses the point – it once again puts Agile Principles in an IT box.

Agile shouldn’t be IT strategy, it needs to be Business Strategy.

Big used to eat small, now, fast eats slow, and Agile is fast. If we can bring the partners of the top consultancies to this line of thinking – then with top down influence we might start seeing some changes.

Comments welcome

#philagiledesign

Scrum governance at the build phase – that is NOT Agile

Agile delivery is very much in vogue, referred to as the latest must-have fashion accessory for your IT department, but far too many organisations are treating just like that, as if it is something that can be bought and installed, like a new shiny server with flashing lights that everyone can look at and say “Oooooh”. Having seen it, the business then drifts away and leave the infrastructure dept to do their thing and we all go back to doing what we did before that moment of excitement.

If you have a small project then it is reasonably credible to have a crack team work with the business to deliver something quickly, try it out and then improve on it. The delivery is small, the budget is probably similarly small and the whole initiative flies pretty low on the governance radar. This is where Agile started and excels.

Now think about a larger project, a more significant technological improvement or something with significant business impact. Now before this gets to any of those techie code writers, this initiative will be thought about… Target operating models refined, strategies penned, there might be process flows, visions, objectives and benefit realisation plans created. This work is vital, most of it falls under the Management Consulting area and for most companies this is taking place outside of the IT arena and therefore away from the shiny new Agile toy.

The business – having done all their thinking then speaks to the IT department and explains the wonderful plan, “we’ve done all the strategy work, you build it and then we’ll implement it”.

“But we’re Agile” the IT department responds, “In Agile we work out the requirements as we go and”….

“Yes but we’ve done most of that already – we’re not going to ask you to define all the work up front because we now know you don’t do that any more with your new shiny Agile, but we go live at the end of the year so work towards that, now off you go”.

At this point there is either a big fight and the company rethinks and there is a fundamental culture change to take Agile principles into the heart of business strategy or, which is far more likely, you have IT setting up an ‘Agile’ project with pretty fixed deliverables and a defined implementation date a long way in the future.

Because the scope of the work is probably significant, and the deliverables reasonably well understood there is a temptation to start large, multiple teams, maybe straight into one of those much misunderstood scaled frameworks. Teams are working away delivering chunks of software under Scrum governance which are connected together in some ever growing test environment, desperately trying to get through enough functionality before the predefined implementation date.

Because this is so realistic for large organisations it is very easy to see this as normal and hence perfectly acceptable, we are lulled along the path comfortable in what is familiar.

THIS IS NOT AGILE. THIS IS WATERFALL

  • The requirements have been roughly designed up front
  • You are now building it against a deadline
  • Then you are going to perform some kind of holistic testing at the end (unit testing continually but end to end user journey / business process lifecycle testing and performance testing often at the end)
  • A pile of tests build up for the end, because with no users there is no urgency to fix them
  • Then you are going to implement it at the end– with associated Change Management

What in that is Agile? – well you have what looks like Scrum governance (or Kanban or similar) during that build phase. So there might be sprints and burndowns and backlogs, but take a step back. Go back to first principles:

DO YOU HAVE WORKING SOFTWARE? – NO.

This is what I refer to as AGILE AT THE BUILD PHASE.

This is a very dangerous place to be and the issue is one of risk. IT projects are de-risked in one of two ways.

  • Work out everything up front, consider all of the options and then don’t change anything
  • Only deliver a little bit, check it works in the real world and then improve it

‘Agile at the build phase’ skips the detailed requirements gathering but also has a traditional waterfall implementation big bang end so you don’t get the continual real work assurance of the iterative implementation either – basically you carry all the risk of the project from start to finish with little mitigation.

Not all risks become issues, maybe the software does work – hurrah – and you get away with it, but sometimes it doesn’t, some integration fails, some data migration is not as expected or the system does what it was designed to do but not what is really needed – and now you have a massive problem – missed dates, huge loss of face and the possibility of a complete write off. It happens, I have witnessed it, and if you haven’t just Google Agile failures and look to see the number of agile builds with few deployments.

Agile isn’t something you do, it isn’t a process and it isn’t limited to the IT department. Agile is a culture, a principle, a set of behaviours. It is something that an organisation IS.

For an organisation to embrace the benefits of Agile then it needs to manifest those principles inside the earliest discussions and right through an iterative delivery – not just leave it as a shiny toy for IT.

Credit to Dan North for the underlying theme of Principles over Process.

Comments welcome, #philagiledesign on twitter.

At the very least, if you start down an Agile delivery then at least change the release plan to deliver something early, and by deliver I mean into production to a real user, it is the wait to the end implementation plan that will kill you.