How company culture changes with product maturity

I recently attended a seminar with Kent Beck, a giant in the software delivery world, the father of eXtreme Programming. The topic was about the need to adapt our delivery approach based on where the organisation was within a growth pattern of, “Explore” to “Expand” and finally “Extract”. This progression was related to financial models: investment vs return, against risk.

3X diagramImage Credit to @antonyMarcano

The progression of the product connects to another standard of product theory, that of BCG’s Growth Share matrix which is a portfolio planning model used to divide up products on two axis, revenue generation and market share.

BCG Matrix

Credit @BCG

The progression from “Question Mark” to either “Star” or “Dog”, with the “Stars” finally becoming the “Cash Cows” (the dogs having been killed off along the way) is shown on the diagram. I believe the flow of “Question” to “Star” to “Cash Cow” is somewhat analogous to “Explore”, “Expand” and “Extract” and in both cases is a reflection of product lifecycle. What Kent Beck has explored is the consequence on organisational psychology in having products at the various phases.

In both models what is happening is a change of risk appetite, reflective of the changing balance between investment and return, and the consequence of failure.

I have merged the two ideas to make:

  • Exploring Questions:  Where we experience an unlikely very high return on low investment for a growing product with a small market share
  • Expanding Stars: Where we see a reasonable likelihood of a substantial return on significant investment, for a product that has a rapidly growing market share
  • Extracting Cows: (An unfortunate mental model) Where we see a very highly likely, small return on a large investment, for a product that is a stable market leader

Given this framework; Product development on Exploring Questions is typically undertaken in such small groups that the relationships between team members can be sustained without any process framework and the association between business and tech, between demand and supply, is so tight and immediate, that even most light touch Agile frameworks appear cumbersome. The best practices here are those that weed out the Dogs quickly, User Centred Design and Experience, fail fast experiments etc.

Most of the practices that we see in an Agile delivery I would describe as being suited to the Expanding Star world, rapid gains with constant feedback to ensure we on track. The key here is sustainable delivery, staying ahead of the pack. It is said that success is always a surprise, because if it wasn’t someone else would be doing it, and in this phase product development are trying to increase revenue and market share to ensure the idea, the surprise, remains theirs.

I see the Extracting Cow phase as markedly different, the risk profile and consequences of failure here are much more akin to traditional engineering disciplines. You can’t add decide to add a swimming pool to the 30th floor of a building after completing the first 28 storeys. Product delivery in this phase is much more likely to adopt frameworks and governance that avoid failure than aim for rapid gain, and we say hello again to our old friend Waterfall.

To use a different analogy, to see the changing behaviour against the consequence of failure consider the following epic conflicts:

  • In the novel, “Lord of the Rings” you have a plucky few to take on an uncountable near unstoppable foe, with a slim chance of success but a massive return of saving the world. They are gung-ho in their exploratory approach making it up as they go along. The consequence of inaction is fatal. They have everything to play for, and in the scope of middle earth, not a lot to lose.
  • The Roman Empire between 150BC and 50 AD, had one of the finest fighting forces on the planet and certainly superior to anything nearby. The gains from conquest were substantial and the likelihood of victory high which lead to an enormous expansion. They expanded because if they didn’t someone else would, they didn’t risk losing what they had, they risked losing the opportunity for more. They were systematic, organised and effective. Their only downfall was the sustainability of holding onto the new territory, grow too far, too fast and the cost to operate without sound investment can exceed the returns, and you fall.
  • In the 1983 film, “War Games”, a computer responsible for nuclear war simulation eventually accepts that the only viable course of action, is to take no action. The only winning move is not to play. The consequence of failure is too great.

For an Agile Coach it is important to understand how this product evolution impacts on our ability to work with organisations. How the mindset shaped by the product lifecycle stage transcends the product, and comes to dominates the parent organisation. Typically small fast companies grow on the back of a successful product and become large, risk adverse corporations. It is as if each company has one product and slowly changes their organisational psychology to match that single product’s evolution. Facebook changing its motto from, “Move fast and break things” to, “Move fast with stable infrastructure” is a clear example. If a company was structured along product lines then it would be reasonable to expect that the area owning an Expanding Cow would be slow and reserved, whilst an area associated to an Exploring Question to be fast and dynamic, but that isn’t the case.

What happens is that the organisation “matures” with its first product, loses that innovative capability and is trapped either trying to innovate within a highly constrained system or not innovate at all – innovation becomes a timid extension of a secure product. In an attempt to reduce risk, a mature company typically chooses to split out certain components to specialists, silos are created, technical component teams propagate, swathes of middle management appear to ensure that nothing goes wrong – and stagnation sets in.

perfection-is-stagnation-900x506

At this point an organisation in this state, paralysed and inert but still sustaining huge revenue, many consider an Agile transformation as a possible route out. Many coaches work in this environment – and most find culture the biggest barrier, most of the Agile practices are not suited for this risk adverse mindset, many Agile coaches have cut their teeth in Exploring Questions or Expanding Stars and are lost in the face of the corporate governance of a gargantuan Extracting Cow, the frustration has prompted so called “Scaling Frameworks” to appear which ape agile values while remaining compliant to the NOT SAFE TO FAIL, siloed environment.

But should the main focus of delivery even be happening there anyway? Back to BCG’s model; Cash Cows should be milked, extracting profits and investing as little as possible. The cash that is derived should be used to fund the next set of Questions and to propel those Stars. Are the mature organsiations doing this? Aren’t they all too often ploughing the cash from their Cows back into their Cows, costing a lot for low return? They hold their market share though economies of scale and the value return on effort spent, is often so low that they deliberately choose not to measure it, so as to not own up to their wastefulness.

I believe large organisations with mature products need to have a strategy that focuses on introducing exciting new products funded by revenue from their established ones. The investment should be high in the growth areas and cut to a minimum on those products that have reached the Extract Cow level. Whilst it may be prudent to have a risk adverse approach to the operation and maintenance of the Extracting Cow, that mindset, those organisational dynamics, MUST NOT be replicated in the other product development areas.

Extracting Cows die, eventually. There are plenty of case studies of huge products that have ceased to exist, some have taken their parent companies with them, some haven’t. The BCG work has shown that for companies to survive the death of an Extracting Cow they must innovate, diversify their product portfolio, use the revenue of their Extracting Cow to fund their next set of Exploring Questions, with a few being successful. Invest in tomorrow, rather than hope that today will never end. Kent Beck’s work helps us to understand that the organisational approach across the portfolio needs be variable. Through combining the two I believe it is not sufficient to diversify your portfolio, you must diversify your approach and governance, maybe even culture to match. For your Questions to become Stars you need an Explore based mindset and approach.

Organisations need to reinvent themselves, not just their product offering, in order to survive; the mindset and governance honed to operate and maintain a mature product is an impediment to discovering its replacement.

 

About me:

I am Phil Thompson, an Agile Consultant, have worked at many places in many industries and with many people, mostly in Europe, mostly in the UK, mostly in London. My opinions are my own, shaped and challenged by the people and companies I have been fortunate to work with over the past fifteen yrs.

You can reach me at @philagiledesign or LinkedIn

 

Advertisements

Agile is consuming itself

The biggest threats to wholesale agile adoption within our business society don’t come from a counter proposal, they come from within. The failings of previous approaches are well known and well documented and in fact have been since their inceptions, but everyone muddled through for lack of alternative. There isn’t going to be resurgence in support for the “Good old days”, too many people can prove it wasn’t that good. Nor do I imagine a new way, a utopian enlightenment to dawn upon us, from which point all programme delivery becomes risk and issue free, there just aren’t sufficient unexplored paradigms in our approach.

If the agile movement is to die, to collapse, it will do so inwards, on itself and from within. It will suffer the fate of Robespierre, the French revolutionary who rose to power through a fervent belief in equality and support for those that had been excluded and repressed under royal tyranny. His passion and success made him increasingly blind to the consequence of his unyielding beliefs and the presence of those that coveted his position. Eventually those that would usurp him turned the populace to revile the fanatical dogma that had wrought so much terror in the name of social progress, and he met the same end that he had brought about for the late king, a short drop from Madame Guillotine.

I suggest the dangers lie in three areas, the ignorant, the exploitative and the manipulative. In all cases the issue is misinterpretation of sound decent values, either innocently or more malevolently.

The first case is ignorance. This is a hard truth I had had to admit to myself, and I am reassured to read postings from other thought leaders I admire, who have humbled themselves in a similar fashion – see here, that has given me the confidence to come clean. Years ago I probably was this person, the well-intentioned but ignorant zealot; armed with too little understanding or experience of Agile Values and human politics, and too much theory and process definition.  I was that guy howling into the wilderness standing on Dunning-Kruger’s mount stupid. You’ve may relate to these kinds of transformation attempts; process and terminology centric backed by dogma and rhetoric that is applied through contextless retrospective coherence. Trying to change behaviours and practices through process, like trying to turn the quiet shy girl at the back of the class into the lead cheerleader by tossing her a costume and couple of pom-poms.

The second case is where the revenue generation consequence of those talented individuals working in organisations to support an agile transformation becomes a motivator for themselves and others. When the Agile philosophy becomes a commercial opportunity, then predictable but none too pleasant behaviours start to emerge. Pyramid style certification schemes, and an attempt to commoditise processes and supporting tooling for the purpose of revenue rather than stakeholder value. The worst excesses of this can be seen in those offerings that do little more than relabel existing familiar enterprise operations with new “Agiley” terminology with a supporting license fee. This undermines the Agile principles by dragging it down to something much closer to the status quo for the purpose of profit.

The last case is the most dangerous, those that speak in our name to further their own agendas. The butt of many a Dilbert joke – “Welcome to Agile – stop documenting anything and now you can work faster”. This is the wrecking ball of Agile, or more usually Scrum, wielded by paranoid power-hungry, non-technical managers who feel they now have a weapon to use against their intractable, awkward IT colleagues. Teams have been made to work longer, harder, with less control, fewer standards and more interference all in the name of Scrum. New developers have been born into this environment and are left believing that this is normal, and the more experienced developers resent the dumbing down on their industry and rage against the framework because they are powerless against their management. There are hundreds of comments on blog boards of people decrying Scrum through valid complaints about business practices that bear no resemblance to Scrum.

Now imagine all three together, well-intentioned but ignorant scrum masters being manipulated by untrusting and overly ambitious management to deliver the impossible, at the expense of the developer workforce, being cheered on by a process, tooling and certification industry laughing all the way to the bank. The end result will be a profitable industry of failing projects and people in a slightly different way to twenty years ago, and critically no real improvement in the enterprise project success rate.

So what is to be done? As a consultant working on Agile Transformation; are we like a few conservationists, trying to save what is left with the grim knowledge that it won’t be enough against the rampant consumerism, selfishness and apathy of humankind?

We have to continue, to give up would be dereliction of duty, and most of us have skin in the game ourselves now, we are part of the problem even as we try to point the finger elsewhere. Firstly we should point out misrepresentation of Agile wherever we see it. We need to stop preaching and learn a little humility, for those that teach Agile theory and concepts end each class with this statement – “you now know  a lot less than you think you do and are now capable of a lot more damage than you can imagine”. For those that are working in environments that are Agile in name only, then call this out, transformation to Agile may be beyond your means but at least stop calling it Agile so as to not further tarnish what was once a noble ideology. We need to focus on delivering value, on return for our clients not for ourselves. Be honest and ethical about the contracts we take and the companies we work for.

I like the proposition that someone attributed to McKinseys (I don’t know if correctly) that we should focus on delivering value to our clients rather than to ourselves and through that the money will flow anyway.

About me:

I am Phil Thompson, an Agile Consultant, have worked at many places in many industries and with many people, mostly in Europe, mostly in the UK, mostly in London. My opinions are my own, shaped and challenged by the people and companies I have been fortunate to work with over the past fifteen yrs.

You can reach me at @philagiledesign or LinkedIn

 

Use Service Design as a tool to challenge

Some would argue that Service Design has been around for ages, for those people that were designing and developing great products years ago, this was what they did, but didn’t take the opportunity to name it, package it and market it. Service Design, as an industry, could be dismissed as the latest reincarnation of common sense – however if it really was so simple and obvious then why weren’t we all doing it, oh how we titter at the common masses for their foolishness, uncomfortable in the knowledge we were just lucky.

Modern Service Design principles and practices are at their most effortless when there is a prevailing wind supporting those activities and their timeline, and there is a very clear vision that focuses on outcomes. Service Design is a structured approach to ensure that users are able to achieve what they need, from their initial desire to the final outcome. Within IT projects it starts with upstream investigations to ensure that what is delivered will fit neatly into the fuller user experience and then manifests more as a user-centric culture from that point on, constantly focusing on the differential between what the user has and what they need. It involves activities such as identifying the users, understanding why they want something, what they are currently doing and how they would naturally approach their need.

My experience with Service Design is less about creating great products but more about identifying and exposing poorly thought through projects. If you follow a service design approach it is very hard to accept a long list of requirements without confidence that they will deliver something appropriately sized in the users’ best interests.

It is more common in a supplier client relationship to feel the need to (and have the opportunity to) challenge the prescribed solution on a table than an in house build. We, the business, have decided we need this widget – please build the widget… This request now usually solicits a slow “Okaaaaaaaaaaaaaaaaay” from me.

The trap in front of you is to ask the obvious question, “tell me about this widget?”. The right question is, “tell me why this widget will solve your problem?” It could be that the response is a full well researched and documented study on user behaviours and needs, with a few supporting usability studies on prototypes which come neatly packaged with a user researcher to join your team. I say ,“It could be…” but really, that isn’t my experience. Careful questioning usually exposes weak assumptions and through pushing a Service Design strategy you can bring everyone to a common path avoiding too much conflict or loss of face.

Projects that proceed without a good foundation on Service Design (or common sense as it was called before it got a name) typically end in one of three situations:

  • Successful with substantial changes during delivery
  • Successful but over-engineered and expensive – and usually late
  • Abandonment

An immediate focus on the widget proposal on the table will typically take you down these paths, I’ve been there, don’t go there.

We need a strategy for this…

Been here before?

“I hear your issue and I think you are right, we need a strategy for this…” (feel free to roll your eyes at this point).

This is typically said in response to an expression of a problem – rather than a request for a strategy; and what is a strategy anyway?

 

Strategy should be a set of principles applied continuously, that support decision making to ensure alignment to your objective. It is not (or should not be) a fixed plan that implies excessive Big Design Up-Front. However I suggest our opening line isn’t referring to either of these strategies, no in this case it is much less. I will rephrase…

“I hear your point and I think you are right, I don’t know what to do. I don’t want to make a decision, because it might be wrong; but I don’t want you to think that I don’t know what to do, and I need you to remain thinking I am important.”

 

Basically the word strategy is over used and often thrown in as an opportunity to procrastinate without losing authority.

So how can we help prevent this response from being given (or even from giving it ourselves).

Firstly, take a stance that doesn’t suggest that solutions can be plucked out of thin air for a problem and then put through an expensive development process that won’t return against its own risk for months or years. Once the expectation is lifted from having to implement a solution with unknown consequences, then it is possible to retain authority whilst investigating, rather than acting.

Next understand the metrics by which the problem, and hence its resolution, can be measured. If you can’t define the metric, then you probably don’t really have your finger on the real problem.

Now suggest an idea that will affect the metric in the right direction and ask people involved what could we do to test whether the implementation of the idea will have the desired impact on the metric – a safe to fail experiment – mindful that most experiments DO fail.

Then do that test. This is a proactive decision to DO something, ACTUALLY DECIDE TO DO SOMETHING. Later assess the findings and then you can decide if the idea is worth progressing with. Now the expensive decision to invest in something is a lot less risky and the deep desire to procrastinate to avoid making a mistake is reduced.

 

Now you can call this User Research, Lean, MVP, Agile or whatever. I have avoided doing so because I don’t want to solicit an emotive response against poor implementations of these things that lead to organisations stating “We don’t do that here”. This is a call to those situations where enormous time and money is wasted with the word “Strategy” because it is an excuse to justify doing nothing hoping the problem will just evaporate!

Agile Manifesto – the Dark Side – Part 2 of 4

In my last post I looked at the Processes and Tools side of the first line of the Agile Manifesto. This time I’d like to look at the second line – Working Software over Comprehensive Documentation.

In many projects I’ve been on just getting any focus on Working Software was a challenge, and by “Working” I mean actually delivered production capability, there is a reasonable argument to change the manifesto to “Working software over everything else” and leave it at that!

But assuming that we are in a situation where we are delivering production quality every sprint and at least on a fairly frequent basis (2-4 sprints) that work is actually being deployed to production, then we can turn to the documentation and ask, so how much is needed and why is there a perception that Documentation is bad?

Firstly I’d like to split all documentation into two camps:

  • Documentation which is an end in its own right
  • Documentation which is a means to an end

Practically speaking any document that you are going to need after the project is in the first camp, and everything else falls in the second. I’d like to make the general sweeping statement that the first camp is useful and should be perceived as project deliverables, the latter should be minimised. As an author of something, ask yourself whether anyone will ever read it after you go live – if not, then you are in the second camp.

Examples of documentation which is a means to an end would be user manuals, API service catalogues and training guides. These typically provide explanation as to how the application or its components function so that once the project development team have moved off others can understand how to use it, reuse parts of it or change it.

Some of these can be expressed in terms of a User story: As an application support user I need an explanation of the system architecture so I know where to investigate should an issue arise. The problem with expressing them like this is obselence, once you have delivered the story, and then the application changes – the document is obsolete. Typically these needs should be identified up front and then worked into the Definition of Done, so if you know there is a need to have a user manual – then updating the master document for each delivered User Story during the sprint will ensure that you always have correct documentation and don’t get into the horrible situation of having to balance the needs of delivering functions or enabling the users to understand the ones they already have.

Now most project documentation doesn’t fall into this group – most of this group would be considered optional (aspirational even) and often is outsourced to a separate group under Change management, desperately trying to catch up. Most project documentation I would argue is about gaining understanding of what we are doing now or how we have done – and really much of it is a waste of time. Here we have Requirements, roadmaps, “Strategies” of all types such as Test Strategy or Comms Strategy and progress reports. Each of these hold no value in their own right, they are only a means to an end, typically successful delivery of the project.

Approach these from the perspective of their purpose rather than their form, templates for example are there to make your life easier to ensure nothing important gets forgotten – more often though they are treated as a ruleset and create additional work. Create the document purely against it’s function, so a feature requirement is usually to enable the team to understand what is needed and have something referenceable for acceptance criteria during development – so really a few notes scribbled on the back of a credit card bill pinned to a wall may achieve that. A Strategy in this context is usually just a set of ways of working, that should be flexible anyway – so a few “agreed principles” posted to your wall should suffice.

There is a big difference in documentation between Waterfall and Agile and it relates to the level of collaboration between work definition and work delivery. Under a waterfall framework the people defining the work may do so far in advance of those delivering it, and may not ever meet – under these situations the “Purpose” of the document is similar but it also has to stand alone, unsupported by the day to day working relationship between the author and the developer, and as such needs to be a lot more thorough and structured. An Agile approach enables the level of detail and structure to be reduced to only what is necessary to get the developer to the end of the sprint when the author is sitting only a couple of metres away.

Also Waterfall derisks through total understanding of what we intend to do and how far we are through it, Agile derisks through delivery, so progress reports are only of interest for the current (small delivery) and the risk they look to manage is similarly much smaller.

I often hear that organisations require their user stories to be fully comprehensive, because those Stories are used as long standing artefacts for application support to reference, or for the training dept to use to create their guides. This is a very messy and inefficient way of operating for a few reasons. The User Stories are not easily referenced, they are usually organised against sprint not current application feature and it is common that no single user story defines any one page or process in the application due to the number of iterations that it has undergone. User stories are written to enable the team to be able to deliver them, they aren’t written with an abstracted Application Support user in mind. This results in over specified stories wasting team time and still leaves the “other” consumers with a headache in trying to find the information they need. It often occurs if the business processes around implementation haven’t been considered at the outset – so the project is probably still clinging to waterfall practices for implementation. It is better that the team just creates what is actually NEEDED as they go along – and keep their requirements for themselves and suitably lean.

In Summary:

  • Try to build useful long standing documents into your Definition of Done
  • Documents that are internal to delivery should be written for their purpose only and time spent polishing them is a waste. Remember, only the USE of the document has any value – intrinsically it is worthless.

Agile Manifesto – the Dark Side – Part 1 of 4

The Agile Manifesto has been around a while now, although looking at some Agile governed projects you’d think nobody had read it. However there are plenty of fallacies in the industry stemming from misapplication of the manifesto.

The Manifesto has statements that recommend focus on one side of four paradigms, however it does not state that the “other end” of the scale is not necessary or intrinsically bad; the term used is “ while there is value in X…” Too often that has been misinterpreted as “Agile doesn’t include X”.

I’d like to take each one in turn and discuss just how much of the Dark Side is appropriate. So starting at the top:

Individuals and interactions over processes and tools

Just how much process and tooling is suitable?

This is a very hot topic on the forums currently as Agile becomes increasingly mainstream and more companies want to get the benefits. Like an onion, the outer layer of terminology, governance and processes are the most visible, and hence can be the easiest to adopt, like wearing a costume. That typically leads to a focus on processes and tools over individuals and interactions and is the single most common failing in Agile projects. However – assuming you have avoided this pitfall, what processes and tools are sensible? (I am only going to refer to those that can be used instead of human interactions as per the manifesto, so Development tools / frameworks, most of which are very beneficial in the right context, are not discussed)

It is fairly well agreed that for effective Agile delivery then you are going to need some form of continuous deployment and that is only sustainable with some manner of supporting automated test suite. Beyond that I’d argue that all tooling should be brought in deliberately due to a defined need, rather than an assumed starting tool kit.

So do we need one of those shiny Agile tracking tools, like JIRA or Version One etc? Instead of assuming you do, try assuming you don’t and then establish why you do, and then specifically what elements of the tool you really need, rather than adopting it blindly wholesale. Distributed teams are usually one of the main drivers – also one of the biggest challenges to team interactions.

Scrum, and even Kanban, come with their own processes, do we need them? I’d argue that Scrum without underlying XP practices is likely to be a lot less successful than the other way round but the original concept in Scrum, for example, three roles, four meetings is a sensible starting point.

Scrum (and Kanban) is a framework, so customising it to your needs is expected. Many teams will also have extra meetings, backlog refinement is a good case, or their own way to run retrospectives, or organise sprint planning. These are not to be shot down in flames with the cry of “Processes are Evil”, because in the majority of cases these are ways of working that the team has consciously adopted to simplify their activities, they might have even made them up. There is a significant difference between an externally defined process applied to a team and an internally customised one adopted from within. Rules (and by extension processes) enable action without full understanding, and consequently stifle innovation. Encourage teams to regularly challenge their processes to ensure that they still support activities rather than define them. If you are going to have a process, then make sure the team owns it, and therefore is free to change it.

Agreeing on some visual representation of the current state of work is nearly always a necessary step – so a task board or a burn down, some manner of backlog etc. It is often helpful to agree some mutual standards if working in a multi team environment – but each team should be free to customise their ways of working independently to a degree.

Processes are often the consequence of a lack of relationships, as the personal relationships between people become strained then processes are often brought in to compensate. The relationships can be strained either through failing trust or often due to scale. The growth pain organisations feel is usually a result of the shift from relationship and trust based operations to one of processes. With good relationships within your team and with people around them, then the need for processes will be reduced, so to identify where process is necessary I’d approach from the other side and ask where and why the relationships are poor?

Unless the whole organisation is Agile minded and relatively small, you are going to need to apply processes at the interfaces into poor relationships. That could be some structured status report into a nameless faceless (at least from the perspective of a lowly team developer) steering committee, or maybe an agreed set of steps into an inflexible established deployment team.

A consequence of the association between relationships and processes is that those team members that are not fully embedded in teams usually have more processes around their work than those in teams. So Architects might have review sessions, UX might need their own schedule of work and SLAs to deliver against etc. Again these aren’t intrinsically bad, but need to be agreed as a compensation for the lack of relationship rather than an opportunity to reinforce it.

In summary:

  • Assume no processes and then adopt them as and when the team needs them, not by default
  • Own your processes, rather than have them applied by central command and control
  • A basic framework to show work and to harmonise group time is usually necessary
  • Accept that for teams short on skills or experience, some guideline rules are going to be beneficial, but encourage the teams to challenge them frequently as their skills and experience improve
  • Consider all the relationships inside and outside the team. Wherever there are issues – either due to building trust or maintain good communications then consider adding a process
  • Processes compensate for lack of relationship, only adopt to fill the gap and resist going further as you may reinforce the lack of relationship

Next up – just how much Documentation is sensible in Agile delivery….

Comments welcome

#philagiledesign

Agile needs to be Business Strategy not IT Strategy

In my last posting I wrote about my frustrations with large projects dressed up to look Agile but without delivering WORKING SOFTWARE, so just compiling risk.

It is very easy to point and throw stones, but what can we actually do about this – and why do we often see Scrum governance at the build phase working within a waterfall implementation phase? Why do we get Water-Scrum-fall?

A place to start would be to observe where this behaviour is most likely to manifest. From my own person experience – and I’d love to get comments from others with differing experience – I see this most frequently in large companies whose main business has IT as an enabler, not as a main product or sales channel. Or basically, IT is not central to business strategy.

So I would suggest to address the issue of failing Agile initiatives / poor adoption of Agile, is to delve deeper into that business strategy, to understand how Agile principles can be incorporated into business leadership.

A common theme at Agile conferences at the moment is corporate culture and examples of Agile principles being applied outside of software engineering, and both of these are tapping away for the same reason, successful Agile adoption requires organisational cultural change and adoption of the principles throughout all departments – not just IT.

I would suggest that the first step would be to express the corporate Target Operating Model in Agile terms – explain how their roadmap can be iteratively achieved considering all elements, people, process and technology. Instead of feature driven end state targets, we should switch to Goal Orientated roadmaps * showing incremental value not a fixation on solutions we can’t be confident in.

* see http://www.romanpichler.com/tools/product-roadmap/ for details

People and Process will need to come before technology in any cultural change. People that value small steps, empowered teams and are customer centric. Processes that enable a safe to fail environment – and don’t mandate stage gate delivery. These will be seismic changes to many organisations – and would require exec level support to take root.

But how is this going to happen:

I would suggest that the Agile movement stops preaching to the choir and starts talking to the real power breakers. Getting project managers, architects and developers together is reinforcing something which is widely supported, but where it is really needed is at the executive level – and there is far too little genuine understanding of the cultural (rather than process) side of Agile behaviours with that group. We should start inviting these people to our arena and tailing our content accordingly. Produce collateral and case studies for IT departments to submit to their senior leadership.

The other route, probably more realistic, would be to focus not on the existing knowledge leaders in the Agile movement, but to those that already have the networks and trust with the necessary audience – the top management consultancies. I am pleased to see that this message is starting to get through as Delotte, McKinseys and the Boston Consulting Group have all set up Digital agency wings, mostly relatively recently, but I can’t help thinking that slightly misses the point – it once again puts Agile Principles in an IT box.

Agile shouldn’t be IT strategy, it needs to be Business Strategy.

Big used to eat small, now, fast eats slow, and Agile is fast. If we can bring the partners of the top consultancies to this line of thinking – then with top down influence we might start seeing some changes.

Comments welcome

#philagiledesign

Scrum governance at the build phase – that is NOT Agile

Agile delivery is very much in vogue, referred to as the latest must-have fashion accessory for your IT department, but far too many organisations are treating just like that, as if it is something that can be bought and installed, like a new shiny server with flashing lights that everyone can look at and say “Oooooh”. Having seen it, the business then drifts away and leave the infrastructure dept to do their thing and we all go back to doing what we did before that moment of excitement.

If you have a small project then it is reasonably credible to have a crack team work with the business to deliver something quickly, try it out and then improve on it. The delivery is small, the budget is probably similarly small and the whole initiative flies pretty low on the governance radar. This is where Agile started and excels.

Now think about a larger project, a more significant technological improvement or something with significant business impact. Now before this gets to any of those techie code writers, this initiative will be thought about… Target operating models refined, strategies penned, there might be process flows, visions, objectives and benefit realisation plans created. This work is vital, most of it falls under the Management Consulting area and for most companies this is taking place outside of the IT arena and therefore away from the shiny new Agile toy.

The business – having done all their thinking then speaks to the IT department and explains the wonderful plan, “we’ve done all the strategy work, you build it and then we’ll implement it”.

“But we’re Agile” the IT department responds, “In Agile we work out the requirements as we go and”….

“Yes but we’ve done most of that already – we’re not going to ask you to define all the work up front because we now know you don’t do that any more with your new shiny Agile, but we go live at the end of the year so work towards that, now off you go”.

At this point there is either a big fight and the company rethinks and there is a fundamental culture change to take Agile principles into the heart of business strategy or, which is far more likely, you have IT setting up an ‘Agile’ project with pretty fixed deliverables and a defined implementation date a long way in the future.

Because the scope of the work is probably significant, and the deliverables reasonably well understood there is a temptation to start large, multiple teams, maybe straight into one of those much misunderstood scaled frameworks. Teams are working away delivering chunks of software under Scrum governance which are connected together in some ever growing test environment, desperately trying to get through enough functionality before the predefined implementation date.

Because this is so realistic for large organisations it is very easy to see this as normal and hence perfectly acceptable, we are lulled along the path comfortable in what is familiar.

THIS IS NOT AGILE. THIS IS WATERFALL

  • The requirements have been roughly designed up front
  • You are now building it against a deadline
  • Then you are going to perform some kind of holistic testing at the end (unit testing continually but end to end user journey / business process lifecycle testing and performance testing often at the end)
  • A pile of tests build up for the end, because with no users there is no urgency to fix them
  • Then you are going to implement it at the end– with associated Change Management

What in that is Agile? – well you have what looks like Scrum governance (or Kanban or similar) during that build phase. So there might be sprints and burndowns and backlogs, but take a step back. Go back to first principles:

DO YOU HAVE WORKING SOFTWARE? – NO.

This is what I refer to as AGILE AT THE BUILD PHASE.

This is a very dangerous place to be and the issue is one of risk. IT projects are de-risked in one of two ways.

  • Work out everything up front, consider all of the options and then don’t change anything
  • Only deliver a little bit, check it works in the real world and then improve it

‘Agile at the build phase’ skips the detailed requirements gathering but also has a traditional waterfall implementation big bang end so you don’t get the continual real work assurance of the iterative implementation either – basically you carry all the risk of the project from start to finish with little mitigation.

Not all risks become issues, maybe the software does work – hurrah – and you get away with it, but sometimes it doesn’t, some integration fails, some data migration is not as expected or the system does what it was designed to do but not what is really needed – and now you have a massive problem – missed dates, huge loss of face and the possibility of a complete write off. It happens, I have witnessed it, and if you haven’t just Google Agile failures and look to see the number of agile builds with few deployments.

Agile isn’t something you do, it isn’t a process and it isn’t limited to the IT department. Agile is a culture, a principle, a set of behaviours. It is something that an organisation IS.

For an organisation to embrace the benefits of Agile then it needs to manifest those principles inside the earliest discussions and right through an iterative delivery – not just leave it as a shiny toy for IT.

Credit to Dan North for the underlying theme of Principles over Process.

Comments welcome, #philagiledesign on twitter.

At the very least, if you start down an Agile delivery then at least change the release plan to deliver something early, and by deliver I mean into production to a real user, it is the wait to the end implementation plan that will kill you.

For today’s work we need Servant Leaders

As a manager, when things go wrong it is natural to want to know what went wrong, where the process broke down, and who didn’t do what was necessary. This often leads to the worst of 20th century style management: Blamestorming; Senior Management being able to point the finger at an agreed culprit (rarely a collective agreement). A more enlightened working environment may look for underlying failures of process or people without attributing conscious responsibility; for example, the training wasn’t good enough. In both examples the motivation is the same, to compartmentalise the issue into a single problem that can be either improved, removed or replaced, and where that problem is not ME.

I would suggest that if something has gone wrong, and you were in any way involved in the delivery, then you are culpable. The phrase goes “if you aren’t part of the solution, then you are part of the problem” – now turn that around. If there is a problem that you are aware of, and the problem still exists, then the solution hasn’t materialised, therefore you are currently part of the problem.

Before the 20th century, management was about the coordination of large numbers of individual resources applied to simple tasks (mining coal, spinning wool etc.) Early 20th century processes and their associated management were focused on increasing efficiency in the execution of defined tasks. More machines, more processes, and strong command aimed to ensure that algorithms were so effective for simple tasks that they could be extrapolated to tackle more convoluted and complicated tasks with multiple steps. However the premise was the same, merely on a larger scale. Control the individual’s actions within a well defined complicated set of tasks. The individual’s role in the system is to follow defined logic. The management approach was hierarchical; command and control, with the belief that the work that is being undertaken can be fully defined and understood.

However, this all breaks down when the issue to be addressed can no longer be expressed as a set of predefined tasks and a degree of independent thought is required. Towards the end of the 20th century in developed societies, there was a shift, and employees were increasingly required for their ability to think rather than just their ability to do.

21st century work is the analysis of complex problems – The identification and execution of small changes / deliverables that progressively reduce a problem to an acceptable size. At the start, there is a choice of which change(s) to make and how they could be made, so a clear set of predefined processes and tasks is not effective. The most effective approach to this type of issue is to work in small teams. The team will approach the problem through collaborative multifaceted reasoning (enhanced by experience) and will rapidly propose progressive actions to minimise the problem, each solution requiring empirical validation. Or in simpler terms, they’ll come up with ideas and they won’t know whether they will work until they try. In this environment traditional management will be counter-productive. Steps to structure, predefine and compartmentalise the work will reduce the opportunity to collaborate, thereby reducing the potential of the team over a group of individuals.

The role of Management’s in our current world is to be the Servant Leader. The person who provides the best environment for others to be the best they can be. The Servant Leader removes problems and provides support. Support could be in the form of guidelines to bring consistency, in the form of training and coaching, or in the form of emotional support to help build the best relationships and forge the best teams.

To provide support, you need to build trust and respect. Trust is difficult to gain, and easy to lose.

So next time, when things are going awry, consider whether you have enabled everyone around you to be as good as they can be. Do you truly support them, and is there mutual trust and respect? If not then I think that is where the blamestorming should start.