Compromised Agility
The authors of the Manifesto for Agile Software Development expound a set of values and principles that define “Agile Software Development”. These values and principles, written in 2001, became the focal point of a revolution in how software developers work.
In the last several years, that revolution has spread beyond software development to encompass other aspects of technology, and beyond technology into operations, management, engineering, business development, sales, marketing and even outside of for-profit organizations into education, health, government, community and charitable organizations. Originally written in the context of software, we can easily generalize the values and principles to other types of work. For example, the Manifesto refers to valuing “working software over comprehensive documentation.” We can easily generalize this to a more abstract statement that we value “results over bureaucracy.” The other values and principles can be similarly abstracted.
As the revolution has spread, unfortunately, the values and principles have also become compromised or selectively applied. Perhaps most obvious is how Agile Lifecycle Management tools such as Jira have often been substituted in place of the first value of the Manifesto: “we have come to value individuals and interactions over processes and tools” [emphasis added]. The irony of this substitution seems to be lost on those selling and buying these tools. Many other compromises or selective applications are common, and they are usually unique to each organization’s circumstances.
This evolution of the application of the Manifesto’s values and principles can be understood negatively or positively. I’m an optimist; yet we need to look at the negative, critical side before we can see the positive.
Cargo Cult Agility / No True Agilist
There are two common ways to understand the failure of organizations to do “true Agile”. Both are negative in the sense that they are criticisms without a reasonable solution to help an organization out of the situation into a better situation. As a consultant seeing many organizations, as a trainer hearing about many organizations, and as an active member of the global community of Agilists, I hear these criticisms quite regularly.
The first approach to understanding this partial agility is by comparison to the cargo cult mentality:
A cargo cult is a millenarian movement first described in Melanesia which encompasses a range of practices and occurs in the wake of contact with more technologically advanced societies. The name derives from the belief which began among Melanesians in the late 19th and early 20th century that various ritualistic acts such as the building of an airplane runway will result in the appearance of material wealth, particularly highly desirable Western goods (i.e., “cargo”), via Western airplanes.
In this approach to understanding an organization’s failure to embrace true Agile, the critic asserts that the leaders and employees of the organization do not really understand true Agile and are only practicing the obvious ritualistic aspects such as daily stand-ups (from Scrum), note cards on a wall (from Kanban), or user stories (from Extreme Programming). This criticism has some merit: many people are told to “do Agile” without proper training and coaching to understand the theory or the contextual applicability of various practices. The critic continues to compare this approach to Agile as a belief in magic: that the benefits of Agile can be gained through an application of the rituals without an understanding of, and more importantly adoption of the values and principles.
The “cargo cult” criticism does not offer a solution. When asked, the critics themselves will say, effectively, “well, you just need to really understand it!” This criticism also suffers from the inherent notion of the superior position of the critic: “I understand it… you don’t.” Not particularly helpful, especially for the staff in an organization who depend on executives and other leaders to support true Agile. And, not particularly helpful for the executives who often do not have the skill to support such a deep change.
The second common approach to understanding the failure of organizations to achieve real agility is the “No True Scotsman” comparison. This is a bit more challenging to describe because it is actually a criticism of other critics. Wikipedia describes No True Scotsman this way:
No true Scotsman or appeal to purity is an informal fallacy in which one attempts to protect a universal generalization from counterexamples by changing the definition in an ad hoc fashion to exclude the counterexample. Rather than denying the counterexample or rejecting the original claim, this fallacy modifies the subject of the assertion to exclude the specific case or others like it by rhetoric, without reference to any specific objective rule (“no true Scotsman would do such a thing”; i.e., those who perform that action are not part of our group and thus criticism of that action is not criticism of the group).
The starting point of this criticism is actually the reciprocal of the cargo cult criticism: the critic conflates true agility with the compromise happening at an organization and then accuses agility of being a failure. This criticism is often brought up in the following style of discussion:
Person A: Agile sucks because Big Corp is trying Scrum but it is really just an excuse for executives to micro-manage every little bit of work.
Person B: But Scrum and Agile are against micro-management! Big Corp isn’t really doing true Agile.[NOTE: this is the cargo cult criticism in brief.]
Person A: That’s just an excuse. You’re using the no true Scotsman argument and it’s a logical fallacy. Agile is what people are actually doing, therefore Agile sucks.
Person B: !
Interestingly, this argument style is often used between competing brands of agility. Leaders of both the Scrum and Kanban approaches to agility are well known for this approach to argument, particularly about each other’s chosen approach: “method X doesn’t work, as clearly seen by all the organizations doing X badly, therefore you should try my method Y which does work.” Again, this is ironic given the first value of the Manifesto for Agile Software Development “…over processes….”
Like the cargo cult criticism, the no true Agilist criticism does not offer a solution, other than reverting to a non-Agile approach to work (or more rarely, another approach that suffers the same imperfect implementation in organizations). And, like the cargo cult criticism, there is some truth in the no true Agilist view: legitimately, many organizations are doing a very very poor job of applying the values and principles (and associated practices). The conclusion that Agile sucks and therefore we shouldn’t even be trying is forgivable. However, the people online proclaiming this problem tend to be loud in their conclusion: let’s go back to our halcyon bygone days where the wind was fresh, the sun was bright, and we were all bucolically happy with our defined roles, our rigid processes, and our tools we could blame for every failure of our efforts (ironic hyperbole intended).
Both criticisms, reciprocal as they are, leave a gap. They don’t offer a full explanation of what is happening, nor do they offer a positive path to improvement. For that, we need to change our perspective just a bit.
Knowledge Generation and the Benefit of Time
I have been working with Agile methods for over 20 years. As a programmer and then enterprise architect, I adopted agile methods relatively early – even before the term “Agile” was applied to software development and these methods were referred to as “lightweight” methods. I mention this not to tout my expertise or even my experience, but rather my perspective. I’ve “been around” enough to know with a fair degree of certainty the following important points:
- There is no large organization that has successfully “transformed” from a non-Agile state to an enterprise-wide Agile state. By “large”, I mean at least 5000 total employees. By “transformed” I mean that it grew up using traditional techniques and has fully switched over to Agile management techniques throughout the total employee population. And by “successfully” I mean that it has sustained the use of Agile management techniques at the organizational level through at least one boom/bust economic cycle, and one change of C-Level leadership. (If you are reading this, and you know of such an organization, I would be excited to hear about it!)
- Large organizations that have high levels of agility typically started as small organizations with high levels of agility. Google, Amazon and Facebook come to mind. Their level of agility includes a high level of technical agility in addition to their management agility. I’m not sure, but I suspect that these two types of agility go hand-in-hand; certainly the Manifesto for Agile Software Development suggests they do go together.
- The only proof of transformation is in the sustainability of that transformation through existential crisis.
- Partial or compromised agility is the only kind of agility that is, so far, successfully sustained in a few rare cases of large organizations. Capital One is an example of this. They have been trying to adopt Agile methods and approaches since 2001. They have tried many techniques, and had many ups and downs in their journey. In 2017 at the Lean Kanban North America conference, two senior Agile coaches from Capital One asserted that they still have a long way to go for full enterprise agility – after working at it for 17 years already.
How does this compare to other management philosophies? The history of the Toyota Production System and Lean manufacturing, at least 50 years in the making so far, has shown us that revolutionary changes in the way work is done, can take many decades to become normalized. In the 1980’s and 90’s many organizations adopted “lean” as a management fad. We saw a swift rise and then fall in the popularity of lean. But the core principles and ideas of lean survived, and have continued to spread throughout many industries. Now, many organizations are “culturally” lean: they won’t revert to other methods of working even in crisis situations… and many others are not yet there.
With a timeline of 50+ years, perhaps we can consider our efforts to transform organizations to a greater level of agility in a more positive light. Human society at large is learning about agility though many many experiments run in thousands of organizations. Sometimes these experiments are motivated by wise and considered thought, but often, they are motivated by the faddish popularity of Agile methods such as Scrum, SAFe, and Kanban. Regardless of the motivation, the compromises organizations are making as they attempt greater levels of agility are part of a larger process encompassing all of human society in which knowledge generation is the primary outcome.
There is still one more problem to address: that individuals and organizations continue to try to improve agility even when they have experienced it done badly or seen it fail to take hold. One of my favourite authors and a prominent figure in the Agile community is Ron Jeffries. In the last couple of years, he has started to call out “Dark Scrum” as a blight in organizations that is causing suffering. I believe there is also “dark kanban”, “dark SAFe”, “dark extreme programming”, etc. These dark implementations of the various paths to agility, aren’t actually paths to agility for the people, teams and organizations implementing them. So again, why do people keep coming back to these methods?
Motivation to Try Again and Beauty
Let’s return to the Manifesto for Agile Software Development and here quote the four values. If you have read them before, even many times, I encourage you to read them again, slowly, and savour them:
We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
That is, while there is value in the items on the right, we value the items on the left more.
I would like to assert that anyone who has read these values, and further, read the principles behind the manifesto has been attracted to the inherent beauty of these ideas. This beauty is the source of the motivation to try again. This beauty is the source of the influence of the manifesto. This beauty is the reason why so many want to “own” it through the creation of their own brands, methods, schemes, and promotions. And, this beauty, while it is constantly struggling against corrupting influences, is powerful enough to inspire people to come back to it even when they have seen the dark side of agility.
Real agility is a culture founded on the beauty that inspires these principles. It’s not any particular method, it’s not a formula, it’s not merely a set of laudable or effective practices. Rather, this beauty is inspired by the Spirit of the Age in which we live. As individuals, teams and organizations, we will rarely live up to that beauty, rather we will experience it in moments, greater or lesser, strung together by our efforts to increase humanity’s collective knowledge. And as that knowledge grows, our patience for the partial, incomplete agility of many organizations will also grow for it is the source of our new-found knowledge.
[This article was originally published on Agile Advice on 27-Jun-2018]
If you find this useful, please consider contributing with our
“Value for Value” model.
Berteig Consulting
Empower Your Entire Organization with BERTEIG Consulting in Agile, Scrum, Kanban, SAFe and LEAN.
