Ron Westrum described three types of organizational culture: pathological, bureaucratic, and generative. The names are unfortunately clinical, but I summarize them as follows:
Pathological culture: Change is fatal.
Bureaucratic culture: Change is futile.
Generative culture: Change is fertile.
DevOps and Digital Transformation demand new ways of operating, and your organizational culture will determine your success. If your organization feels that change will be fatal, your transformation will require intensive expert help. If the feeling is that change will be futile, your transformation will be merely difficult (and expert help is still warranted). And if the pervasive feeling is that change is ground for excitement and innovation, then you have already slain the main bugaboo of change.
Your organization delivers software, so it should be obvious that improving your software delivery means improving your organization. But how to improve your software delivery organization? I’m going to tell you how, and you’ll be amazed at its simplicity: To improve an organization’s performance you need to improve all the components of the organization. And, you need to know how to improve each component.
An organization is made of…
An organization is composed of people, process, and artifacts.
An organization is made of three types of ingredients: people, process, and artifacts.
People
People includes you and your staff, the relationships among them and between them and others, and the culture that shapes these relationships. When we talk about “improving people,” we mean developing peoples’ skills and shaping the organizational culture.
Process
Process includes the path that work follows through the organization as value is added at each step along the way to the customer. Improving process focuses on optimizing this flow of value to the customer.
Artifacts
Artifacts are the inputs that flow into the processes, the outputs that result from them, and the tools and technologies utilized in processing.
A model for improvement
The Lean methodology for improvement: build, measure, learn, repeat.
The Lean Startup movement inspired a very effective model for rapidly improving in uncertain and risky business situations. By rapidly iterating through successive Build-Measure-Learn cycles, you systematically reduce risk and uncertainty. Each iteration through the cycle examines and eliminates the greatest remaining risk as the organization learns how to reduce its shape, size, and probability. Over time, the business carries less risk and delivers better products and services, with increasing precision (to the right customers), reliability (service quality), and control.
Improving the components of your organization
By applying the Lean model to each component of the organization, we can arrive at a prescription for improving all of them.
Build people
By “build people” we mean improving peoples’ skills and the overall culture. Several methods are effective:
Model new behavior. Leaders adopt, demonstrate, and reward the new behavior. This is the quickest way to change culture.
Exercise the new skills. This is the most effective way to acquire new skills.
Train.
Provide a mentor.
Measure people
The extent to which a person’s skills have delivered the expected results can be measured by:
Review by others.
Test.
Learn what to improve next in people
By “learn” we mean identifying the next area that needs improvement. Two ways are effective:
Conduct retrospectives and blameless postmortems.
Analyze the results of the reviews and the testing (performed in the “Measure” step).
Build process
To build—that is, to improve—process you mercilessly simplify it. For example:
Reduce redundancy, handoffs, and re-work.
Reduce batch sizes and cycle times so less work is in progress but is completed more quickly.
Increase visibility so the current status of all work is obvious.
Empower people to use their judgment.
Shift the drudgery to automated tools.
Process improvement cannot be delivered in isolation, however. The people and the artifacts needed to support the process must be developed in order for the process to have the desired result.
Measure process
To measure the extent to which a process has achieved its desired result, audit the process. Look for evidence that the process was followed and that the desired results were obtained.
Learn what to improve next in process
To identify the next area of process improvement to pursue:
Conduct retrospectives and blameless postmortems.
Analyze the results of the audit performed in the “Measure” step.
Build artifacts
To build (improve) your inputs, tools, and outputs:
Reduce variance.
Increase tolerances.
Raise standards (i.e., decrease margin of error).
Measure artifacts
To measure the extent to which your inputs, tools, and outputs have improved:
Test (ideally automated).
Survey customers, internal and external.
Learn what to improve next about artifacts
To identify the next area of process improvement to pursue:
Conduct retrospectives and blameless postmortems.
Analyze the results of the tests and surveys performed in the “Measure” step.
Applying it
Try the above model for improving your software delivery organization. Like my other clients, you will see your software delivery performance improve dramatically. Contact me if you need help.
You may have heard that the goal of DevOps is to rapidly deliver and reliably operate high quality software-based services. But that is only partially true. True, that is the desired result. But fostering the habits that produce that result consistently is integral to the goal.
Habits foster results. The goal is both.
In my work with clients I have found several cornerstone habits essential to creating DevOps results, and I have helped software organizations embrace these habits and earn the payoffs. Let’s explore the cornerstone DevOps habits and why these habits are as much a goal as the results they promote.
Habits come in two flavors, and the first flavor of habits is practical.
Habits of Practice
Habits of practice are the behaviors we routinely engage in, often without thinking about it. We buckle our seatbelt when sitting in a vehicle, or we stash our wallet in the same pocket or the same place in our purse. What today is an unconscious habit once began as a conscious choice to behave in a certain manner. Perhaps we were reminded by our parent to fasten our seatbelt, and we complied, consciously, until repetition forged an unconscious habit. Perhaps we were alarmed at the thought that we would forget our wallet or keys someplace, so we patted down our pockets, consciously, like Peter Falk in the final scene of The Princess Bride, until that, too, through repetition, became an unconscious habit. Whether conscious or unconscious, these habits of practice create the results that we are safer while sitting in the car, and that we do not forget our wallet.
Habits of practice lead to results.
Habits of practice create results. In DevOps, we want to deliver high quality software-based service rapidly. The habits of practice that create these results include, generally, automation, measurement, and sharing.
The particular practices of these general categories vary with each organization, as each has its own business situation to contend with. But it is impossible to deliver high quality, reliable service without these practices, especially as the scale of the operation grows to significant size.
We want to foster these habits of practice. In order to successfully develop them we need the other flavor of habits.
Habits of Mind
Habits of mind are the manners of thinking and the internal monologue we conduct inside our own heads. Just as our behavior can be habitual, so can our thought patterns. Habits of mind are important because they can create and change habits of practice. For example, we all know the feeling of frustration when something in our lives isn’t working for us. It grates at us, it taunts, and it refuses to budge. Frustration is a sign that we need to change something about our lives. Our internal monologue upon encountering frustration influences whether we will change and what kind of change we will make in response. When our thoughts reflect passivity, I always have bad luck, we are not motivated to change. When our thoughts are empowering, this is an unfortunate but temporary setback, we are moved to change something about our situation—to try again, to try in a different way, or perhaps to let off steam and simply get over it.
In order to consciously create a new habit of practice, you need to have certain essential habits of mind. The essential habits of mind drive you to change your behavior over time, until you have improved your habits of practice. Without the supporting habits of mind, habits of practice are created by default, without our awareness, and these default practices seldom are the most effective at reaching our desired goal. Even once you have changed your practice, you can only improve by honing those practices, and the drive to improve your practice stems from your habits of mind.
Habits of mind foster habits of practice, which create results.
In short, habits of mind create habits of practice create results.
Culture drives the practices of automation, measurement, and sharing, which yields DevOps results: High quality, reliable software-based service.
In the DevOps field the factors that influence practice have been collectively termed “culture.”
But culture is an opaque term that does not lend itself to scrutiny. It is more useful to consider the habits of mind that foster habits of practice.
Shlomo’s Four Cornerstone Habits of Mind for DevOps Practices
The four cornerstone habits of mind necessary to foster DevOps practices and DevOps results are agency, awareness, respect, and healthy selfishness. Read on for more details on each of these habits of mind.
Agency
Agency means thinking that you can change things, that you can improve your lot through your actions, that you are an agent in your own life. If you believe you are able to change things then you can summon the motivation to try to change your behavior. On the flip side, if you lack agency—if you believe nothing you do will change anything—then you will be disheartened and won’t bother taking any of the steps necessary to change behavior. The more agency is ingrained as a habit of mind, the more you will feel in control and able to change your behavior.
This is a cornerstone habit of mind for DevOps because DevOps habits of practice require constant refinement, and you must believe that you can change and improve the current situation in order to do so. In an organizational setting agency is sometimes called “empowerment.” Organizations, too, cannot improve their practices unless members believe that they can change the way things are done. This is why agency as a habit of mind is so important for cultivating DevOps habits of practice.
Awareness
Awareness is the mental habit of perceiving what is around you. We are not aware of everything that transpires in our lives, and often we do not recognize that our practices can be improved. We cannot change our habits of practice if we aren’t aware that they need changing. This is true in life in general and with DevOps in particular.
Respect
Respect means acknowledging the feelings and opinions of others. Organizations consist of, and exist to serve, individuals with varied feelings and opinions. If you want to change habits of practice you must acknowledge the feelings and opinions of others who are involved. This doesn’t mean you need to feel those feelings yourself; it only means you need to believe that those feelings exist and are valid.
Healthy Selfishness
Healthy selfishness is thinking of your interests so that you can help others. It means thinking about how to overcome your obstacles to success and enabling yourself to do the same for your teammates. It means wanting to make your world better so you can make the world at large better. It is the driving force behind economic and technological progress. And healthy selfishness is essential for DevOps because it is the motivating force behind improving habits of practice.
What About Empathy?
Empathy is the ability to understand and share the emotions of others. It requires lowering your emotional defenses and allowing yourself to be vulnerable to the stirrings of another’s heart. Within the context of a supportive friendship or life partnership, empathy is vital. Understanding and sharing your partner’s emotions is essential to support them through pain and adversity and to grow together with them—in fact it is the essence of such a long-term relationship.
But empathy’s importance for DevOps has been drastically overstated. At our jobs we do not need to open our hearts to feel pain. Demanding that someone exhibit empathy evokes alarm and raises their defenses. Even professionals who do directly confront emotional pain, the psychologists, counselors, and ministers, do not allow themselves to be moved by the emotions they are exposed to in the course of their work. They maintain their emotional defenses. For the rest of us who do not deal directly with pain at work, which includes everyone in the IT and software industry, respect—acknowledging feelings and opinions of others—is enough for the professional setting. Insisting on empathy at work is irresponsible.
The Habits Are the Goal
DevOps is the continuous improvement of practices to rapidly deliver and reliably operate high quality software-based systems. Continuous improvement means regularly, consciously, and intentionally developing and honing those skills. In other words, DevOps is more than just the goal of delivery and operations. It is also the habits—of mind and of practice—leading to continuous improvement.
Shlomo’s cornerstone habits of mind, agency, awareness, respect, and healthy selfishness, foster the habits of practice, automation, measurement, and sharing, that drive the results of DevOps: high quality, reliable software-based service.
Fostering the three DevOps habits of practice—automation, measurement, and sharing—requires the four cornerstone habits of mind: agency, awareness, respect, and healthy selfishness. In all, those are the Seven Habits of Highly Effective DevOps.
The classic formulation of DevOps, Culture, Automation, Measurement, Sharing—”CAMS”—is an expression of habits of mind and habits of practice: culture is a habit of mind, automation, measurement, and sharing are habits of practice. To succeed at a DevOps initiative, you and your team members must exhibit these four habits of mind. They are the keys to forming productive habits of practice and, ultimately, the results you want to achieve: rapidly delivering and reliably operating high quality software-based services.
Like companies producing any kind of offering, software companies require three elements in order to successfully deliver: knowing what to build, knowing how to build it, and knowing how to deliver it to the customer. When each of these three functions – research, engineering, and delivery – does its job independently, you get slow progress and uninspired, sub-par products. But when all three functions work together in a customer-centric fashion, you get excellence – in implementation, innovation – in concept, and world-class results – by delivering those two to the customer.
With the explosion of internet-based services in the past decade delivery has by necessity become an ongoing activity. And so in recent years much attention has been devoted to integrating development with technical operations – DevOps. DevOps integrates into a single team the skills of how to build the service and the skills of how to deliver it. But, as pointed out earlier, DevOps teams often lack the most critical element: the customer’s involvement. DevOps teams can build and deliver really fast, but don’t know what to build.
Integrating development with the customer-facing functions (marketing, sales, etc.) without technical operations results in an organization that knows what to build and how to build it, but doesn’t know how to deliver it to the customer. This organization is full of ideas – even innovative ideas – and implements them, but they don’t see the light of day. I call this a “feasibility practice” – it can show you what is feasible but it can’t deliver it to customers.
Teams that know what to build and how to deliver it to the customer, but not how to build it, also do not provide value to the customer. Such teams are close to the customer both on the inbound (understanding the customer’s needs) and the outbound (explaining the company’s offering) directions. Marketing sits squarely in this space: it knows what to build and how to deliver but can’t transform that knowledge into value for the customer.
As an executive leading a software delivery effort, make sure your organization occupies the very center so it can understand, build, and deliver the right value to the customer. The customer is here in the center as well: he participates in validating the ideas, critiquing the implementation, and accepting the delivery. It is only here that your organization can achieve excellence, innovation, and world-class results.
[Update August 2011: Blogger.com has removed Adrian’s blog so that link no longer works.]
Some important takeaways for application developers:
Your unit tests are as valuable as your code. The tests ensure the code works to spec, and they should be used as frequently as possible during development.
Make your code easy to test, with sensible defaults that require no external dependencies: e.g. don’t require internet connectivity.
Cloud limitations, both general (such as eventual consistency) and cloud-specific (such as a limit on the number of buckets per S3 account), will require careful consideration in your code (and tests).
Some things can only really be tested against a live cloud service. As Adrian points out, the only way to test that an instance launched with the desired customizations is to ssh in to that instance and explore it from the inside. This is not testable using an offline stub emulator.
But the key lesson developers can learn is: Whenever possible, use an existing library to interface with your clouds. As Adrian’s post makes patently clear, a lot of effort goes into ensuring the library works properly with the various supported APIs, and you can only benefit by leveraging those accomplishments.
On the other side of the fence, API developers can also learn from Adrian’s article. As William Vambenepe recently commented:
Rather than spending hours obsessing about the finer points of your API, spend the time writing love letters to [boto author] Mitch and Adrian so they support you in their libraries.
In fact, Adrian’s blog can be viewed as a TODO list for API creators who want to encourage adoption.
API authors should also refer to Steve Loughran’s Cloud Tools Manifesto for more great ideas on how to make life easy for developers.
