How to achieve DevOps consensus: The what and how of DevOps
Posted May 29, 2018 | 11 min. (2266 words)DevOps is a complex, multi-dimensional topic. It is context-sensitive. Those who attempt to learn about and implement DevOps bring their roles and cultural perspectives to the process. Diversity of opinion and expertise can be an important advantage. However, it can also lead to friction and contention in developing DevOps consensus.
Why is DevOps consensus necessary?
Consensus is needed because DevOps is not just one thing; it’s a family of ideas that addresses a wide range of problems. Any team that works together to solve problems must first agree:
-
What problems are we trying to solve?
-
What is the best tool to solve a given problem?
-
How do we measure improvement (or lack thereof?)
Enterprises and teams that want to use DevOps to answer these questions must first agree on the answers to two questions:
What is DevOps?
How can we use it to improve our software delivery?
Each enterprise must ask itself these questions to get started with DevOps, and the answers are different for every enterprise. The process of asking and answering itself yields insights into what is working, what isn’t working, and what can be done to promote effective change.
The DevOps discussion
Enterprises that are considering DevOps should start with a discussion about what DevOps is and how it is used. Resources for learning about DevOps are plentiful (see the Further reading section below for more examples.) If you are tasked with facilitating that discussion, then you’re in luck—there are many effective ways to lead the conversation. Your first step is research. An understanding of the foundational concepts will equip you to introduce them to a diverse group of stakeholders.
“Leading the conversation” doesn’t necessarily mean “teaching your coworkers about DevOps.” Your first task in order to achieve DevOps consensus is to lay the groundwork, then let the discussion proceed organically. The first conversations will be far more effective as a collaborative “discovery” session. The conversation leader tends to be more effective as a coach and referee, not as an educator. Introduce the basic ideas, then facilitate the conversation by listening and asking questions.
If you are preparing a presentation on DevOps, I recommend you limit the use of the term “DevOps” early in your introduction. That term may be fraught with preconceptions, especially if your audience has had only a little exposure to DevOps. Focus on the foundational principles, with an emphasis on how they can be used to address specific problems. Try to avoid and discourage reductive definitions of DevOps. You’ll hear things like:
“DevOps is just about automation”
“DevOps is really process improvement”
“DevOps Engineers are experts in Configuration Management”
These are examples of fast-formed opinions that do not fully appreciate the breadth and depth of DevOps. DevOps is about automation, process improvement, and configuration management, and much more. DevOps should not be treated as an orthodoxy, with a strictly prescribed way of doing things. It should be viewed as a set of tools that are used to achieve business goals.
As the conversation progresses, document areas of agreement. What is the business most concerned about? Make a short, prioritized list of those concerns. This process will focus the group on the most critical business challenges. Once they are agreed upon, then DevOps can be evaluated as a set of tools to address them.
If your enterprise is considering a large-scale strategic adoption of DevOps, your DevOps consensus discussion will need participation from a diverse leadership team:
-
Executive Management (CEO, CFO, CSO (Security), CIO/CTO)
-
IT Operations Management
-
Software Development Management
-
Project Management
These roles each have unique perspectives to offer. It is vital to ensure that each role is represented, and has the opportunity to voice concerns and ask questions. An open forum discussion will allow this cross-functional team to consider the tradeoffs of proposed approaches, and balance the concerns across all roles. This team should learn about DevOps together, in workshops and working sessions, to collaborate on the development of a definition of DevOps that is meaningful to your enterprise.
What is DevOps?
If we count from the first DevOpsDays conference in Belgium, DevOps is almost ten years old. During that time, much has been written about it. Enterprises of all sizes are beginning to get on board. I’ve had many “DevOps Conversations” since I presented at the 2014 DevOps Enterprise Summit. However, I believe we’re still fairly early in the evolution of DevOps. Many people, in a wide variety of roles, are still hearing about it for the first time.
DevOps is not just one thing, so it’s difficult to write a single, all-encompassing, one-sentence definition. DevOps is many things, and those things are open to interpretation. That does not mean that trying to define DevOps is futile. Quite the contrary—DevOps is grounded in some very well-proven concepts. These ideas and principles inform the way DevOps is practiced, but they are not a roadmap leading to ‘DevOps nirvana.’ They are guideposts, and ways of thinking about problems and solutions.
The first step to achieving DevOps consensus in your team is to introduce these foundational ideas to your colleagues, in an accessible and non-dogmatic way. DevOps and Agile share many conceptual underpinnings. To keep the conversation flowing about goals first, and methods second, begin your introduction with a focus on these foundational ideas.
Here is a list of some of the first principles of DevOps to help you reach DevOps consensus:
Short Feedback Loops: Agile uses ceremonies such as the stand-up to ensure regular communication between developers and stakeholders. DevOps also emphasizes shortening and amplifying feedback loops, as in the Second Way described in The Phoenix Project.
Lean IT: The principles of Lean IT, derived from Lean Manufacturing, are primarily focused on the elimination of waste in production. DevOps and Agile both play a role in curbing excess and increasing the efficiency of production processes.
Theory of Constraints: The Theory of Constraints was developed by Eliyahu Goldratt, and presented in his book, The Goal. The Theory of Constraints provides a method for improving throughput by a continuous process of identifying and “breaking” (eliminating) constraints. Constraints are defined as any “bottleneck” step within a multi-step production process, where work slows down and begins to accumulate as a backlog. Agile practices such as kanban visualize the flow of work, helping to expose work blockages. DevOps extends this thinking beyond the software development lifecycle, to the entire software delivery lifecycle.
Value Stream Mapping: All goods and services are produced by varyingly complex processes. Those processes are referred to in Lean as value chains, and the process of documenting them is called value stream mapping. Complex processes are rarely understood by a single individual. Each step (called a “workstation” in manufacturing) is often handled by staff with specialized skills. Software development workstations include development, testing, deployment, and maintenance.
Creating a Value Stream Map is, ideally, a collaborative process. Teams from each workstation come together to tell their part of the production story, which will help to achieve DevOps consensus. The entire story emerges from the process of value stream mapping. This exercise often exposes inefficiencies in the overall process. DevOps uses Value Stream Mapping to align production efforts with strategic enterprise goals.
Continuous Improvement (kaizen): Continuous Improvement is a scientific practice of seeking and eliminating inefficiency in a process. The practice, known in Japanese as kaizen, is associated with the Toyota Production System (TPS.) TPS is credited as the basis for modern Lean Manufacturing and Lean IT methods. Continuous Improvement teams constantly evaluate the process and results of a production team. Theories are developed about how constraints or other inefficiencies are impacting production. Teams then make small changes to the process and measure for improvement (or lack thereof.) This process is never-ending, always looking for opportunities to eliminate waste and improve productivity. DevOps practitioners use kaizen as a method to holistically measure and improve the process of software delivery.
Most of what you’ll learn about DevOps is rooted in one or more of these basic concepts. There are more foundational principles to be found in the literature (see References.) A “bottom-up” approach to your DevOps discussion will create a shared, common understanding of the first principles. That shared understanding is a strong platform for the next step: figuring out how to adapt DevOps to the unique requirements of your enterprise.
How does an Enterprise adapt DevOps?
You’ve introduced your cross-functional team to the basics of DevOps. You’ve worked together to identify the most important business goals and challenges. You’re now ready to pair those up and start talking about how DevOps can address those challenges, and you’re on your way to DevOps consensus. The conversation is ready to transition from abstractions, to approaches, and finally to a plan for implementation. It’s still a complex job, so your best approach at this point is “divide and conquer.”
DevOps can be divided into three major areas:
-
Culture / People
-
Process
-
Technology
There is certainly some interdependency between these areas, however, they can be addressed separately, in parallel, and by different stakeholders. Each of the roles listed earlier will have a stake in each of these major areas. However, there is a natural area of focus for each one.
DevOps culture
The term “DevOps” started with the dysfunctional state of the relationship between Development and Operations organizations. They are often separated into “silos,” organizationally and culturally. This separation leads to a lack of trust and misalignment of goals and incentives. They are pitted against each other in a zero-sum game. Developers want to move fast, Operations need to slow the pace of progress to maintain stability. Under this regime, both cannot win at the same time.
DevOps aims to bring Development, Operations, Security, and other stakeholders in software delivery together under a common mission. Developers should appreciate and understand the operational requirements of the software they produce. Operations need visibility into the development roadmap to be ready for new releases. Security must have a seat at the table throughout the process. The entire point of software delivery is to deliver business value to the client, which is ultimately represented by Executive Management.
Everyone has a role to play in a cultural shift towards DevOps. Executive Management, however, should be ultimately responsible for the messaging behind the shift. Communications from executives, in support of a collaborative environment, can lead the change from the top.
DevOps process
The process of DevOps is based in part on Agile software development methods. If your organization already uses Agile, you’re ahead of the game. You will likely discover some areas where your Agile practice should change to adapt to a larger DevOps practice. If you are not practicing Agile, this is a good place to start your DevOps journey. A common question about DevOps is, “can DevOps be practiced without Agile?” In short, yes, but the return on your efforts will be much greater if you implement DevOps as the larger part of an Agile practice.
Continuous Improvement is also a major theme. IT Operations and Development staff can work with Project Management and Business Analysts to identify areas of inefficiency and waste in all processes involved in software delivery. Value stream mapping can be helpful in this area, to visualize the entire end-to-end process.
Technology
The technology area of DevOps methodology is centered on one theme: automation. Anything that can be automated must be automated. This includes Continuous Integration, Infrastructure as Code, automated deployments, testing, user acceptance, and more. Just as automation during the Industrial Revolution changed industry and manufacturing, automation in software delivery has fundamentally changed the process of delivering software.
Automating tasks makes them more repeatable and reliable. Automated testing is an example of an area where automation adds immense value. Testing is often delayed, and sometimes outright skipped, when crunch time hits ahead of a release date. Automating the testing process makes it repeatable and lowers its cost. The process of testing early and often is referred to as “shift left,” as in shifting quality to the earliest steps in the software delivery process.
Automation also improves a key metric: velocity. Velocity is, essentially, the frequency at which software is released. Small, frequent releases are better than large, infrequent releases. This is related to another Lean principle: small batch sizes. When features are released as part of small changes, they are less risky. Rolling back and reworking small changes is not nearly as disruptive as rolling back a huge, complex release.
Your IT Operations and Software Development teams are best equipped to focus on the Technology area of DevOps.
In summary
DevOps can provide tools to deal with many business challenges related to software delivery. First, form a cross-functional team of stakeholders. Build DevOps consensus around a short list of business challenges that you believe DevOps can be used to solve. Those challenges will likely fall within one or more of the DevOps pillars: Culture, Process, and Technology. Identify a key challenge then frame it in the context of the DevOps pillar that fits it best. Now you’re ready to think implementation.
When it comes to implementation, start small. The adaptation of DevOps is a Continuous Improvement exercise. Take a scientific approach. Don’t try to change too much at once, or you’ll confuse the experiment. Agree on the problem, the tool, and the metrics that will indicate improvement. “Failure” in this context is not calamity; it’s learning. Keep experimenting with small changes until you see the improvement you’re targeting. With patience, persistence, and focus, DevOps may be the right toolbox for your enterprise.
