Mike Strange had the wonderful opportunity to share his perspective on DevOps during Interop in Las Vegas.
Mike: All right. So welcome to a Thursday morning here at Interop. I’m Mike Strange for those who I haven’t had the chance to meet. I’m responsible for the Los Angeles office, the consulting company named Pariveda Solutions. We’re about 500 consultants nationwide, and what we do is technology-centered, technology improvements.
So, what I’m gonna try to do today is to kind of tie together some of the things we heard in previous sessions. And give you some examples from some of the projects that we have done over the last 18 months and lessons that we’ve learned.
So, what you have in front of you, a little gift. Number one, ugly mug is me, and on here is the contact information. So, please, you know, start a chat. Connect with me on LinkedIn. We also follow each other on Twitter. I’d love to kind of have conversation with people about DevOps, about technology, and so whether you hear anything that I say today that you agree with, that’s similar to your experiences, or something you disagree with, let’s connect and keep the conversation going. To me, I think, you know, these kind of sessions, these kind of conferences are about sharing information, helping each other. And DevOps is such a wildly complex base with so many different attributes to it. No person could possibly inaudible all of it, so to me it’s all about kind of sharing, which is why I’m here today.
So, my goal is just to give you some information that’s hopefully helpful, and then begin a conversation that, ultimately, gives back to me, and that’s my goal. So I’m not gonna talk a lot about Pariveda. I’d happy to talk to anybody who’s interested in talking about that, after the fact. That’s out of respect for the conference. Like I said, the goal is to share. And I’m gonna be quoting examples from a number of projects that we’ve worked on for small companies, mid-sized, and large companies. I’m also not gonna be talking about the names of those companies out of respect for them so just to give you kind of that context.
On the back of the card, the iChart, is a maturity model. So this is our gift to you. This is a genericized version of a maturity model that we’ve used on a number of projects to talk about all the different attributes of DevOps, everything from people, to culture, to tools, implementations, etc. So it’s sort of a genericized form, and there are many of these out there. It’s very, very similar to many of the ones that you can see online. This is just our version of it that we’ve found from experience. So, I’m hopeful that that’s helpful to you.
I’m gonna be going over the uses of this in the presentation, and the good and the bad because there’s some misuses of maturity models. So I’m gonna talk about it generally, but the specifics are here for you to take home, okay, and a small gift. All right.
We’re also gonna be recording the session just for our purposes and live streaming actually, but no one’s picture is gonna be shown, except for your identity, don’t worry about that, okay? All right. So let’s go ahead and get started.
And let’s get started. Okay. So, let’s start with expectations. So one of the things that we have found about DevOps and looking at the variety of projects we’ve done is that expectations vary widely, about what it is the people hope and want to achieve. And so, one of the things we’ve noticed is that what people ask for, what they say they want, is often heavily colored by what they expect, right? And expectations of sponsors are often created by small snippets of information that they have collected from a variety of highly reputable sources like airline magazines, right?
So, it’s very interesting. If you think about it from the perspective of a CEO or a CFO of a, you know, Fortune 500 company that has 50 different things on their list, my guess is that the subtleties of implementations of DevOps is not on the list, right? So, the question is, what do those kinds of people expect from these kind of initiatives?
So, we created this sort of chart. These are words that were taken in classic cloud art form, WordArt form from Charters. So, these are actual words that were taken from a number of different DevOps project charters. And so the words that people use to describe what it is they’re hoping for matter. And so there’s a couple of good things about this, and there are couple of things that I don’t like about this. It’d be interesting, for you all, to look at the charters and the documents that you all have and how it aligns with this.
So, the first thing I noticed is that development and operations are fairly commonly mentioned, right? So, at least they got the acronym right so that’s good, good start. There’s also a few things on here that we’ve noticed that people talk about visibility, transparency, process, those kind of things, really, really good awareness about some of the procedural implications, right?
A couple of things that I don’t like that I often see and I worry about, one of which is project. So, I don’t know if you guys see and hear that in the way you talk about DevOps and the way it’s received in your companies. I’m deeply concerned every time I hear somebody say, “DevOps is a project that has a start and an end in ROI.” Right? Big trouble. And a lot of times, that’s the way things are perceived. That’s the lens that a lot of executives look through, right?
I’ll give you some examples of things that I worked on in the past. I was the CTO of a public company, and so I’m very familiar with some of the conversations that go on around this, and a lot of things were looked at as a project, right? And so DevOps properly done is not a project, and I wrote a whole LinkedIn post on this. Go to my profile, and I wrote a whole long, you know, rant about why this is not the case.
The other one is tools. So, one of the most prominently mentioned things about DevOps is it has something to do with tools, either that concept generally or specific items like Jenkins or something, right? Very, very common. So, the more that we think of DevOps as a implementation of a tool in order to execute a project, the more off the rails, I think, we get. That’s just my personal opinion. And so then if you look back, after the fact, at these projects, you get an interesting perspective about what actually happens in reality.
So, we took a half a dozen projects, and looked at the actual results after the fact, and then compared them to this, which were the expectations at the beginning, and came up with kind of a different point of view. And I’d love to get you guys’ reaction to this.
Our perspective is that DevOps is a combination of two things. Yes, it has everything to do with architecting a solution, and architecting does not mean necessarily technical architecture. It means process architecture, and people, and roles, and definitions, and responsibilities, and standards, and tool selection, of course, etc., etc., right? Pipelines, and cloud usage, and all kinds of things would come into that. Yes, it’s extremely complex, architected solution.
However, it’s also a journey, right? So, the idea that we’re actually connecting with virtually every participant in the IT ecosystem, in some way, emotionally, in order to understand what it is their roles are. And the fears, and the responsiveness to all of that are one of the biggest factors we’ve seen that hold people back from getting what they expected in the beginning, right? So DevOps, at least in some of its higher-level instantiations is all about accountabilities, for example. Like having developers that have greater awareness of the operational implications of their actions, operations personnel that are more aware of the development possibilities that can happen early. All of those things create fears, right? And there’s also a lot of other stakeholders that might be affected by this, executives and auditors. I’ll talk to you in a second about the perspective of external views of DevOps and how this actually goes against the grain of some classic IT paradigms, right, because we’re all about risks. And we’ll talk about separation of duties in a second and some issues like that, that cut across the grain, right?
So, our conclusion is, in hindsight, looking at these projects, these are the two things that are happening simultaneously. The reason we sort of highlight that in that way is because a lot of the failure conditions actually exist in category 2. I don’t know if you guys see that and feel that, right? Most of the time, we don’t script syntax, right? Right? The tools are fine. The people, the people is the hard part. And so if you think about DevOps as an initiative that it incorporates those two pieces simultaneously, and it’s gonna be a long-running program with continuous improvement, which means that both have to have to be addressed continuously. Then you step back from that and think, “Am I running my DevOps program with this kind of a view?” But if you’re not, if you’re not talking to your sponsors about what they need to do to sponsor things in both categories, something is gonna get missed. So these are our conclusions kind of looking back at projects.
So, I’m gonna take each of these two things, and break ’em down a little bit, and try to give some experience and some expertise in each of these, and give some examples. So, first, expectations start with value. So, the description about what it is we’re all trying to achieve, and going back to all those charter documents, tends to breakdown in number of different categories. Super, super important to get this right. If your whole conversation is about cost, and ROI, and “What am I gonna take out of this year’s budget,” then we’re not thinking about all the aspects, right? That can be part of it, right? And it’s clear that there’s ways of making this, you know, self-fulfilling or self-funding, certainly. So, there’s a lot of cost aspects that are super positive, but one of the big ones that shows up, of course, is saving money.
I’m really glad to see that most of the projects that we see involve some sense of responsiveness, right? The business says, “I want you to be more responsive to my ongoing needs. There’s a constant need for change, and I’d like you to do something, whatever that is, to become more responsive,” right? So the idea of doing deployments more often, being more aware, having more involvement, all of that transparency, awesome, right? So, glad to see that.
Improving effectiveness always tends to show up in some form. So, yes, there’ll be less meetings, more effectiveness, more visibility, more understanding, perhaps less misunderstandings. The best thing is going to production, right? Some of that certainly happens.
One of the things that we see often in a number of projects that has never…I’ve never seen mentioned, in the projects that we looked at, was satisfaction. So it is actually possible, assuming that people can get on board with all of these, to improve employee satisfaction, right? What do people want? Visibility, understanding, respect, awareness, the ability to influence the outcome, right? Where my accountabilities and my authorities align, I tend to be happier, right? It’s possible to achieve those, right?
Does anybody here have a charter document that talks about employees sat? No. Okay, I don’t think so. It’s an awesome goal, though, and it’s actually quite achievable. We see it, we see it on many, many projects where satisfaction actually goes up, right? That’s a really cool goal. Does anybody sit and talk with the HR department about employee sat from DevOps? No. I’m guessing no, right? It’s true.
And clarifying accountabilities kind of goes along with that. So there’s always something about making sure that people are accountable for what they have authority to do. I was classically trained as a developer so I tend to be on that side. And so more awareness of the implications of my actions just makes me more empowered, right? How are we gonna measure this in production? How does security gonna look at this? How are the auditors gonna look at this? What about improvement? What about all these rollback scenarios that you’re constantly worried about that I didn’t have to worry about, right? I think the more awareness just increases accountability and makes me happier, right, but also creates a lot of fear. So suggestion from all of these is get really clear about what the goals are, and they tend to break down in this kind of categories.
Another thing we found that’s super important is starting some place that builds some trust early. So, start with a project, with some subset of your portfolio, whatever, and choose the place that you start with intention, right? So, pick a project that falls into a category that gives you the outcomes that you wanna look for in the first couple of months so that people get comfort around what their original goals were, right? Don’t choose the simplest application that’s used only for internal purposes that’s released twice a year, right? Terrible place to start. Nobody’s gonna believe this, that this is an extensible model, right? So, start with something that is relevant, that’s appropriate, that falls into the category of applications in your portfolio, on which you intend to use these principles, right? And get clear with your sponsors about what it is you’re trying to achieve.
We came up with kind of a framework for thinking about where to start. And so you might choose things based on value, of course. Business risks, you probably don’t wanna start with your topmost, mission-critical application while you’re still working out your pipelines. You wanna look at dependencies, so you don’t wanna start with a project that’s gonna take you eight months in order to change all these behaviors and all the deployment scenarios, right? So you wanna start someplace where you can show progress early. And then, we actually found a number of cases where applications were chosen just based on the need to be building to start with, right? So, that’s a perfectly logical place to start. You’ve already got sponsored dollars that you can leverage. These are practical issues, right? So, we tend to find where to start falls under one of five categories. So, we’ll spend some time thinking about that.
As you do that, here’s the things that we tended to get asked for in the first, say, months, six weeks or so. Show me something that is…that you can demonstrate, right? If you’re gonna go down this whole, continuous deployment pipeline concept, and you wanna actually reach for roll forwards and some of these really advanced concepts, then show me, right? Show me enough to build trust, that I know that the organization is capable of handling all of these, right? If you’re gonna change the process, if you’re gonna change role definitions, create cohesive teams…We had an awesome session yesterday about cohesive teams, right, all different roles that can come together. If that’s your new model, then show me, show me that actually working, right?
So, what is it that sponsors need to see early to build comfort, right? And back to that emotional connection. So often we miss this, right? We’d say, “Oh, we’re gonna stand up in Jenkins Server. It’s gonna be great, right?” No, that’s not the demonstration I was looking for, right? What I’m looking for is the demonstration of how all this is gonna come together.
Repeatable, so, again, don’t start with a simple application that’s not the norm, right? I wanna see that it’s repeatable, that it’s practical, that people are gonna be able to make this leap, right? So don’t stretch…we’ll get to the maturity model in a second.
All right, maturity model. So, what you have in front of you, like I said, is the detailed version. This is a generalized concept that breaks down DevOps into five pieces. And like I said, it’s pretty standard and there’s a number of ’em published. This is actually adapted from some original works that came out of Amazon. But there’s five different categories that represent the entirety of what we’re talking about. So there’s people, there’s the tools and the architecture. There’s actually the whole deployment pipeline concept. Testing and verification, you can add security to that and then the information that comes out of it, right?
So the question then is where do you want to be in terms of your maturity in each of these categories? And there’s ton of details. These are just very, high-level bullet points, but it breaks down into a ton of detail. So couple of things about this. One is, be really clear about where you’re starting. Most people are starting at relatively low levels of maturity if you consider this in a DevOps framework, which is totally fine but be really, really clear about your destination. So, I’ll give you an example right here in a second of somebody who actually chose moderate maturity for very particular reasons.
So, one of the failure scenarios that we see with this model or any of the other ones that look and feel very similar to it, is that everybody thinks that a 5 is an A, and a 4 is a B, right? So, “I need to be an expert in everything, or else I haven’t achieved my goal as a technology professional,” right? Dangerous. We’ve seen this actually done where people will go through and give themselves a score, like 3-3-3-4-2. Therefore, I got a 15 out of 25 and I failed, right? We do it all the time. So this is not a scorecard, right? I would not think of maturity as a scorecard, much like CMM. Anybody who studied CMM concepts, all that kind of thing? This is not a scorecard to evaluate whether or not you become good enough. It’s an idea of where is the destination and where does it fit into our portfolio?
There’s other subtleties that we’ve seen people do a really good job of breaking down their portfolio into two different groups. Got at that bi-model concept that says, “I got a portion of my portfolio that’s high risk and it might actually accept a slightly lower maturity, right? I don’t wanna do daily deployments.” And then there’s other areas that are focused on innovation. And so, you know, machine learning or something like that where the risk is low but the desire for innovation is high. So you can actually segment your portfolio and have two different maturity models, maybe even two different sets of pipelines based on how your portfolio ranks up. Because generally, most organizations have two sets of goals, predictability and innovation, right? Generally, not at the same time. So, you might apply the maturity model in two different ways, and we’ve seen that. People choose two different destinations.
So here’s the high-level concepts that are described in more specifics here. You’ll see people talk about groups working together and collaborating, becoming more transparent as you become more mature, and then moving toward the higher order form of people, which is deep ownership, right? So do you have teams of people that are cross-functional and collaborating, that own everything from requirements clarity to production stability, right? Can you actually have cross-functional ownership? So that’s the highest order form. And then you can go down through the other categories, talk about automation, which is the one we always…everybody loves to talk about. Testing automation, scripting, etc., super awesome to get mature in that category
And then the information, the idea that we can collect more and greater information about what’s happening throughout the entire pipeline through these wonderful tools is awesome, right? And I’ll give you an example here in a second, all right? So here’s one example of a multi-billion-dollar enterprise that actually…and this is one of their two models. This is an organization that shows two different levels. And so the little dot at the bottom is where they started, and the area which where they’d like to get to in a reasonable time-frame, in this case, two years. Okay.
So, just couple of interesting things to note about this. One is that they chose relatively moderate level of maturity across everything except information, right? And that’s a perfectly valid paradigm, and we’re working with them right now. We’re roughly halfway through this right now. It’s perfectly valid to say, “Hey, in my portfolio, I have…I desire an intermediate level of pipeline automation, deployments. I’m really not gonna go to continuous deployment. I’m not entirely comfortable with this roll-forward scenario,” right? In all cases, that’s my knee-jerk reaction. So, those have implications, right?
In this case, this organization is largely outsourced. They have a lot of vendors, including a number of offshore, a lot of complexity, right, in the environment. They got service-level agreements, longstanding arrangements, a lot of things that are a little trickier to move. And so, they said, “Hey, in that environment, I’d rather have a moderate level of build and deploy with increased accountability, better collaboration. But what I really, really need in this complex environment with thousands of applications and literally, you know, most of my organization is outsourced, is I want monitoring, right?”
“And so as vendors, as consultants continue to work on my projects, what I want is better information. So I wanna see what’s happening throughout the pipeline so I can evaluate them as far as their partnership, and how it is they’re adopting and becoming more innovative?” So in this case, they chose moderate for the technology, and high on information. All right. So does that make sense? And now you can kind of measure progress against that.
Woman: [Inaudible 00:20:27].
Mike: Yeah, in this case, it was, you know, thousands of applications, right? So, they wanted to say, “For this portion of the portfolio, given the fact that we have a lot of our maintenance outsourced and offshore. We got service-level agreements and things that are already in place that are gonna be difficult to re-negotiate, and we set up two-year timeline.” So they said, “Hey, practically, realistically, I think I can only move that needle little bit every year.” So it’s a time-frame thing I think is the answer to your question. I mean the goal, in this case, certainly would be to get to a high degree of automation, but it’s a two-year thing. All right? So, be careful about this. It’s not a five-out-of-five kind of a thing. All right.
A couple of other things we found super helpful that are specific sort of technical artifacts that can be created along the way as part of the DevOps journey and not the DevOps project, is that this, to get really clear. So, we do need scorecards, we do need red, yellow, green, right? And the maturity model is not it, right? So this is not your evaluation criteria, but you need one. So, if you break down everything that’s in here, you say, “This is what I wanna achieve,” then you can take all of it and come up with a readiness checklist. It says, “Where am I in terms of role definitions, pipelines, tool usage, automation, scripting, cloud adoption, skill levels, all those different components,” right, and give yourself a score. It’s fine, right? But give yourself a score compared to your destination. So, yes, it is useful to have scorecards, and it’s useful to have red, yellow, green indicators, and in this case, it didn’t have that many items. It got about eight things on it, right? So at a program management level, you can have a…you should have a scorecard.
The roles map to the process. So, on the top, you’ll see kind of the high-level process map at the very top of that item, and then underneath it are the role descriptions. So, one of the things that we found, you guys have probably found the same thing, is that in many cases, the roles of a lot of people, their responsibilities change in fairly significant ways. And there’s a lot of reactions to that, including a lot of passive-aggressive behavior, and some of those kind of things are just to natural human beings.
So, be really clear about, “Hey, we’re gonna go everything from requirements ownership all the way through production stability, and talk about who owns what,” right? And including product owners. It’s not just an IT thing, it’s an IT-ecosystem thing. So coming up with the roles, right? So what does increased accountability or increased, you know, responsibility mean for developers? Be really clear about that. So they can go across and say, “Hey, for my role in the organization, this is what you’re saying is my new destination, and I’m comfortable with that,” right, so… And then people can ask for training, and ask for support, and mentoring or whatever else they need in order to achieve that. So this was super, super helpful in a number of our projects. And then the process clarified, of course. So those two things kind of go together.
So, these are the specific technical artifacts that we found useful, right? Do other people use things like this? Forms of the same thing? Just in a different form? Okay. All right, so that’s the architecture.
Now, let’s get to the second part. So, it’s a journey. So who here has got a little bit of fear related to DevOps? Okay. You didn’t put your hand up. You’re there, you’re just lying, right? Okay. Like really high, right?
I’m afraid, right? You should be. Not because the risk is not worth it but because there are risks that we need to talk about, right? There are understandings that we need to talk through. There are expectations that may or may not be aligned with reality, right? So the question is, you know, how do we deal with the fears and the uncertainties that we see and perceive as part of these project teams?
So, couple of stories, everybody knows where Waterfall came from, right? So Waterfall was created as a methodology to try to increase predictability, right? It’s a classic reaction to IT issues. And so, whenever we have a problem in IT, we tend to wanna apply controls, right? I’ll get to the auditor story here in a second. But the response to IT predictability is, “I know what we’re gonna do, we’re just gonna write everything down and make somebody sign off on it,” right? And then nothing will change, of course, and it’ll be great, right? And we’ll have extreme predictability of the outcomes of IT development projects. Perfect, right? It didn’t work, right?
And the reason that didn’t work is because predictability is only one of the IT goals, right? Innovation and the artistic side of what we do is also part of it, right? So you don’t just want manufacturing. That is not a manufacturing industry. There is a manufacturing component, right, but there’s also an innovation artistic component. And we don’t want to stamp out innovation in order to achieve predictability.
And so the idea that the business knows everything that they want a year from now with absolute precision is ridiculous, right? So what do we end up with? We ended up with, you know, short cycle times. It’s called Agile, and Scrum, and all of that, right? It turns out that the best way to focus on predictability, the best way to achieve predictability is not to focus on predictability, right? The best way to achieve predictability, in terms of the outcome, is to give people cohesive teams and increased discussions, right, increased transparency. I think they’re a perfect correlator for where we are with DevOps, right?
So, classic standard in the industry. I worked for KPMG for 10 years. I was an IT auditor and so I know that side of it as well. So I’ll tell you a story about auditors. So, one of the classic standards in IT risk assessment that internal and external auditors will look at is risks and associated compensating controls, right? Everybody’s familiar with this? Has anybody ever gotten audit finding, right? Okay, a couple people got audit findings. Good. I got two. We’ll talk about it in a second.
So, auditors look at risk and so where are your IT risks and where are your compensating controls, and will any of that, in aggregate, affect the going concern, right? That’s the point. So, one of the things that they looked at is separation of duties. This is a problem for DevOps. This is a direct problem. Every single project that’s in audit finding, this is a big deal, right? So now me as an executive, I have to write up a whole description about why it is that I respect and understand your concerns, and that this is a valid concern, and here’s what I’m gonna do to mitigate it. And now I’m responsible for doing whatever it is I promised in the coming year, or else you’re gonna hit me again, right?
So this is the way it goes, right? And I’ve been on both sides of this table. So, what I do is I sit down with the other partner, and I say, “I tell you what, I understand the core essence of your concern, which is production stability and stamping out potential collusion, right, so that we can have people who do things nefariously.” I get that. I believe that in order to answer those issues and increase accountability, increase transparency, and get more out of your IT function, what we really, really need is to blend those two roles, right?
What I want is developers to have more awareness of the implications of their actions, operators that are more aware of what’s possible, product owners that are more involved, security people that are involved in the process, right? So that we do all the assessments all together, which means that we have not only higher predictability, but I think by working together, there’s much less chance of collusion because what we’re gonna…
Reach out to Mike Strange to continue this conversation on DevOps!
