Q&A: Nicole Forsgren talks about getting real results from DevOps

The idea of DevOps is getting a lot of attention in the enterprise IT space. But moving from big words to big results can be challenge for many organizations. I caught up with Nicole Forsgren, professor of MIS and accounting at Utah State University, who along with a few others launched the latests State of DevOps study, which dived into some of the characteristics of organizations that are getting measurable results from DevOps initiatives. 

Service Virtualization: What do you see as the most easy to verify and significant performance differences between companies that implement DevOps and those that have not?

Nicole Forsgren: First, I should point out that based on my data, I can only make statements about performance differences we see among teams that use differing levels of DevOps. That is, I don’t really have a significant set of data from companies or teams that don’t do DevOps at all to compare to companies or teams that do DevOps; what I do have is data from respondents that rate the level of DevOps that their team does (which I have statistically summarized into low, medium, or high), and then used that to look at corresponding performance differences.

From here, we can look at the impacts of DevOps on performance of the IT function, or performance of the organization. When talking about performance in the IT function, the biggest performance differences we see are in agility and reliability. In terms of agility, the highest performing DevOps teams have 30 times more frequent code deploys and 8,000 faster lead times when compared to the lowest performing DevOps teams. In terms of reliability, the highest performing DevOps teams have twice the change success rate (when implementing code) and 12 times faster mean time to recovery when code does fail.

When talking about performance of the organization, I looked at how the DevOps performance in the IT function translated to organizational impacts. High performing DevOps organizations were twice as likely to exceed profitability, market share, and productivity goals when compared to their low DevOps peers. When available, I looked at stock market data and found that, for those companies that were publicly traded and had data for the previous three years, high performing DevOps organizations saw 50% higher market cap growth than their low performing DevOps peers.

With such tremendous benefits, what do you believe have been the biggest challenges or impediments keeping organizations from implementing a comprehensive DevOps strategy?

Quite honestly, I think it’s the thoughtfulness that is required to implement a good DevOps solution. In order to really see impacts make their way all the way to the bottom line, you can’t just buy a server (or any other technology solution) and throw it in a closet. Or said another way, it’s not just tools. It has to be the right mix of technology, and culture, and process, and reengineering. Because it’s not just IT, it’s the practice of IT. And that mix will be different for every organization, and often for every team. It isn’t a one-size-fits-all solution that you can purchase and install; nor is it an engineering equation you can plug your variables into and solve. You must be thoughtful in your approach, because you must include technology and process and culture. It’s often the culture piece that is messy and tricky, but the culture can also be the most important and impactful piece if you get it right.

What can DevOps champions do to build the case required to successfully fund DevOps initiatives?

The best place to start is to get a solid understanding of what is important to your organization, and build your case around delivering value to that key function. This is the place to start because saying that you want DevOps to keep systems up or to deliver functionality faster “because reasons” isn’t enough in business; be prepared to make the case and connect the dots if you want to be the champion.

Once you can clearly speak about how DevOps practices will directly support the goals of the business by delivering content more reliably, find a management champion. This champion should be high enough in the organization to have political power and the ability to request resources, such as funding or headcount. This person can also help you refine your understanding of the business goals and your pitch about how DevOps will help the business reach those goals in significant ways.

From here, it’s adding to the above information by planning and budgeting for tools, technology, and headcount. If at all possible, I would also strongly suggest budgeting for a consultant or another external professional to come in and help architect a solution. I mention this because outside help is almost always trusted and esteemed higher than someone inside the organization, especially when making drastic changes such as implementing DevOps.

Aside from securing the budget, what other things do DevOps champions need to consider in building a successful initiative in terms of budgeting, benchmarking, resources  analysis, and communicating costs using tools like net present value?

Wow … this is quite the question, and I could probably talk about this for a very long time.  I’ll try to keep it to two basic things:

The first is education. As a DevOps champion, part of your role will be to educate those around you about what DevOps is in general, what it means to the organization, what it is to their team, and what it is to them as a manager/developer/IT operations/security/etc. And you will likely need to continue educating people throughout the process and as the initiative grows and changes. Some things that can help early on include encouraging them to read The Phoenix Project, or reading Continuous Delivery, or attending training briefings. Setting up weekly lunches or other communities of practice can also be very beneficial.

The second is adopting a continuous improvement process. By establishing short-term goals and checking in every week, or two weeks, or month (depending on the goal), the team can clearly evaluate how they are doing. The team should then re-establish the next set of short-term goals and repeat. This allows for creativity and adjustments throughout the process.

Why do you believe that 10,000+ employee organizations are less likely to be high performing versus 500-employee organizations, and what can they do to keep pace with smaller ones?

I actually don’t believe that 10,000+ employee organizations are less likely to be high performing compared to 500-employee organizations. I have seen several high performing enterprise organizations, and I see them throughout the data when I conduct studies. I might agree that the enterprise organizations are less likely to adopt DevOps practices quickly, or might find adopting DevOps practices more challenging, but that is just because they are operating at scale. Recall earlier I said that implementing DevOps is different for each organization and often for each team? Well, in 10,000+ employee organizations, they likely have several more teams than 500-employee organizations, so the scale for implementing DevOps is just bigger, and often, more daunting. Then add in the complexity of integrating those teams… you get the picture. However, we are seeing more and more enterprise organizations implementing DevOps practices with great impact. One common way enterprise organizations are successfully introducing and implementing DevOps is to roll out in special teams or special projects, then let the practices spread to other teams; it doesn’t have to be a full scale tear down and implementation of DevOps to the entire organization at once for it to be considered a success.