Romano Roth is Chief of DevOps at Zühlke. In this interview, he explains why DevOps is not bullshit, how transformation succeeds in companies, and what IT students really need to learn.
An interview by Daniel Ziegener, published on January 7, 2023, at 11:14 AM on https://www.golem.de/news/devops-wenn-wir-ueber-devops-reden-muessen-wir-ueber-menschen-reden-2301-170592.html
For some, DevOps is a collection of methods for product development; for others, it’s simply bullshit. In the Golem.de newsletter “Chefs von Devs”, Romano Roth, Chief of DevOps at Zühlke, explained why DevOps is not bullshit, and why some people still think it is. Because developers “who say DevOps is bullshit are often trapped in a role somewhere between Dev and Ops,” he says.
Golem.de: Mr. Roth, what is DevOps to you, and why isn’t it bullshit?
Romano Roth: DevOps is not bullshit because it helps with an important problem: We keep throwing things over the Walls of Confusion. The business has many good ideas. They write these ideas as requirements in Word documents or Jira tickets and then throw them over the Wall of Confusion to the developers. The developers look at them and say: If that’s what you want, we’ll develop it for you.
After development, it goes into the build pipeline, where the testing team notices that the result doesn’t match the requirements. They test something anyway, somehow it turns green and moves on. Operations wonders how this is supposed to be operated at all, but they figure it out somehow. Then it goes back and the business says: That’s not what we ordered!
On one hand, we have the problem of organizational silos; on the other hand, we have the problem that we’re still working in projects but developing products. In a project, we focus on output: number of features, stories, code, and so on. With a product, we focus on outcome. We want to solve the customer’s problem, change their behavior. It might be that we can achieve this with a single feature.
A product also has no start or end date but is continuously developed. Therefore, we need to organize ourselves along this continuous value stream, instead of specifying for months, then implementing for months, and testing for months. This needs to happen incrementally and continuously. And that’s where DevOps helps.
Golem.de: How can a concrete term like DevOps represent the entire value stream?
Roth: When we talk about DevOps, we also need to talk about people. DevOps is a poorly chosen term because it technically means “Development and Operations.” Now some people try to fix this by including Security. Then we have DevSecOps. Others say: “But the business is even more important” - and we get BizDevOps. But even this term falls short. Because it’s about all the people who work on the value stream.
We would actually need a term like DevSecBizArcComQaOps. And I’m sure I’ve forgotten someone there too. We could also call it Dev*Ops or DevXOps. But let’s just call it DevOps. Because DevOps is about bringing all people, technologies, and processes together to continuously create value. That’s the core of DevOps.
Golem.de: So is the “bullshit debate” mainly about misunderstandings and terminology?
Roth: I believe: yes. The people who say DevOps is bullshit are often trapped in a role somewhere between Dev and Ops. They only see a small part of the entire value stream, and can’t optimize it either. This leads to friction and frustration. In the articles that call DevOps bullshit, it’s usually about a developer being overloaded or overwhelmed because they’re supposed to wear so many hats but can’t change the system.
When management thinks they can eliminate Ops and have the Devs take over, it leads to exactly these problems. But it’s not the purpose of DevOps to eliminate a person or their skillset, but to bring people together so they can continuously create value.
Well-intentioned, poorly implemented#
Golem.de: So DevOps is well-intentioned but poorly implemented.
Roth: Oh yes. When we want to organize along the value stream, it means we need to change the organization. When changing an organization, there’s always a loss of power.
Decisions are now made by a team, and you as a manager suddenly have less decision-making authority. This often leads to problems because leadership must also change along with the organization. The classic manager must now transform into a leader or coach. Unfortunately, most companies don’t take this step. I’ve seen this actively torpedoed because careers are at stake.
I’ve talked to people in companies where we conducted a transformation who told me: “I don’t know what to do anymore; I’ve worked toward this position for ten years, and now this entire career path no longer exists.” They were department heads, but when you dissolve silos and empower teams, this position is no longer needed. I can understand this frustration. However, this is how it must be done if things are really going to change: Either you change the process organization, or, even better, the structural organization. But most companies shy away from the latter.
Golem.de: Zühlke is no longer a young company. How was the structural change implemented at Zühlke?
Roth: We had exactly the same problems, and it took a while for this to be understood. With us too, some people had to give up power. Previously, we always had to pitch to the executive board when we wanted to implement a new idea that required time and money. Sometimes the executive board understood the idea, sometimes we got budget, sometimes not.
We introduced an adapted form of the helix organization and organized ourselves along the value stream. On the horizontal, we have market units that focus on our core markets. On the vertical, we have our practices that provide the competencies. Each market unit and practice has its own budget and uses it as they see fit. This has led to more autonomy and entrepreneurial thinking. We now have fewer managers, and some of them had to accept new roles.
Golem.de: What are good ways to introduce DevOps from your perspective, perhaps also in a company that doesn’t have this engineering mindset?
Roth: It’s like everything: Start small. Try to select a team with suitable people who have a certain lighthouse character and are well networked in the company. Show the company that it works. And from there, you change the next product and the next, and so on. Do this on a rolling basis; don’t try to introduce it everywhere at once.
Such change processes can also paralyze companies. If you introduce change too big, your company will only be busy with this change. If you introduce it continuously, you have the chance that the company can better digest the change and learn continuously. But it also takes longer.
Too little patience in transformation#
The annoying thing for the CEO is that such an approach takes several years. They often want quick results. Especially with publicly traded companies, they want to make the big throw and then complete the transformation. The thing is: Such a transformation never ends. It’s a continuous process.
Golem.de: Now we’ve talked about how companies transform their processes. But how did you come to the conclusion that DevOps is the best way to organize such a value stream?
Roth: Basically through laziness (laughs). When I started at Zühlke as a programmer after university in 2002, it annoyed me that I always had to run the same tests on my applications. You make a change and have to run the same tests manually again. Because that was too stupid for me, I started automating, after all, I’m a programmer!
The tests grew larger over time, more sophisticated, but also more maintenance-intensive. But they were always run locally. Then came Team Foundation Server. A continuous integration system. That was my first gentle touch point with “DevOps.”
Golem.de: So you laid the foundation for your current position 20 years ago.
Roth: Basically, yes. It’s a journey where you make mistakes, learn, and continuously build the mindset. But I think the biggest impact came from the books The Phoenix Project, DevOps Handbook, and Accelerate. Before these books, I had various solution approaches for challenges. After reading these books, I really understood why these solution approaches work.
Good education and startup promotion in Switzerland#
Golem.de: Zühlke is headquartered in Switzerland. How is the technology industry there?
Roth: You see many startups that come but also go. There are pure software startups like Doodle. Much more interesting are startups like Cellsius, developing the battery-powered aircraft E-Sling. Or Cutiss, producing artificial skin by the meter that can be used for burn victims.
When I look at the startup scene in Switzerland, I see a trend toward cyber-physical systems, a combination of hardware and software. Currently, you see many fintech startups having problems and disappearing again.
Golem.de: Why are these products more common in Switzerland than pure software applications?
Roth: I see an interesting point here: DevOps comes from Agile, Agile comes from Lean, and Lean comes from the Toyota Production System, from factories. Currently, I see this approach also increasingly arriving in hardware-related development in Switzerland.
Golem.de: Does such a workflow, such a philosophy, directly influence the type of products created with it?
Roth: I believe the influence is very large. You have an iterative approach with which you can take small experiments, get feedback, and then build your product so that it solves the customer’s problem. The startups in Switzerland have figured this out.
Golem.de: Does Switzerland owe its startup scene to good education?
Roth: Good education is very important, and ETH really has sensationally good education. But I wouldn’t attribute it solely to that; company founding is also supported. Startup promotion is the key, but if our students weren’t so well educated, they wouldn’t even get that far.
End-to-end development “criminally neglected”#
What I also criticize repeatedly in Switzerland is that while you learn the technology, how to program or set up a Kubernetes cluster, end-to-end product development is still criminally neglected in my opinion. Yet you save a lot of money and time when you develop the right thing correctly, so that it’s also testable and operable.
Golem.de: Should students learn more about processes in a company?
Roth: In my opinion, yes. Students should think not in projects but in products that have concrete benefits, are testable, and are operable.
Golem.de: How often do you see code yourself?
Roth: There are things you can solve brilliantly with code, but at some point, you reach the point where you realize there are challenges you can’t solve with code. You also recognize that solving these challenges solves many problems for programmers, yes, that it even leads to code not having to be written, or the right code being written correctly. Nevertheless, I try to always stay fit at coding.
I participate in the Open Hacks at Zühlke; there we mostly do pair programming. Of course, I’m no longer as fit as my colleagues, but I try to stay on it. In product development, I’m usually away from code, but through my broad technical know-how, I can drive many things in the right direction.
Golem.de: And what is the right direction?
Roth: With the speed of new technological possibilities, companies need to reorganize so they don’t miss digitalization. This means they need to organize along the value stream.
Product teams continuously deliver value for customers via digital conveyor belts (CI/CD pipelines). These digital conveyor belts are, in turn, the product of the Platform Engineering team, which ensures that product teams can deliver value as efficiently and independently as possible. This is the industrialization of software development, the digital factory of the future.
Original Article: Golem.de: DevOps: “Wenn wir über DevOps reden, müssen wir über Menschen reden”