When I joined an infrastructure team as a manager at will bank I saw a bunch of improvement opportunities. A specific one was related to improving our efficiency, given that the team was short-sized. This is a simple small history of improvement that we achieve.
The problem
Initially, the team was called every day to solve a lot of issues and was constantly interrupted by many teams. We didn’t have visibility of how many issues were arriving at the people’s hands because some fast requests were solved quickly — which caused constantly interrupting. The problem is that we didn’t realize how many requests we solved per day. Engineers did not create the cards because the requests were solved fast, and the importance or value of creating them was not evident.
Visualizing the flow
Before doing any kind of significant change, we simply started to visualize this invisible work that was being done by engineers. The steps of a new problem to be solved were the same every time: 1) a developer sent a DM for some infrastructure engineer; 2) the infra engineer was trying to solve the problem, requesting more info about the issue; 3) the infra engineer resolved the problem. Given that, we start to see how to make this flow clear.
We created a Slack workflow to centralize the requests in a channel (#firefighter-devops) and create them inside the team’s board. Everyone could see the progress through the board and complement it with information, if necessary.
In the first week, we saw a lot of requests hitting our board, and after a while, we started to categorize them. This was very important because, after that, we saw a lot of opportunities to decrease the requests, using automation and creating some DevOps products.
The first product
The biggest request creation category was related to pipelines. Every new service that teams did create was a new request to create the pipeline to the infra team. This was not scalable, given that the process was the same for every new project. We decided to automate this process.
We moved the creation of pipelines from Jenkins to Bitbucket Pipelines because the way of creating a new one was easier. So, instead of requesting through #firefighter-devops, the developers could create their own pipeline, using a good practices guideline. This seems simple, but we decrease the number of requests for this small DevOps team considerably.
Incremental value
In addition to decreasing the number of requests for pipeline creation, the team started to see a possibility of delivery small and continuous. The first release of the automated pipeline creation was big, and we didn’t deliver either a small batch or in a fast-paced way.
After analyzing the first release, the team realized that could deliver only for some stack of languages instead of to some ones that just a few teams were using.
The subsequent releases now started to follow the premise of delivering small and continuous instead of a big release.
Conclusion
We need to focus on the problem more than starting to use frameworks or methodologies. A simple solution like that generated considerable value for the company. Probably some people would advocate against the ticket solution that we started to use initially, but if we didn’t do that, we couldn’t see the flow’s issues and discover where the bottlenecks were.
The future
By decreasing the volume of tickets the team now has space and time to evolve the current tools and products or even create new ones. Of course, now, they did need to create documentation to help the developers on using those tools and products, but the positive consequence of being more focused on the products than solving tickets is clear. It’s better to maintain these products instead of being a ticket-oriented developer.
Today, the team starts to act as an enabling group to other teams solving specific problems, more consultive than demanded.
Leave a comment