DORA metrics have become the de facto standard for measuring software development and delivery success, but a study last month identifies where those measures alone were lacking.
The first-ever Engineering Benchmarks Report, recently released by software delivery management platform provider LinearB, provides insights from 1,500 organizations into what makes one DevOps team more effective than another, and what sets the top tier of development teams apart from the rest.
RELATED: LinearB’s Engineering Benchmarks Report
The four key DORA metrics look at deployment frequency, lead time for changes (cycle time), change failure rate, and mean time to recovery. Ori Keren, co-founder and CEO of LinearB, says those metrics helped DevOps teams move “from darkness to visibility.”
But organizations that only track DORA metrics, he said, can miss the business implications of the software work being done. Keren explained: “I like this analogy to a car and an engine. The engine could be working fine, but you could be navigating this car in the wrong direction.” So DORA can show teams that they’re working efficiently, hitting all the metrics and humming like an engine, but Keren said you need to balance those with business metrics to know that you’re moving in the right direction.
Part of the problem, Keren said, is that DORA metrics are lagging indicators. The report noted that “acting on leading indicators and KPIs is more effective than missing [a goal] and backtracking to correct problems.” And some DORA metrics that are important to the engineering team might not be meaningful to the business, according to the report.
The study also lays out 10 benchmarks (see image at top) to define teams in terms of being elite, strong, fair, and needing focus on certain metrics. Included in those benchmarks is a line called planning accuracy. Keren explained that means if the organization is committing to certain things in a certain timeframe, how much of what was committed to was delivered.
Among the data that somewhat surprised Keren is that elite organizations, which he expected would deliver on 100% of what they committed to in planning, are delivering only around 80%. And, he pointed out, most of the companies are not in that target area.
“But just the fact that we have those benchmarks and now [an organization] can check itself as to where it is, that’s the beginning of this big shift that you can start tracking, aiming in the right direction,” he said.
To begin to approach elite status, the report says that organizations should embed business goals in the team’s workflow, and anchor check-ins and recurring ceremonies around those goals.
Then, he advised, before you plan your work, cut it to small chunks that you can move quickly through the development process.
Finally, Keren said it is important to reduce the friction that exists in the development life cycle, and Keren said that more automation in those areas is key. “With things like generative AI, code is being written fast. But the process of orchestrating everything that needs to happen to the code from when it’s written until reaching production, all those phases are definitely where more automation can help.”
By bringing Agile and DevOps practices, as well as value stream management and business agility, into strategic planning, teams can modernize their development processes and become elite engineering organizations.
Read the full report here.