Episode #423 – December 3rd, 2013

This week we watch, lint, memoize, error handle, say hello to and write more idiomatic Ruby.

8% of pull requests are doomed

July 24, 2018

Today we’ll look at three terminal pull request outcomes and one way to increase velocity in your engineering process.

Every pull request has one of three outcomes

Every pull request has costs: engineering labor, product management, and opportunity cost, to name a few. Each also has an outcome: merged, closed without merging, or abandoned due to inactivity.

Here’s a look at how pull requests fare across the industry:

If you group closed and inactive pull requests together (“Abandoned PRs”), you can estimate that the average engineer abandons 8% of the pull requests they create, which is equivalent to a loss of $24,000 per year¹, or the cost of a 2018 Toyota Camry Hybrid.

(We consider pull requests that have had zero activity for more than three days to be abandoned because our data shows a very low likelihood that PRs that go untouched for so long get merged later.)

Achieving zero abandoned pull requests is an anti-goal, as it would require being extremely conservative when opening them. However, a high rate of abandoned PRs can indicate inefficiency and opportunity for improvement within an engineering process. Reducing PR loss by 20% on a team with 10 engineers could save $48,000 per year.

How does my team stack up?

Using an anonymized, aggregated analysis of thousands of engineering contributors, we’re able to get an understanding of how an engineering organization compares to others in the industry:

This density plot shows that the average pull request loss rate across our dataset is 8% (with a median of 6%). A loss rate above 11% would be in the bottom quartile, and a loss rate below 3% would be upper quartile performance.

Improving pull request outcomes

Abandoned pull requests are, of course, a lagging indicator. You can tell because it would be ridiculous to go to an engineering team and say, “All those PRs that you’re closing… merge them instead!”

Potential drivers lie upstream: late changing product requirements, shifting business priorities, unclear architectural direction and good ole’ fashioned technical debt. If you have an issue with abandoned pull requests, soliciting qualitative feedback is a great next step. Talk to your team. Identify something that is impacting them and talk about how you might avoid it next time. Then, rather than focus on the absolute value of your starting point, you can monitor that your abandonment rate is going down over time.

After all, you’d probably rather not send a brand new Camry to the scrap yard every year.

¹ Assumes a fully loaded annual cost of $300k per developer.

[…]

The Ruby Rails Aggregator