• Category Archives: data-driven-engineering

8% of pull requests are doomed

Today we’ll look at three terminal pull request outcomes and one way to increase velocity in
your engineering process.

Every pull request has one of three outcomes

Every pull request has costs: engineering labor, product management, and
opportunity cost, to name a few. Each also has an outcome: merged, closed
without merging, or abandoned due to inactivity.

Here’s a look at how pull requests fare across the industry:

If you group closed and inactive pull requests together (“Abandoned PRs”), you
can estimate that the average engineer abandons 8% of the pull requests they
create, which is equivalent to a loss of $24,000 per year1, or the cost of a
2018 Toyota Camry Hybrid
.

(We consider pull requests that have had zero activity for more than three days
to be abandoned because our data shows a very low likelihood that PRs that go
untouched for so long get merged later.)

Achieving zero abandoned pull requests is an anti-goal, as it would require being
extremely conservative when opening them. However, a high rate of abandoned PRs can
indicate inefficiency and opportunity for improvement within an engineering
process. Reducing PR loss by 20% on a team with 10 engineers could save $48,000
per year.

How does my team stack up?

Using an anonymized, aggregated analysis of thousands of engineering
contributors, we’re able to get an understanding of how an engineering
organization compares to others in the industry:

This density plot shows that the average pull request loss rate across our
dataset is 8% (with a median of 6%). A loss rate above 11% would be in the
bottom quartile, and a loss rate below 3% would be upper quartile performance.

Improving pull request outcomes

Abandoned pull requests are, of course, a lagging indicator. You can tell because it
would be ridiculous to go to an engineering team and say, “All those PRs that
you’re closing… merge them instead!”

Potential drivers lie upstream: late changing product requirements, shifting
business priorities, unclear architectural direction and good ole’ fashioned
technical debt. If you have an issue with abandoned pull requests, soliciting
qualitative feedback is a great next step. Talk to your team. Identify something
that is impacting them and talk about how you might avoid it next time. Then,
rather than focus on the absolute value of your starting point, you can monitor
that your abandonment rate is going down over time.

After all, you’d probably rather not send a brand new Camry to the scrap yard
every year.

1 Assumes a fully loaded annual cost of $300k per developer.

Read more at the source

Velocity is out of beta

Our mission at Code Climate is to help engineering organizations improve their processes, teams and code. We see a future where everyone from individual developers up to the CTO has access to a full picture of their engineering work in the form of clear, timely and actionable quantitative data.

In February, we opened our Velocity public beta. Over the past five months, we’ve spoken with hundreds of engineering leaders, processed a nearly-overwhelming amount of product feedback, and added dozens of top-requested features.

We’ve been floored by the excitement from engineering leaders:

“If you haven’t tried @codeclimate’s new Velocity product, and you’re interested in non-vanity measurements of productivity, and a baseline from which to measure process improvements, try it now. It’s very exciting.”

– Avi Flombaum, Dean and Chief Product Officer, Flatiron School

“Velocity is quickly becoming one of my favorite tools for engineering management.”

– Tomas Becklin, VP of Engineering, DroneBase

Today, Velocity is launching out of beta, and we’re ready to help your engineering organization turn on the lights.

Click here to book a Velocity demo today.

Everyone who books a demo before Thursday, July 26th will receive our introductory launch pricing of 20% off for life. This is a one-time offer that we won’t be repeating anytime soon.

Still on the fence? Keep reading.

Most engineering decisions are anecdote-driven

Today, engineering organizations are often forced to make decisions based solely on anecdotes, gut feel and incomplete information. We understand that qualitative information is highly valuable – there’s no substitute for experience and intuition. However, the lack of quantitative data within engineering processes is a missed opportunity, especially given how data has transformed DevOps.

Historically, engineering organizations looking to incorporate data into their processes have faced two problems.

First, unless they’re working within a behemoth like Google, there simply aren’t enough developer resources to spare to invest in such efforts. This is the problem of “The Cobbler’s children having no shoes,” as analytics has transformed so many departments like sales, marketing, and finance.

Second, even if metrics were available, they would be hard to interpret. After all, if someone told you that your team averages 1.9 review cycles per pull request, is that the best as you could reasonably aim for or an opportunity for improvement?

Get data-driven with Velocity

Velocity helps you unlock the full potential of your engineering organization with data-driven insights to manage risks, eliminate bottlenecks, and drive continuous improvement.

It’s built on one simple notion: The happiest developers work on the most productive teams, and vice versa. Applying these practices, which we call Data-Driven Engineering, puts you in the position to achieve both.

Velocity gives you:

  • Custom dashboards and trends – engineering metrics with full historical trends
  • Team insights – actionable data to level up your engineering teams
  • Industry benchmarks – high impact opportunities for improvement by comparing your metrics against other engineering teams
  • Real-time risk alerts – identify and resolve risks before they become problems.

As a software company ourselves, we’re committed to improving the process of engineering, for everyone involved: developers, product managers, executives, and more. Velocity is a core part of our foundation to pursue this goal. If you’re excited about this prospect as well, check out Velocity today:

Click here to book a Velocity demo today.Act by July 26th and get 20% off for life.

It takes 10 minutes to set up by connecting to your GitHub (or GitHub Enterprise) account, and soon you’ll have dozens of reports (with full historical data) to easily identify risks and opportunities.

Onward, to a data-driven future!

-Bryan, Noah and the entire Code Climate team

Read more at the source

Turning on the lights

Welcome to the first installment of Code Climate’s new “Data-Driven
Engineering” series. Since 2011, we’ve been helping thousands of engineering
organizations unlock their full potential. Recently, we’ve been distilling that
work into one unified theme: Data-Driven Engineering.

What’s Data-Driven Engineering?

Data-Driven Engineering applies quantitative data to improve processes, teams,
and code. Importantly, Data-Driven Engineering is not:

  • Ignoring qualitative data you don’t agree with
  • Replacing collaboration and conversations
  • Stack ranking or micromanaging developers

Why is this important?

Data-Driven Engineering offers significant advantages compared to
narrative-driven approaches. It allows you to get a full picture of your
engineering process, receive actionable feedback in real-time, and identify
opportunities for improvement through benchmarking. Most importantly,
quantitative data helps illuminate cognitive biases, of which there are many.

What can Data-Driven Engineering tell us?

After analyzing our anonymized, aggregated data set including thousands of
engineering organizations, the short answer is: a lot.

Over the coming weeks, we’ll explore unique and practical insights to help you
transform your organization. We’ll share industry benchmarks for critical
engineering velocity drivers to help our readers identify process improvement
opportunities. Here’s an example:

Pull requests merged per week (PR throughput) per contributor1

This plot shows that an average engineer merges 3.6 pull requests per week, and
a throughput above 5.2 PRs merged per week is in the upper quartile
of our
industry benchmark.

You might be thinking, “Why do some engineers merge almost 50% more than their
peers?”… and that’s exactly the type of questions Data-Driven Engineering can
help answer.

1 We included contributors who average 3+ coding days per week from
commit timestamps.

Read more at the source
close