• Category Archives: featured

8% of pull requests are doomed

Today we’ll look at three terminal pull request outcomes and one way to increase velocity in
your engineering process.

Every pull request has one of three outcomes

Every pull request has costs: engineering labor, product management, and
opportunity cost, to name a few. Each also has an outcome: merged, closed
without merging, or abandoned due to inactivity.

Here’s a look at how pull requests fare across the industry:

If you group closed and inactive pull requests together (“Abandoned PRs”), you
can estimate that the average engineer abandons 8% of the pull requests they
create, which is equivalent to a loss of $24,000 per year1, or the cost of a
2018 Toyota Camry Hybrid

(We consider pull requests that have had zero activity for more than three days
to be abandoned because our data shows a very low likelihood that PRs that go
untouched for so long get merged later.)

Achieving zero abandoned pull requests is an anti-goal, as it would require being
extremely conservative when opening them. However, a high rate of abandoned PRs can
indicate inefficiency and opportunity for improvement within an engineering
process. Reducing PR loss by 20% on a team with 10 engineers could save $48,000
per year.

How does my team stack up?

Using an anonymized, aggregated analysis of thousands of engineering
contributors, we’re able to get an understanding of how an engineering
organization compares to others in the industry:

This density plot shows that the average pull request loss rate across our
dataset is 8% (with a median of 6%). A loss rate above 11% would be in the
bottom quartile, and a loss rate below 3% would be upper quartile performance.

Improving pull request outcomes

Abandoned pull requests are, of course, a lagging indicator. You can tell because it
would be ridiculous to go to an engineering team and say, “All those PRs that
you’re closing… merge them instead!”

Potential drivers lie upstream: late changing product requirements, shifting
business priorities, unclear architectural direction and good ole’ fashioned
technical debt. If you have an issue with abandoned pull requests, soliciting
qualitative feedback is a great next step. Talk to your team. Identify something
that is impacting them and talk about how you might avoid it next time. Then,
rather than focus on the absolute value of your starting point, you can monitor
that your abandonment rate is going down over time.

After all, you’d probably rather not send a brand new Camry to the scrap yard
every year.

1 Assumes a fully loaded annual cost of $300k per developer.

Read more at the source

Turning on the lights

Welcome to the first installment of Code Climate’s new “Data-Driven
Engineering” series. Since 2011, we’ve been helping thousands of engineering
organizations unlock their full potential. Recently, we’ve been distilling that
work into one unified theme: Data-Driven Engineering.

What’s Data-Driven Engineering?

Data-Driven Engineering applies quantitative data to improve processes, teams,
and code. Importantly, Data-Driven Engineering is not:

  • Ignoring qualitative data you don’t agree with
  • Replacing collaboration and conversations
  • Stack ranking or micromanaging developers

Why is this important?

Data-Driven Engineering offers significant advantages compared to
narrative-driven approaches. It allows you to get a full picture of your
engineering process, receive actionable feedback in real-time, and identify
opportunities for improvement through benchmarking. Most importantly,
quantitative data helps illuminate cognitive biases, of which there are many.

What can Data-Driven Engineering tell us?

After analyzing our anonymized, aggregated data set including thousands of
engineering organizations, the short answer is: a lot.

Over the coming weeks, we’ll explore unique and practical insights to help you
transform your organization. We’ll share industry benchmarks for critical
engineering velocity drivers to help our readers identify process improvement
opportunities. Here’s an example:

Pull requests merged per week (PR throughput) per contributor1

This plot shows that an average engineer merges 3.6 pull requests per week, and
a throughput above 5.2 PRs merged per week is in the upper quartile
of our
industry benchmark.

You might be thinking, “Why do some engineers merge almost 50% more than their
peers?”… and that’s exactly the type of questions Data-Driven Engineering can
help answer.

1 We included contributors who average 3+ coding days per week from
commit timestamps.

Read more at the source

Launching Today: Velocity

Data-driven insights to boost your engineering capacity

Today we’re sharing something big: Velocity by Code Climate, our first new product since 2011, is launching in open beta.

Velocity helps organizations increase their engineering capacity by identifying bottlenecks, improving day-to-day developer experience, and coaching teams with data-driven insights, not just anecdotes.

Velocity helps you answer questions like:

  • Which pull requests are high risk and why? (Find out right away, not days later.)
  • How does my team’s KPIs compare to industry averages? Where’s our biggest opportunity to improve?
  • Are our engineering process changes making a difference? (Looking at both quantity and quality of output.)
  • Where do our developers get held up? Do they spend more time waiting on code review or CI results?

Learn more about Velocity

Why launch a new product?

Velocity goes hand-in-hand with our code quality product to help us deliver on our ultimate mission: Superpowers for Engineering Teams. One of our early users noted:

“With Velocity, I’m able to take engineering conversations that previously hinged on gut feel and enrich them with concrete and quantifiable evidence. Now, when decisions are made, we can track their impact on the team based on agreed upon metrics.” – Andrew Fader, VP Engineering, Publicis

Get started today

We’d love to help you level up your engineering organization. Request a free trial and we’ll be in touch right away. As a special thank you for our early supporters, anyone who begins a free, 14-day trial before Friday, February 16th will get 20% off their first year.

Read more at the source

How Codecademy achieves rapid growth and maintainable code

We sat down with Jake Hiller, Head of Engineering at Codecademy, to find out how they use Code Climate to maintain their quality standards while rapidly growing their engineering team.

Codecademy Logo

Manhattan, NY
Ruby, JavaScript, SCSS
Since May 2013

Code Climate keeps our process for creating PRs really low-effort so we can quickly test ideas and ship sooner.

Why Code Climate

Like many rapidly developing teams, Codecademy was running into growing pains for both engineering onboarding and code review. They had tried using local analysis tools but found them cumbersome to integrate as development environments varied across the team.

With an engineering workflow centered around pull request reviews, and a desire to reduce friction in committing and testing code, they needed a solution that would optimize their pull request review process and enable new team members to quickly become productive.

Codecademy had been using Code Climate for their Ruby stack since 2013. When Head of Engineering, Jake Hiller, joined in early 2015, he saw an opportunity to alleviate their code review and onboarding issues by rolling it out to the whole team.

“We wanted to avoid anything that blocks engineers from committing and testing code. Other solutions that use pre-commit hooks are invasive to both experimentation and the creative process. Code Climate’s flexibility helps us maintain rules that are tailored to our team and codebase, while offering standard maintainability measurements. Plus it enables us to defer checks until code is ready to be reviewed, so we can quickly test ideas and ship sooner.”

“Code Climate helps us transfer knowledge to new engineers – like our coding standards, why we’ve made decisions over time, and why we’ve chosen certain structures and patterns.

Increased speed and quality

Since rolling out to the whole team, Hiller says Codecademy has seen an improvement in the quality of their code reviews and the ease with which new team members get up to speed.

“Code Climate helps us transfer knowledge to new engineers – like our coding standards, why we’ve made decisions over time, and why we’ve chosen certain structures and patterns. New engineers can look through the Code Climate issues in their PR, ask questions, and propose changes and suggestions to the team.

“It’s also increased the speed and quality of our pull request reviews. We’ve been able to spend more time discussing the important functional aspects of our code, and less time debating smaller issues. There are a lot of issues that can’t be fixed with an auto formatter, which is where Code Climate will always be really helpful for our team.”

About Codecademy

Codecademy was founded in 2011 as an immersive online platform for learning to code in a fun, interactive, and accessible way. They’ve helped 45 million people learn how to code, covering a wide variety of programming languages, frameworks, and larger topics like Data Analysis and Web Development. Their recently released Pro and Pro Intensive products provide users with more hands on support and practice material to help them learn the skills they need to find jobs.

Read more at the source