CI/CD and deployment

For much of software history, releasing was an event. Teams worked for months, merged everything in a stressful “integration phase”, then deployed over a weekend and hoped. CI/CD is the set of practices that turned releasing from an event into a routine. The central insight is counterintuitive: if something is painful and risky, do it more often, in smaller pieces, until it stops being painful.

Continuous integration is a practice, not a server

Continuous integration means every developer merges their work into the shared mainline frequently, at least once a day, and each merge triggers an automated build and test run. If the build breaks, fixing it is the team’s top priority.

Many teams say they “do CI” because a server runs tests on every push. But if developers live on feature branches for weeks, they are not integrating continuously; they are postponing integration and building up a big, risky merge. The value of CI comes from keeping the gap between your code and everyone else’s code small, so conflicts are trivial and a failing test points at a change made an hour ago, not a month ago.

Trunk-based development

The branching model that supports this is trunk-based development: everyone commits to one main branch, directly or through short-lived branches that last hours or a day or two. Code review still happens, just on small changes.

The obvious objection is: what about features that take three weeks? The answer is to break them into small, safe steps and hide incomplete work, which is where feature flags come in.

Delivery versus deployment

These two terms are often blurred, but the distinction is simple:

• Continuous delivery: every change that passes the pipeline is releasable. Deploying to production is one button, and a human decides when to press it.
• Continuous deployment: every change that passes the pipeline goes to production automatically, with no manual gate.

Continuous delivery is the foundation; continuous deployment is a further choice that suits some products and not others. A regulated medical device and a web app have different constraints, and both can benefit from always being releasable.

The deployment pipeline

Humble and Farley’s deployment pipeline is the automated route from commit to production. A typical shape:

commit
  -> build artifact + fast unit tests      (minutes)
  -> integration / acceptance tests         (longer)
  -> deploy to staging, smoke tests
  -> deploy to production

Two principles matter most. First, fail fast: cheap, quick checks run first so most problems surface within minutes. Second, build once: the artifact that passed the tests is the exact artifact promoted to production, with only configuration changing between environments. Rebuilding per environment risks shipping something you never tested.

Feature flags: deploy is not release

A feature flag is a runtime switch around a code path. It separates two things that used to be the same: deploying code to servers and releasing a feature to users. Incomplete work can ship dark. A finished feature can be enabled for staff, then 5% of users, then everyone, and a misbehaving feature can be turned off in seconds without a redeploy.

Flags have a real cost. Every flag doubles the paths through that code, and forgotten flags accumulate into confusing, hard-to-test conditionals. Treat release flags as temporary and delete them once the rollout is complete.

Release strategies: limiting the blast radius

Even with good tests, production is the only true test environment. Release strategies make that less frightening.

Blue-green

Run two identical environments. Blue serves traffic; deploy the new version to green, check it, then switch the router so green takes traffic. Blue stays running, so rollback is just switching back. The difficulty is shared state: both versions must work with the same database during the switch.

Canary

Send the new version a small slice of traffic, say 1 to 5 percent, and compare its error rate and latency to the current version. If it looks healthy, widen gradually; if not, pull it. A bad change hurts a few users briefly instead of everyone. This depends on good monitoring, because a canary nobody watches is just a slow rollout.

Rollbacks and the database problem

The fastest fix for a bad release is usually to roll back to the last known-good version. That is easy for stateless code and hard for data. If a release dropped a column, the old version can’t run against the new schema.

The standard answer is the expand and contract approach: make schema changes backward-compatible in steps. Add the new column, deploy code that writes both, migrate data, switch reads, and only later remove the old column. Each step can be rolled back independently.

Measuring it: the four DORA metrics

The research published in Accelerate by Forsgren, Humble and Kim identified four key metrics that predict software delivery performance:

1. Deployment frequency: how often you release to production.
2. Lead time for changes: how long from commit to running in production.
3. Change failure rate: what share of deployments cause a failure needing remediation.
4. Time to restore service: how long it takes to recover when something breaks.

The first two measure throughput, the last two stability. Their most important finding was that these do not trade off. High performers were better on all four at once.

Small batches are the mechanism: a small change is easier to review, test, deploy, diagnose and undo.

One caution: these metrics describe a system, and they degrade quickly when turned into individual targets. Use them to find bottlenecks, not to rank teams.