Rolling out nreactive to a team

Adopting an automated fix pipeline is as much a social rollout as a technical one. A team that gets it wrong ends up with reviewers ignoring nreactive PRs, a dashboard full of noise, and nobody quite sure whether the automation is helping. This post is the rollout plan we've seen work — four weeks, low-risk, with trust-building baked in.

Week one: install the SDK, capture only

Install the SDK on one application, in staging if you have one, otherwise production behind a feature flag. The goal for week one is capture without action. Runtime errors flow into the nreactive dashboard but no fixes are generated yet.

Spend the week reading the dashboard. Is the fingerprinting sensible? Are the severity classifications what you'd expect? Is the deny list coverage right, or are there sensitive files you'd want to exclude from future analysis?

Tune the ignore-error patterns and the deny list based on what you see. At the end of the week, you should trust the capture layer — the data going in is clean.

Week two: enable analysis, keep auto-merge off

Flip analysis on. The pipeline now generates fix suggestions, creates branches, and opens PRs. Auto-merge stays off, so every PR goes through your normal review workflow.

Expect the first few PRs to be a mixed bag. Some will be clearly correct and trivial to merge. Some will be correct but the reviewer wants to rephrase the change. Some will be wrong and the reviewer closes them. All three outcomes are useful data.

Track the merge rate. If it's above two-thirds, the pipeline is producing useful PRs. Below a third, the deny list or the confidence threshold needs work. At the end of the week, tune whichever is weakest.

Week three: raise confidence threshold, enable scheduled scans

With two weeks of data, you have enough signal to make two decisions. First, raise the confidence threshold if you're seeing false positives at the low end. Errors below the new threshold will open tickets in Linear or Jira instead of PRs, which is often the right call for marginal cases.

Second, enable scheduled scans at a weekly cadence. Scan PRs have a different rhythm from error-driven PRs — they're opt-in improvements rather than reactive fixes — so give reviewers a heads-up about the format. The first scan PR on a long-lived repo is often longer than the next few because there's more accumulated signal to surface.

Week four: selective auto-merge

By week four you should have enough confidence to enable auto-merge selectively. Pick one or two categories where you've seen the pipeline produce reliable fixes — null guards and missing awaits are common starting points. Set the auto-merge threshold high (0.9 is the default; some teams go to 0.95) so only the obviously-correct ones qualify.

The 48-hour regression window catches problems quickly. If the first few auto-merged PRs regress, lower confidence or turn auto-merge off. If they don't regress, you're in the stable operating regime.

What to measure

Four metrics tell you whether the rollout is working:

• Merge rate. Percentage of generated PRs that get merged (manually or automatically). Aim for above two-thirds.
• Regression rate. Percentage of merged PRs that regress within 48 hours. Aim for under five percent.
• Dashboard noise. Subjective, but important. Does the team look at the dashboard voluntarily, or avoid it because it's overwhelming?
• Time to fix. How long from error captured to PR merged. Auto-merge collapses this to minutes; manual review makes it days. Both are fine — pick the trade-off that fits your team.

Common early mistakes

Three mistakes we've seen teams make:

• Enabling auto-merge in week one. No team has enough data to trust the pipeline with direct-to-main writes that fast.
• Leaving the deny list empty. Every repo has a few files the pipeline shouldn't touch; excluding them upfront prevents the awkward PR where the AI edits your generated client.
• Treating low-confidence errors as failures. They're not — they're tickets. A good low-confidence analysis surfaces a real issue without committing to a fix.

Communicating with the team

A short internal doc helps. Cover what nreactive does (automated fixes), what it doesn't do (bypass CI, override branch protection), how to handle an nreactive PR (review normally, use the confidence score as a scan aid), and what to do with a regression alert (investigate the PR, consider rollback, re-trigger fix if appropriate).

Clear expectations beat clever tools. A team that knows exactly how to handle an nreactive PR will use the tool well; a team that's unsure will underuse it.

Steady state

By the end of the first month, the pipeline should feel invisible. Errors appear, PRs open, CI runs, merges happen. Occasional regressions get caught and addressed. The dashboard is something you check when something feels off, not a constant source of work. That's the goal — automation as quiet infrastructure.