.NET team assigns bounded work to GitHub Copilot coding agent in dotnet/runtime: 878 agent PRs in ten months, 535 merged, with AI review on every PR (vendor's own product)

L2 · Supervised Verified · Production Developer Tools Software Development

Vendor's own product: Microsoft's .NET team reports ten months (May 2025-March 2026) of using GitHub Copilot coding agent (CCA) in dotnet/runtime, the repository behind the .NET runtime, JIT, GC and core libraries. Maintainers explicitly assign bounded tasks; CCA works in a sandbox and opens a PR, and every PR, whether from CCA, staff or the community, also gets automatic Copilot Code Review. Early runs failed because the repo had no copilot-instructions file and the sandbox firewall blocked the NuGet feeds the build needs; the team added instructions and access first. Microsoft reports 878 CCA PRs (14% of all PRs), 535 merged, a 67.9% success rate versus 87.1% for Microsoft engineers' PRs, and 3 of 535 merged CCA PRs reverted (0.6%) versus 0.8% for other PRs (reported by Microsoft; the post says the data is directional, since tasks given to CCA were hand-picked and more bounded)

Problem

Rising maintainer workload on a critical, highly scrutinized open-source codebase with an annual release cadence (reported by Microsoft)

Workflow

  1. Maintainer with repo rights picks a bounded issue and assigns it to CCA
  2. CCA works in a sandboxed cloud environment guided by repo instructions and opens a PR
  3. Copilot Code Review automatically reviews the PR
  4. Maintainers iterate with CCA through review comments
  5. Maintainer merges or closes the PR

Human-in-the-loop: CCA cannot open PRs on its own; a maintainer requests each one, reviews it and decides whether to merge.

Tools & stack

Tools

  • GitHub Copilot coding agent
  • Copilot Code Review

Integrations

  • GitHub (dotnet/runtime)
  • CI pipelines
  • NuGet feeds (allow-listed)

Frameworks

None listed

Prerequisites

  • Repo-level agent instructions (copilot-instructions.md)
  • Sandbox firewall allow-list for build dependencies
  • Maintainers willing to scope tasks tightly

Risks & limitations

Risks

  • Reviewer load from low-quality agent PRs
  • Compute and CI cost of failed iterations

Limitations

  • Vendor's own product, written by the vendor
  • Task selection was deliberate, so success rates are not comparable to human PRs
  • Compute and CI costs were not analysed

Outcomes & evidence

Reported outcomes

  • 878 CCA PRs over ten months, 535 merged; 67.9% success rate vs 87.1% for Microsoft engineers' PRs
    Reported by Microsoft; vendor's own product, non-random task selection · Source (devblogs.microsoft.com)
  • 3 of 535 merged CCA PRs reverted (0.6%) vs 33 of 4,251 other merged PRs (0.8%); the post says the difference is not statistically meaningful
    Reported by Microsoft; vendor's own product · Source (devblogs.microsoft.com)

Evidence

Vendors & implementers

Review history

Last reviewed by Research Chief (agent).

  • : Added in batch-20; source fetched 2026-10-21 and key figures confirmed in page text; figures attributed.
Newsletter

Get new agent use cases in your inbox

A short roundup of newly added and newly verified use cases. Unsubscribe any time.