TL;DR: Autonomous software delivery levels describe how much of the coding, pull request and bug fix loop an AI agent owns. This four level model is ContextQA's own framing, not an industry standard: assisted coding, agent written PRs, agent fixed bugs with verification, and a governed factory. Each level needs stronger verification. The 2024 DORA report linked a 25% rise in AI adoption to an estimated 7.2% drop in delivery stability (DORA, 2024). Ship, the ContextQA autonomous quality engineer, tests every pull request before merge.
Most teams handing work to coding agents can't name their current level or the checks they need to move up. This guide covers the verification, gates and human review each level needs, and what to change on Monday.
Quick answers
What verification does each level need before a team moves up? Level one, assisted coding, needs normal code review and CI tests. Level two, agent written pull requests, needs regression checks before merge plus a human reviewer. Level three, agent fixed bugs, needs a reproduction and passing evidence. Level four, a governed factory, needs least privilege credentials, audit trails and build gates that block the release.
How do I tell which level my team is at today? Look at who writes the change and who checks it. If developers write code with AI suggestions, you are at level one. If agents open pull requests that humans review, you are at level two. If agents fix reported bugs and each fix is checked against a reproduction, you are at level three. At level four, policy, permissions and audit evidence control the whole flow.
Where does Ship fit for a smaller product team? Ship, ContextQA's self serve autonomous quality engineer, is the verification layer for levels two and three. It tests every pull request with impact analysis and targeted regressions before merge. It also reproduces bugs reported in Jira, Linear, Slack or a service desk, then gives engineers the likely root cause and the evidence (Ship docs).
What are autonomous software delivery levels?
Autonomous software delivery levels are a maturity scale. They describe how much of the path from ticket to production an AI agent owns, and how much verification and human oversight each step needs. ContextQA created this framing, and it is not an industry standard. It helps engineering leaders decide what to hand to agents next.
Three questions define each level: who writes the change, who proves it works, and who approves the merge.
- L0, manual: Engineers write the code and tests, and humans run regression.
- L1, assisted coding: An IDE assistant suggests code, and the engineer owns every line.
- L2, agent written PRs: Agents open pull requests. Each one needs passing tests and a human review.
- L3, agent fixed bugs with verification: Agents reproduce a reported bug, fix it, and attach evidence that the fix holds.
- L4, governed factory: Agents work in parallel under policy, with scoped credentials, audit trails and build gates.
- L5, fully autonomous: No human is in the loop. Treat it as a reference point, not a target.
The shape loosely borrows from the six SAE J3016 (2021) driving automation levels, which run from no automation to full automation. This guide recommends L1 to L4 for software delivery, and at each of those levels a human keeps owning the release decision.
What can a delivery maturity model do for your team?
A delivery maturity model turns a vague debate about AI agents into specific decisions. It ties each grant of autonomy to a verification requirement, so leaders can approve agent work one workflow at a time instead of going on gut feel. It also gives QA, security and engineering one shared vocabulary for scoping permissions and gates.
- Scope agent permissions: Match repository access, credentials and environments to the level a workflow has earned.
- Set merge gates: Decide which checks must pass, and who must sign off, before agent code reaches the main branch.
- Plan review capacity: Move reviewer time away from reading every diff and toward checking behavior and evidence.
- Answer audit questions: Show who approved each change and what evidence they saw.
Why do these levels matter in 2026?
They matter because teams adopted AI coding faster than they built ways to check its output. Most developers already use AI tools, but many still distrust what those tools produce. A shared ladder lets engineering leaders decide how much autonomy each workflow gets. It also sets what testing and review that autonomy needs before anything merges.
- Adoption: 84% of respondents to the Stack Overflow 2025 Developer Survey use or plan to use AI tools in development.
- Trust: Many respondents to the Stack Overflow 2025 Developer Survey distrust the accuracy of AI output.
- Workplace use: 90% of respondents to the DORA 2025 State of AI assisted Software Development report use AI at work.
The terms differ in who checks the work. AI assisted: a human writes and merges code with AI suggestions. Agentic: an agent opens pull requests or fixes bugs, and humans review each change. Fully autonomous: agents ship through governed pipelines, with automated verification as the main gate.
How does each level work in practice?
An agent gets more responsibility at each level only after the team can check its output without relying on the agent's own report. In practice, every step up adds a stronger merge gate, more independent evidence and a clearer human owner. The list below gives the verification and oversight each level needs under ContextQA's four level framing.
- Assisted coding. Engineers accept or reject AI suggestions in the IDE. Verification: the existing code review and CI tests. Oversight: a human reads every changed line.
- Agent written PRs. An agent opens the pull request. Verification: impact analysis and targeted regression tests before merge, which Ship runs on every pull request. Oversight: a reviewer checks behavior and test evidence instead of reading every line.
- Agent fixed bugs with verification. A coding agent proposes the fix. Verification: the reported bug is reproduced first, and the reproduction must fail before the fix and pass after it. Oversight: a human reviews the likely root cause and the evidence before approving the merge.
- Governed factory. Agents work across the backlog continuously. Verification: build gates that stop the release on every result that is not a success. Oversight: least privilege test identities, isolated environments, audit trails and a named human approver for deployment, as covered in Ship's security and controls.
Which level fits your team best?
Pick the level where your team already verifies every change, not the level a vendor advertises. The right level depends on three things: how much regression coverage you have today, who reviews agent output, and how costly an escaped defect is. If you move up without the matching checks, you swap review time for production incidents.
- Thin regression coverage: Stay at level one. Agent written PRs will arrive faster than you can prove they are safe.
- Reliable CI and an owner for test results: Level two is realistic once every PR gets targeted regressions before merge.
- A steady bug backlog in Jira, Linear or Slack: Level three pays off when every agent fix is checked against a reproduction of the original report.
- Many repositories and compliance obligations: Level four only makes sense with scoped credentials, approved repository lists and reviewable evidence for each change.
Rule of thumb: move up one level only after the current level's verification runs on every change and a named owner reviews the evidence.
How does ContextQA support each level?
ContextQA supports these levels through Ship, its self serve autonomous quality engineer. Ship adds verification at the point where agent output enters the codebase. It tests every pull request with impact analysis and targeted regressions before merge. It also reproduces bugs reported in Jira, Linear, Slack or a service desk.
- Agent written PRs: Every PR gets impact analysis and a targeted regression run before merge, whether a human or an agent wrote it.
- Agent fixed bugs with verification: Ship reproduces the reported bug first and gives engineers the likely root cause and the evidence. The fix is judged on that evidence, not on the agent's claim that it worked.
- Governed factory: Humans still own merge, release and scope decisions.
Governance starts at setup. The Ship onboarding guide recommends a staging or QA environment, disposable data and a least privilege test identity, because the crawler can create, update or delete records. Connect your coding agent through MCP (Model Context Protocol) only after those limits are in place.
If agent written pull requests reach your main branch without verification, see how Ship tests every PR and reproduces reported bugs: Get started free with Ship, no credit card needed.
What should you do right now? A checklist
Start by naming the level your team actually works at today. Then add the checks that level needs before you let agents do more. Under ContextQA's framing, each step up needs a merge gate, an evidence trail and a named human owner. A team can confirm every item below this week.
- Record your current level and the evidence for it.
- List every repository where agents open pull requests.
- Use branch protection to require a human reviewer on every pull request an agent writes.
- Run targeted regression tests on each pull request before merge, chosen by impact analysis (Ship).
- Send bugs reported in Jira, Linear or Slack to reproduction first, and reject any agent fix until the reproduction passes.
- Run evaluation crawls against staging with throwaway test data, not production.
- Give the test identity only the permissions it needs, and connect only approved repositories (security and controls).
- Name one owner to review generated coverage and evidence every week.
Bottom line
Treat the four levels as ContextQA's planning model, not an industry standard. Each step from assisted coding to a governed factory needs more verification and oversight. Before raising autonomy, give every agent change evidence a reviewer can check, a gate that stops the release on failure, and a named owner. New to this? Read what an autonomous quality engineer does and the basics of test automation.
Sources
These are the primary sources behind the statistics and comparisons in this article, plus the ContextQA documentation pages used to describe each Ship workflow and setting.
