Ship: The AI QA Agent That Tests Every Pull Request

Isometric illustration of a rocket merging into a git pull-request branch with a shield checkmark, representing an AI agent that tests every pull request

TL;DR: Ship is a separate, developer-led AI QA agent, not a module inside the ContextQA platform, that analyzes each pull request's blast radius, runs the regressions that PR actually needs before merge, and reproduces bugs reported in your issue tracker. Pricing runs from a 15-day free trial to $49 a month (Starter) to $249 a month (Pro), billed in credits at $0.025 each, with each PR analysis costing 10 credits.

Quick answers

Is Ship part of the ContextQA platform? No. Ship is a separate product built for developer-led teams, with its own pricing and its own GitHub integration.

What triggers Ship to analyze a pull request? Deployment events and PR activity. Ship's product page describes an automatic, illustrative workflow rather than a manual test plan you configure and trigger yourself.

How much does one PR analysis cost? 10 credits, at $0.025 per credit, so $0.25 per PR analyzed, within whatever monthly credit allotment your plan includes.

What Ship actually does

Ship's job starts before a human reviewer even opens the diff. It analyzes a pull request, understands its blast radius, and identifies what needs testing, then runs the relevant regressions before merge. Ship's own product page labels this an illustrative workflow, a demonstration rather than a documented, step by step guarantee of exactly how the trigger fires.

When a bug shows up in an issue tracker instead, Ship reproduces it, captures evidence, and investigates the likely root cause, the same analyze-and-report pattern applied to a report instead of a diff.

GitHub is the primary integration. Ship is built to keep developers working in GitHub rather than a separate tool, and teams can choose any PR to analyze on demand, with promotion PRs such as a dev to QA merge given as a typical example.

Pricing

PlanPriceCreditsGitHub reposParallel runs
Free trial$0 for 15 days3,000 one-time12
Starter$49/month2,000/month15
Pro$249/month12,000/monthMultiple10
EnterpriseCustomVolume by contractUnlimitedCustom
Source: ship.contextqa.com pricing page
Developers reviewing and optimizing code together before merging a pull request

Where Ship fits versus wiring your own CI gate

If you'd rather configure your own trigger and plan inside GitHub Actions, see ContextQA's guide to running automated tests on every pull request with GitHub Actions, which calls the main ContextQA platform's REST API with a token you control. Ship takes a different approach: it's an agent that decides what to test on its own, rather than running a plan you configured, and it reproduces bugs from your issue tracker in addition to testing pull requests.

Honest limitations

The workflow is illustrative, not a documented spec. Ship's own product page labels its PR-analysis demo an illustrative workflow, so the exact trigger mechanism and timing aren't published as a guaranteed, step by step process.

Mobile testing isn't live yet. Real-device mobile testing is listed as coming soon on the Pro plan.

Repo limits bite early. The free trial and Starter plan cap out at 1 GitHub repository, so a team running several repos needs Pro or Enterprise from the start, not as a later upgrade.

Do this now

  1. Start the 15-day free trial and connect 1 GitHub repository.
  2. Let Ship analyze your next real pull request instead of a throwaway test PR, so the blast-radius analysis has something real to work with.
  3. Review the regression results and the bug-reproduction report together, not just the pass or fail status.
  4. Count how many credits a typical week of PRs uses, then compare that to Starter's 2,000 a month before you commit.
  5. If you run more than 1 repository, budget for Pro from day one instead of hitting the Starter limit mid-trial.

The bottom line

Ship's pitch is that you stop deciding what to test on every PR and let an agent that already read the diff decide instead. That only holds up if it actually catches what a human reviewer would have caught, so start on the free trial, point it at your real pull requests, and judge it on the reports it hands back before paying for a seat. Start the Ship free trial.

Written by Deep Barot.

Share the Post:

Author

Deep Barot

CEO @ ContextQA | Agentic AI for Software Testing | Context-aware Testing

Deep Barot is the Founder and CEO of ContextQA, the only AI testing platform that understands context. He brings decades of experience across DevOps, full-stack engineering, cloud systems, and large-scale platform development.
AI Insights
Real User Intelligence Platform

Turn live sessions into test coverage. No prompts, no manual design - just pointed at your URL and generating suites within minutes.

Minutes
From URL to generated test cases
Zero
Prompts or manual test design needed
40%+
Average coverage increase after first run
100%
Based on real user behavior, not guesses

Frequently Asked Questions

No. Ship is a separate product built for developer-led teams, with its own pricing and its own GitHub integration, not a module inside the main ContextQA platform.
Ship analyzes the PR's blast radius and identifies what needs testing itself, rather than running a test plan you configure. Ship's own product page labels this an illustrative workflow, so treat the exact trigger mechanics as a demonstration, not a documented guarantee.
A 15-day free trial with 3,000 one-time credits, then Starter at $49 a month (2,000 credits), Pro at $249 a month (12,000 credits), or custom Enterprise pricing. Credits run $0.025 each, and each PR analysis costs 10 credits.
The free trial and Starter plan are limited to 1 GitHub repository. Pro supports multiple repositories, and Enterprise supports unlimited repositories.
Real-device mobile testing is listed as coming soon on the Pro plan, so it isn't available yet.
Related Blogs