TL;DR: Old automated tests get more expensive to maintain every year. Frameworks grow complex, small UI changes break many tests, and the people who wrote them move on. You don't have to fix everything or replace everything. Sort your tests instead: keep the stable, important ones, rebuild the ones that keep breaking, delete the ones you no longer need, and move tests off frameworks that are no longer supported. Many teams face this: 64% of organizations say legacy systems hold back their test automation, according to Capgemini's World Quality Report 2024.
Test automation is supposed to save your team time. But after a few years, many teams spend more time fixing old tests than writing new ones. When that happens, the question changes. It's no longer just "how do we fix this test?" It's "is this test worth fixing at all?" Our customers ask us this often. Here is the simple framework we use to help them decide.
Quick answers
Should I fix or rebuild a failing legacy test?
Fix it if it tests something important, runs on a framework that is still supported, and can be repaired quickly. Rebuild it if writing a new test would take less time than figuring out the old one. Delete it if it tests a feature nobody uses anymore.
How do I compare the cost of maintaining versus rebuilding?
Don't just look at how long today's fix will take. Estimate the total cost over the next 12 to 24 months. For keeping the test, count repair time, framework upgrades, infrastructure, and time lost to unreliable results. For rebuilding, count the time to create, check, and run the new test. Then pick the cheaper option.
Does AI make rebuilding automated tests cheaper?
Partly. AI tools can create tests by exploring your application and can suggest fixes for broken ones. That makes rebuilding faster. But a test created by AI is not automatically a good test. It still needs real checks, good test data, and a place in your CI/CD pipeline.
What is the test automation maintenance trap?
The maintenance trap is when the tests that were meant to save your team time start costing it time instead. It rarely happens on purpose. Most teams start with a few simple automated tests, and the test suite grows as the product grows.
Every new feature adds new tests. Existing tests need updates. The framework gets more complex to handle new needs. After five or ten years, a company can end up with thousands of tests written in different frameworks and languages.
Then problems start to show. Tests fail after small UI changes. Locators break. Libraries go out of date. Upgrading the framework means rewriting large parts of it. Sometimes nobody even remembers why a test was written. At this point, the real question is not how to keep these tests running. It's whether keeping them running is still worth the cost.
Why does legacy test automation become expensive?
There are four main reasons, and each one makes the others worse.
1. Framework complexity accumulates
Over time, teams add their own libraries, helper functions, and extra layers on top of the framework. A setup that started simple becomes so complex that only a few people know how to work on it.
2. Application changes trigger cascading failures
One small change to the user interface can break dozens of tests at once. This happens most when many tests depend on the same fragile locators or shared code. It also hurts trust. As Martin Fowler explains in Eradicating Non-Determinism in Tests, when tests fail at random, teams stop believing the results. Our flaky test playbook covers how to diagnose these failures at the root.
3. Test ownership disappears
The engineers who wrote the tests may have moved to other teams or left the company. New engineers inherit tests they don't fully understand, so every fix starts with working out what the test was supposed to do.
4. Maintenance competes with new coverage
Every hour spent fixing old tests is an hour not spent testing new features, adding coverage, or finding real bugs. Over time, the team ends up looking after the test setup instead of improving the product.
When should you keep maintaining existing tests?
Keep a legacy test if it checks something important that rarely changes, and fixing it costs less than replacing it. Old test suites often contain years of knowledge about how your product works, including complex business cases that would be hard to recreate. Keeping a test is usually the right choice when:
- Tests cover important, stable business functionality.
- The framework is still supported and easy enough to work with.
- Failures can be repaired quickly and reliably.
- Rebuilding would cost more than keeping it running.
The hard part is telling the difference between tests that are truly valuable and tests you keep only because they already exist.
When does rebuilding with modern test automation make sense?
Rebuild a test when writing a new one is faster than understanding and fixing the old one, and the new test will be easier to look after. Modern tools, especially AI-powered ones, have cut the time it takes to create a test. Tasks that once meant writing scripts and locators by hand can now be done by describing the steps in plain English or letting the tool explore your application.
Imagine a checkout test written six years ago in a heavily customized framework. It breaks after an app update. An engineer might spend hours tracing the code, updating locators, and debugging the framework. A modern tool might recreate the same test in much less time.
But there's a catch: generating a test isn't the same as creating a reliable test. A new test still needs to be checked. It needs real assertions, good test data, and a place in your CI/CD pipeline. Rebuilding only makes sense if the new test is clearly easier to maintain and covers at least as much as the old one.
Not sure whether to fix or rebuild? Plug your team's numbers into the QA ROI calculator to see what test maintenance costs your team today, and how much you could save by spending less time on repairs.
What is a hybrid test modernization strategy?
A hybrid approach means you don't fix everything and you don't replace everything. You decide test by test. Start by reviewing your test suite and grouping each test by how important it is, how reliable it is, and how much it costs to maintain. Then take the matching action:
- Stable tests covering critical workflows: keep them and maintain them.
- Important tests that break often: improve them or rebuild them.
- Tests for features you no longer use: delete them.
- Tests tied to unsupported frameworks: move them first. Our UFT migration guide and Selenium to Playwright migration guide cover two of the most common moves.
- Duplicate or redundant tests: merge them.
- Features with no tests yet: write new tests with modern tools.
This way you keep the tests that still add value and stop paying to maintain the ones that don't. For more on keeping a test suite healthy, see our guide to automated regression testing best practices.
How does AI change the test maintenance equation?
Rebuilding a test suite used to be a huge project. Recreating thousands of tests by hand was often too slow and expensive, even for teams that wanted to modernize. AI tools change that. They can help teams:
- Create tests from how the application behaves, from requirements, or from existing test cases.
- Identify which tests are affected by an application change.
- Analyze failures and suggest repairs, for example with AI root cause analysis.
- Rebuild outdated tests with modern tools.
- Find duplicate tests and missing coverage, for example with a visual coverage map.
The bigger benefit goes beyond creating tests faster. Tests can now adapt as the application changes, which is the idea behind self-healing test automation. Instead of fixing every broken test by hand, teams can decide for each one whether to repair it, recreate it, or delete it. Test maintenance becomes a regular review, not a constant emergency.
Some teams go a step further and hand the ongoing work to an AI QA agent. Ship, ContextQA's autonomous QA engineer, maintains the tests you already have, adds coverage where it is missing and investigates failures before an engineer has to.
How do you decide whether to fix or rebuild?
Don't decide based on how long today's fix will take. Look at the total cost over the next 12 to 24 months for both options.
Cost of maintaining
- Time spent investigating and fixing failures
- Framework upgrades and dependency updates
- Infrastructure and execution costs
- Engineering time lost to unreliable test results
Cost of rebuilding
- Time to recreate and validate tests
- Migration and integration effort
- Training and adoption costs
- Risk of losing existing business coverage
- Expected future maintenance requirements
Then apply a simple rule:
- Maintain if the old test will cost less to keep running than building and running a new one.
- Rebuild if building and running a new test costs less, and it still covers everything the old test did.
In other words, this is a business decision, not just a technical one. We use the same approach in our breakdown of the ROI of test automation.
Is a bigger test suite a healthier one?
No. For years, teams measured automation success by the number of tests or the percentage of features covered. But more tests don't always mean better testing. Google's testing team made a similar point in Just Say No to More End-to-End Tests: adding more and more end-to-end tests often makes feedback slower and less reliable, not more useful.
A smaller set of reliable tests that adapt to changes can be worth more than thousands of fragile ones. Now that AI makes it faster and cheaper to create tests, teams can let go of an old habit: keeping every test just because it already exists.
The goal isn't to protect your test code. The goal is to stay confident that your software works. Sometimes the best way to get there is to start fresh.
Spend less time fixing old tests. The ContextQA AI testing platform heals tests automatically when the UI changes, explains why tests fail, and lets you rebuild broken tests by describing the steps in plain English. Book a demo and bring one of your problem tests.
What about your team?
When an old automated test fails, what does your team usually do? Fix it, rebuild it with a modern tool, or ask whether you still need it? If you'd like a second opinion on your test suite, talk to our team.
