Self-healing tests: what works, what is marketing
A self-healing test is a good idea with a dangerous default. What works: when a test fails on a page that changed, a tool proposes one small change to the test, proves the changed test on a fresh session, and waits for a person to approve it. What is marketing: a test that quietly picks a new element and goes green, so a real regression and a renamed button look the same.
The failure self-healing is meant to fix
Most failures after a deliberate change are not bugs. A label was renamed, a button moved into a menu, a step was added to a flow. The product is fine; the test is out of date. Fixing each one by hand is the maintenance cost that kills suites, and it is the cost self-healing promises to remove.
Where the quiet version goes wrong
Say a test clicks "Place order". After a change the button is gone and a "Continue" button sits where it was. A quiet healer finds the nearest match, clicks "Continue", and the test passes. Was the order placed? Nobody knows, and the green says yes.
The same mechanism hides real regressions. If a required field disappears and the healer skips the step that filled it, the test passes on a page that no longer collects the field. A suite that can heal its way past a bug is a suite you cannot trust, and a suite you cannot trust is not worth running.
What a repair should do
One small change. A new target for one step, a new value, a corrected assertion, or a new path to the same outcome. Not a rewrite.
Proven before proposed. The repaired test replays on a fresh session, more than once, and only a repair that is green each time is shown. A repair that works once is a coincidence.
Shown with its reason. The step before and after, and why the model chose the change. "The button labelled Place order is now labelled Continue inside the same form" is a reason a person can check in five seconds.
Approved by a person. Nothing changes until someone says yes. Rejection leaves the test as it was. The approval is the moment a human confirms the product changed on purpose.
Declined when no repair fits. If the outcome the test asserts is simply gone, the honest answer is that the test cannot be repaired, and that is a finding about the product, not the test.
A worked example
A composed example: the team, the timings and the counts show the shape of the work and are not measurements from one customer.
A team renames "Save" to "Save changes" across their settings pages. Fourteen tests fail that night. Their tool proposes fourteen repairs, each the same shape: the click target changes from the old label to the new one, proven twice on fresh sessions. A reviewer opens the list, sees the same reason fourteen times, approves them in two minutes.
A week later one test fails again on the same page. This time the tool declines: the assertion "the saved banner appears" cannot be satisfied because no banner appears at all. That one is a bug, filed with the screenshot, and it would have been invisible under a quiet healer.
The question to ask a vendor
"Show me a repair being refused." If every failure can be healed, the tool is not testing anything.
FlowQA explores your site, writes the tests, proves them against a broken page, and keeps proving them after every change.
See FlowQA