How QA Changes When the System Can Be Wrong Without Breaking

In traditional testing, a test usually had a clear outcome: the system worked, or it broke. AI failures are harder to catch because a wrong answer can still look perfectly fine. The system continues running, appearing fully functional. The test may even pass despite producing the wrong answer.

That is the fundamental shift in modern quality assurance. Now QA has to catch answers that look right and still miss the truth.

That puts LLM evaluation squarely in QA’s lane. Tools such as DeepEval address areas traditional tests missed: whether the answer is actually true. As AI changes software, it is also changing the tools and skills QA teams need.

TL;DR

AI changed the shape of failure. A request can complete successfully, return a polished answer, and still produce the wrong outcome. That means QA has to define what acceptable behavior means before release and measure the system against that standard. More of the routine maintenance around brittle tests can now be automated. The hard calls still belong to the team: deciding which failures matter and how much uncertainty is acceptable in production.

What Changes When the Same Input Can Produce a Different Answer

For the last thirty years, software testing has relied on one simple assumption: code is deterministic. You give the system the same input, and you get the same output. Traditional automation is built around repeatability, an assumption that breaks down with non-deterministic AI systems.

The same request can yield different valid responses because the model’s behavior is nondeterministic.

What an AI Failure Actually Looks Like

Consider an LLM handling customer support. If it makes a mistake, everything appears normal. The system remains stable. The answer sounds convincing and internally consistent. It is just wrong. From the system’s perspective, the request completed successfully.

That raises the stakes on a missed defect. The bigger risk is silent failure: the system keeps running while bad answers pile up. Sometimes customers catch the problem first. Sometimes journalists do.

Why AI Quality Became a Business Risk

The World Quality Report 2025–26 still lists model reliability as one of the first barriers boards cite. By late 2026, that translates into a concrete question: who owns the golden set and has the final say on what “good” means.

How NIST Gives Teams a Structure for AI Risk

NIST’s AI Risk Management Framework gives companies a practical basis for testing and managing AI risk. NIST’s framework can slot into existing compliance programs such as ISO 27001 and SOC 2 while providing teams with a structured approach to AI risk.

What Govern, Map, Measure, and Manage Look Like in Practice

Govern puts a name next to the risk. Governance requires team-wide ownership beyond just QA. Typical outputs include AI testing policies. Teams may also need dedicated budgets for validation tools.

Map shows where the model goes, what it touches, and who gets hurt if it goes wrong. It also shows who could be affected when the system fails. Cover the entire map to ensure full risk visibility.

The measure tells you whether the system is staying within the lines. And whether that behavior changes over time.

Manage is where identified risk turns into action. And how they monitor them in production.

The goal is to catch failures early and fail safely when one gets through.

How TEVV Changes the Way AI Systems Are Evaluated

TEVV is a structured AI evaluation framework, and the basic approach actually predates generative AI. In August 2026, NIST published the initial public draft of NIST AI 200-2, the TEVV-Athlon Framework. Comments are open through October 6, 2026. Treat it as a framework for defining Events, Tools, and Blocks — rather than a ratified standard. For a support bot, a Block can be factuality.

For a customer support chatbot, teams can group automated checks into evaluation blocks based on the risks they want to measure. Instead of a single pass/fail result, the evaluation produces scores across several risk categories.

Pass/fail has become insufficient. Teams must evaluate model behavior throughout the system’s life cycle and decide how much residual risk the business is willing to carry.

Red teaming changes the job from checking expected behavior to actively trying to break it.

AI erases much of the old boundary between QA and security by introducing threats such as prompt injection. A user might trick a support bot into revealing system instructions or bypassing business rules, so QA teams also need adversarial testing.

How Teams Use One Model to Evaluate Another

Here is where testing gets a little strange: using one LLM to evaluate another.

This approach is known as LLM-as-a-judge evaluation. The setup is simple: a second model evaluates the first model’s output against a scoring rubric. Once a system produces thousands of outputs per day, human review struggles to scale.

The risk: the judge can produce errors too. Worse, its errors can be systematic. Effective controls are necessary to prevent baking bias into the release gate.

Three Biases You Have to Account for in an LLM Judge

Position bias can cause an LLM judge to favor whichever answer appears first. Flip A and B and the score can move with them. It is similar to an examiner consistently favoring the first paper in a stack regardless of its quality.

Self-preference is another problem because a judge may favor outputs from its own model family. Letting a model grade its own family can put a thumb on the scale.

The third is verbosity bias, in which the judge tends to rate longer, more detailed answers as higher quality, even when they contain factual errors or filler. A judge may prefer a longer response even when a shorter one is more precise. The judge can mistake length for quality.

Teams lock the judge to a specific model version because provider updates can move the goalposts overnight. If the provider changes the model behind the scenes, the same outputs may suddenly receive different scores, causing CI failures. Your code remained fixed, while the goalposts shifted. The result looks like a flaky build, except the flake is in the referee, not the application.

Teams use tolerance ranges because the judge itself is inherently uncertain, so an exact cutoff can pretend the system is more precise than it really is.

Teams also use a fixed benchmark, often called a golden test set. A golden set gives the team a fixed yardstick. Putting a judge on every production request gets expensive fast because each judge call adds cost and latency. At millions of requests per day, an additional model call for every response quickly becomes expensive and adds latency.

How the Main AI Testing Tools Fit Into the Workflow

The LLM evaluation tool market is crowded, but most teams require only a subset of the available tools.

Each tool addresses a specific stage of the workflow, so teams usually combine tools for different stages:

• Before release (CI/CD): Teams can use DeepEval or Promptfoo to catch low-quality changes before they ship.
• RAG evaluation: RAGAS specializes in systems where models retrieve context from a knowledge base before answering.
• Production & tracing: Braintrust handles tracing multi-step agents on live traffic.

What the QA Role Looks Like Today

Self-healing addresses a different QA problem: keeping automated tests stable as the UI changes.

Anyone who has maintained UI automation knows the problem. Modern component frameworks often change the DOM as the interface evolves. A UI test may locate an element using a CSS selector, which means a small DOM change can break the test even when the feature itself still works. Tools like Applitools can reduce false failures by focusing on meaningful interface changes.

Brittle tests waste engineering time by breaking whenever the UI changes. In this case, AI helps solve a longstanding QA problem rather than creating a new one.

Where Self-Healing Actually Saves the Team Time

The potential savings are significant. Vendor and analyst notes still quote 40–45% less selector maintenance for large UI suites when self-healing is enabled. Use that only as a planning range. Measure your own: hours spent fixing locators each sprint. If the suite is API-first, the number will be smaller.

A self-healing system can recognize the same UI element even after its class name or DOM position changes. Without self-healing, an engineer must manually update the broken selector. 

How Self-Healing Can Work Without an LLM Call

Self-healing can work in several ways, and not all of them require an LLM.

A common approach is to use Playwright with an LLM to repair broken selectors and automatically rerun failed tests.

At scale, those extra model calls add high cost and latency across a regression suite with thousands of tests.

Many self-healing systems avoid model calls by matching elements using stable semantics rather than brittle CSS selectors.

In one e-commerce testing setup, the same approach worked across desktop browsers and on iPhone without an LLM call.

What the Healer Agent Can Handle on Its Own

Human review remains essential despite self-healing capabilities.

In one six-month deployment, the agent handled routine test repairs while testers focused on business logic.

That automation has defined boundaries: the agent repairs how the test finds the UI, but it lacks the ability to verify business rule accuracy. If the test still fails after several recovery attempts, the system stops digging and hands the case back to a person.

You can stub external integrations, but someone who knows the workflow still has to make the call on what to simulate and how those stubs should behave.

Aspect	2020 QA	2026 QA
Main Focus	Bug Hunting / Finding Defects	Quality Engineering / Risk Coverage & Prevention
Code Type	Deterministic Code	AI-driven / Non-deterministic
Tooling	Manual/Standard Automation	AI-Native (DeepEval, RAGAS, self-healing)
Key Metrics	Defect counts, pass rate	Golden-set score, judge agreement, hallucination rate on pinned cases, incident rate after release
Data & Security	Security handled largely outside QA	QA works more closely with security on AI-specific risks
Skills	Scripting, exploratory testing	Definition of acceptable behavior, judge rubric, red-team prompts, review of production traces

What Modern QA Is Really Responsible For

QA now manages systems capable of producing errors while appearing fully functional, so that pushes QA well beyond classic test execution. QA teams now evaluate model behavior while managing the automated systems used to test it. Good coverage now means automated evaluation with human judgment at the edges. A 2026 release gate for an LLM workflow is a business-approved golden set. HTTP 200 is the easy part. Deciding whether the answer deserves to ship is the real test.