26 Oct 2026 Continuous testing: which tests belong at which step of your CI/CD pipeline

In short: Continuous testing means automated tests run at every step of your CI/CD pipeline, so each change gets feedback within minutes. Fast checks run on every commit, broader tests on every merge and the full regression set every night. Each step has a time budget and a clear rule for when it stops a change. Most teams get stuck on the tests, not on the pipeline.

Setting up a CI/CD pipeline takes days, not months. Plenty of teams have one and still wait a week for a release. The code is ready, but the tests aren’t. Some checks are manual, the test environment is busy, and only one person knows why the nightly run went red. Continuous testing is how you close that gap. It starts with a simple question: which tests should run at which step?

What is continuous testing?

Continuous testing means your automated tests run as part of the delivery pipeline, at every step, instead of in a test phase at the end. A CI/CD pipeline is the automated route a change takes from a developer’s laptop to production. CI stands for continuous integration: developers merge small changes often, and each one is built and tested automatically. Martin Fowler’s article on continuous integration is still the clearest explanation. CD stands for continuous delivery: the software is always in a state where you could release it. Continuous testing is what makes both work. Every change gets feedback within minutes, so a developer can fix a problem while the code is still fresh. The decision to release then rests on test results, not on a gut feeling after a week of manual checks. In practice that means many small test runs, each with its own job.

Continuous testing doesn’t make testers less needed. The repetitive checks run by themselves, so people can spend their time on the work that needs judgement.

Which tests run at which step of the pipeline?

Run the fastest tests most often and the slowest tests least often. Each step in the pipeline gets its own tests, a time budget and a quality gate. A quality gate is a rule that stops a change from moving on when its tests fail. The table shows a split that works in most teams.

A smoke test is a quick check that the main functions start and respond. It doesn’t prove everything works, only that nothing big is broken. The middle steps are where most teams win time. API tests check business rules and the links between systems without a browser. That makes them fast and stable enough to run on every merge. Long UI test sets are better kept for the nightly run. At e-commerce clients we often see the reverse. CI/CD gets slowed down by long UI test sets and manual testing, so the pipeline waits on the slowest part.

What usually blocks continuous testing?

What blocks continuous testing is rarely the tool. In our experience, setting up the pipeline is the easy part. Teams get stuck on everything that can’t run through it reliably and automatically, and that’s almost always the testing side. Four blockers come back again and again.

• Testing can’t keep up. Developers deliver faster than ever, especially with AI writing part of the code. The tests don’t grow at the same pace.
• Regression is still manual. Each release, someone clicks through what already worked. That doesn’t fit into a pipeline at all.
• Environments and data aren’t ready. Test data changes between runs, environments differ from production, and someone has to prepare things by hand.
• Quality depends on one or two people. If only they know how the tests and the release really work, you can only release when they’re around.

Our post on why QA gets called the bottleneck goes deeper into the first one. The short version: adding testers doesn’t help much when the cause sits in how work flows through the team.

How do you keep a pipeline’s tests trusted?

A pipeline is only useful when the team believes a red result. Once people start pressing “run again” until it turns green, the gates stop protecting anything. Four habits keep trust high. First, take a test that fails at random out of the gate right away, and fix it within days. Our post on flaky tests explains the usual causes. Second, give every red nightly run an owner by the next morning. Third, guard the time budgets. When the merge step creeps from 15 minutes to 40, people start to skip it. Fourth, make results visible to the whole team, not only to testers. DORA’s research on continuous delivery links this kind of fast, reliable feedback to both speed and stability.

Two trade-offs to name. Not every team needs to release on every commit. If you release once a month, continuous testing still pays off, because problems show up the day they are made. And no pipeline replaces people. Exploratory testing finds what nobody wrote a test for. Acceptance testing by business users checks that a change fits how they work. Plan both in, especially before big releases.

What this means for your team

A pipeline runs by itself. Its tests don’t stay healthy by themselves. Every sprint adds features that need new tests, and every change can break old ones. Someone has to keep the suite fast, fix what fails at random and keep test data under control. In many teams that work sits with whoever has time, and nobody has time just before a release.

That ongoing work is what we do with Test Automation as a Service. Together with your team, we put backend checks, integrations and real end-to-end scenarios into your pipeline. Your tests run every night, and we keep them reliable. The suite stays yours, and your team decides what ships. One TaaS client sums up the result on our site: “We are able to release on demand thanks to the automated regression suite”.

A first step you can take this week: look at your last three releases. Write down how much time passed between “code done” and “live”, and how much of that was spent waiting for tests.

Frequently asked questions

What is the difference between continuous testing and test automation?

Test automation means tests run without a person clicking through them. Continuous testing means those automated tests run at every step of your pipeline, with clear rules for when they stop a change.

Which tests should run on every commit?

The fastest ones: unit tests and code checks that finish in a few minutes. Slower API and UI tests run on merge or at night, so developers don’t wait.

What is a quality gate in CI/CD?

A quality gate is a rule in your pipeline that stops a change when it fails its tests. For example: no merge when an API test fails. Gates only work when the tests behind them are reliable.

Do you need continuous testing if you release once a month?

Yes, it still helps. Problems show up on the day a change is made, not three weeks later. Fixing them is faster and cheaper while the code is fresh.