02 Nov 2026 Test automation tools: how to choose one that still fits in two years
In short: Choose test automation tools by starting from your application, your team and the people who will maintain the tests. Then compare tools on fit, skills, maintenance, pipeline support, cost and lock-in. Before you commit, build one real business flow in the tool. The tool matters less than how well the tests are kept running.
Picking a tool feels like the first step in test automation. In practice it’s often the step teams take too early. A year later the tool works fine, but the suite doesn’t. Tests break after every change, and only one person knows how to fix them. Choosing between test automation tools is a real decision, with a licence and years of maintenance attached. This is how to make one that still fits in two years.
What should you decide before you compare test automation tools?
Decide three things first: what you want to test, on which layer, and who will maintain the tests. Those answers shrink a list of fifty tools to three or four. A web shop, a mobile app and a Salesforce org need different tools. Business rules are faster to test through the API, the layer where systems exchange data, than through the screen. And a tool that suits developers who code every day may not suit testers who don’t. Skip these questions and you end up with a tool that was the right choice for somebody else.
At clients we often see what happens without that first step. API and UI tests run side by side without a plan. Postman, SoapUI, Robot Framework, Selenium, Playwright or Tosca were each picked by whoever started the work. None of these tools is bad. But nobody chose them as one setup, so nobody can say what the suite as a whole protects. Our post on API testing shows which checks belong on which layer.
We see the same pattern in hiring. Job ads ask for Playwright or Tosca experience. Yet risk analysis, test design and knowing the business process matter at least as much. A tool expert who tests the wrong things still leaves you exposed.
Which criteria matter most when you choose a tool?
Six criteria decide whether a tool still works for you after the first year. Feature lists rarely show them. Each one comes down to a question you can ask a vendor, or test yourself in a week. Weigh them for your own situation. A team of developers will care less about low-code features. A team with one tester will care more about how easy it is to fix a broken test.
| Criterion | Question to ask | Why it matters |
|---|---|---|
| Fit with your application | Does it support our browsers, apps, APIs and platform? | Packaged software often needs a specialised tool |
| Team skills | Who can write and read the tests today? | A tool nobody can use stays on the shelf |
| Maintenance | How long does a fix take when a screen changes? | Maintenance is most of the work over the years |
| Pipeline support | Does it run in our CI/CD pipeline with clear reports? | Tests that run by hand get skipped under pressure |
| Total cost | What do licences, hosting and people cost per year? | Free tools still cost time |
| Lock-in | Can we take our tests with us if we leave? | Tests in your own version control stay your asset |
Many tools now add AI features. They draft tests from a user story or repair a test when a button moves. That saves time, and our post on AI in test automation shows where. But a person still decides whether a test checks the right thing. Exploratory testing from a real user’s point of view stays human work, and so does acceptance testing by your business users. Treat AI as a feature of a tool, never as the reason to pick it.
Open source or a commercial tool?
Neither is cheaper by default. Open source tools such as Playwright, Selenium and Robot Framework cost no licence. But you need people who can code, and someone has to build and keep a framework around the tool. Commercial tools come with support, ready-made building blocks and often a low-code editor. They also come with a licence fee, often per user, every year. In both cases the biggest cost is the same: keeping the tests working while your software changes.
So compare total cost over three years, not the price on the website. Count the hours to build the first tests and to fix them after each release. Then add the cost of the people who can do that. If you’re torn between the two big open source names, our comparison of Playwright vs Selenium goes into detail.
Packaged platforms are a separate case. Salesforce, for example, builds its screens in a way that general UI tools find hard to follow. A tool built for that platform can save a lot of repair work. In our Lease Company case, our tester first researched the automation options for Salesforce Service Cloud and then introduced Provar. The company later used Provar on other Salesforce projects as well. That is what a good choice looks like: it keeps fitting after the first project. We are a Provar partner, so weigh that remark with that in mind.
How do you test a tool before you commit?
Run a short proof of concept on one real business flow, not on the vendor’s demo app. Demos are built to look smooth. Your application has the slow pages, odd logins and integrations that decide whether a tool works for you. Let the people who will maintain the tests do the work, so you learn what it asks of them. Two to four weeks is usually enough. These steps keep the test honest.
- Pick one critical flow. For example placing an order, including a login and one integration.
- Build it in the tool. Use your own test environment and your own test data.
- Change something. Move a field or rename a button, then time the repair.
- Run it in your pipeline. Check that the report makes sense to someone who didn’t write the test.
- Write down the cost. Hours spent, licence, and the help you needed.
Do this for two tools side by side if you can. The differences show up fast, and they’re rarely the ones on the feature list.
What this means for your team
The tool is the smaller decision. The bigger one is who keeps the tests running after the choice. Every release changes screens, rules and data, and every change can break a test. That repair work competes with new features, and it usually loses. Then the suite slowly stops being trusted, whatever tool it runs on.
We’re independent: no shareholders, no parent company and no vendor quotas. So we can advise the tool that fits your application and team, or tell you the one you have is fine. With Test Automation as a Service, we review your current testing with your team first. Within the first month there’s a working proof of concept. Then we build, document and maintain the suite, so it stays your asset and doesn’t depend on one person. Your tests run every night, and we keep them reliable. Your team decides what ships.
A first step for this week: write down what you test, on which layer, and who maintains it today. If the last answer is one person, start there.
The right tool, and tests that keep running long after the choice.
Frequently asked questions
What is the best test automation tool?
There is no best tool for everyone. The best one fits your application, your team’s skills and your pipeline, and is easy to maintain. A short proof of concept on one real flow tells you more than any ranking.
Are free test automation tools good enough?
Yes, for many teams. Playwright, Selenium and Robot Framework are used widely. But free means no licence, not no cost: you need people who can build and maintain a framework around them.
Will AI tools replace test automation?
No. AI helps draft tests and repair small breaks, which saves time. Someone still has to decide what to test, run the tests, keep them reliable and judge the results. Exploratory testing and acceptance testing stay human work.
How long does it take to switch test automation tools?
That depends on the size of your suite. Many teams don’t move everything at once. They build new tests in the new tool, move the critical flows first and retire old tests as they break.