The coming explosion in QA testing
How companies like Ranger are betting on AI to fix a broken industry
Disclosure: I’m a small angel investor in Ranger.
The AI revolution isn't just going to transform how we write code— it's also going to trigger an explosion in software testing. AI tools like Cursor and Devin are going to make developers dramatically more productive, but they will also massively increase the amount of Quality Assurance (QA) testing needed to make sure all this AI-driven code actually works.
This massive increase in demand for QA is the problem Ranger is trying to solve. Ranger is an AI product that writes, runs, and maintains QA tests to prevent bugs. They publicly launched today with an $8.9m fundraise led by General Catalyst and XYZ Venture Capital.
When Josh Ip, my friend and CEO of Ranger, approached me about writing about the company for its launch, he told me that he didn’t want a puff piece. So I’m not going to talk about the 100-mile races he runs or his other accomplishments (uh... oops). Ranger is also still in its early stages and Josh doesn’t pretend to have all the answers to the future of QA or AI.
But I also found Josh to be needlessly humble: Ranger’s initial traction is incredibly impressive. OpenAI used them for o1 safety testing (see OpenAI’s brief here) and they are used by other fast-growing startups like Suno and Clay. Their product has the ability to take QA testing that took hours and do it in 5 minutes. Some customers have said the product saves 5 hours per engineer per week, which equates to tens of thousands of dollars of savings per engineer annually.
But as someone who’s never been a software engineer or particularly technical, what intrigues me most about Ranger is how their approach offers an interesting window into the future of software engineering, how AI-native products are being built, and how AI is going to disrupt services industries that startups have traditionally had trouble with.
Why QA is broken and how AI will fix it
The reality today is that many companies do the bare minimum of QA testing. This is especially true for startups, but it’s also the case for well-funded public companies (see the recent Sonos debacle). Why? As Josh noted to me recently, QA is often seen as a necessary but unloved chore by software engineers. “It’s kind of like asking the chefs to wash the dishes when they should be focused on making the food.”
Open-source software like Playwright has made it easier for software engineers to automate their web QA (instead of someone manually clicking buttons and filling out forms). However, they still require engineers to write complex testing code that can easily break from small changes to a website.1
It’s therefore been simpler for some teams to outsource QA to (cheaper) QA engineers, either employed directly or outsourced to another firm, instead of having your engineers adopt complicated new software. Ranger estimates ~$35b is spent annually on QA engineers in the US alone. But few are satisfied with this experience given it’s still expensive and slow. As a result, QA is often the first to get cut when trying to save money or ship more quickly.
AI changes this by allowing a 10x better product experience than before.2 Instead of having your expensive engineers write code, you can tell Ranger’s AI agent the test you want to run in plain English and it generates the code for you. Maintenance of code will become significantly easier too — because AI semantically “knows” the goal of a test, it can also update testing code when products change.
Here’s an example of Ranger being asked to book a campsite at Yosemite. Ranger’s proprietary web browsing AI agent can complete web tasks with just a natural language prompt for direction. At the same time, it’s writing out code on the right side of the screen that will be used to test going forward.3
Why AI will lead to an explosion, not a reduction, of QA
With AI improving software engineering efficiency, is there a risk that it gets so good at writing code that we don’t actually need to do much QA anymore? Will the QA engineer go the way of the bank teller?
It’s possible but as I’ve written before, I think the opposite will happen because Jevons Paradox will win out. This describes when technology increases the efficiency of a resource but paradoxically leads to an overall increase in consumption, not a decrease.4 This has happened historically with software engineering, as we employ more software engineers than ever despite each software engineer today being orders of magnitude more productive than 30 years ago.
This is likely to apply to QA too.
AI will make QA testing much cheaper and faster
Companies already want to do far more testing than they can afford
The amount of code being shipped is about to increase exponentially
AI-generated code may need even more rigorous testing than human-written code
So even if software engineers get more efficient at writing code with AI, the combined effect is likely to be more QA work being done, not less. As AI proliferates, small startups will run the type of comprehensive testing that only large enterprises can afford and manage today.
The race to own QA's future
Ranger has a compelling product but they aren’t alone in trying to take advantage of AI in the QA market. Incumbents like Tricentis are racing to incorporate AI and many other AI-native startups are emerging with different approaches.
Ranger’s strategy is somewhat distinct. Instead of selling their software directly, their internal team currently uses it to power their QA-as-a-service for startups. Not everything is automated with AI and human employees will still double-check the code and edit when necessary. With companies hitting “tool fatigue”, Ranger is taking the approach of owning their customers' problems end to end (see this great piece by Nikunj Kothari on this trend). I’d bet this model delivers better outcomes to customers than pure SaaS today but it is undoubtedly harder to scale as quickly.
Whether Ranger’s model wins out or not, I’m increasingly convinced that AI will both cause an explosion in the need for QA and AI-native products like Ranger will meet that demand and fundamentally change how QA is done.
Some startups have tried to solve this by building no-code tools so less technical users could build and maintain QA tests. But the improvements have been largely incremental and not impactful enough to be worth learning a new tool for.
As I’ve written about, this is something we are seeing happening to many services industries, too. For example, traditional legal tech solutions were only incrementally better (e.g., E-Discovery tools), so many law firms didn’t adopt them and continued with traditional labor-intensive processes. But generative AI is a 10x better solution than previous products, so the math is now a no-brainer for legal firms integrating new AI-driven software solutions.
Ranger only services Web QA today given their system uses the HTML/DOM to take web actions.
In Econ 101 terms, when demand for a good/service is very elastic, a reduction in price can cause such an increase in quantity demanded that the total consumption increases.