What AI can and cannot do with a load test
Should QA engineers use large language models during load tests?

Francis Scialabba
Should QA engineers use large language models during load tests?

Francis Scialabba
When software engineers hand some of their day-to-day load to AI tools, they might include their load test, a common and important application check.
A load test reviews how a system behaves when subjected to a specific amount of user traffic or workload. Quality assurance (QA) engineers and developers define expected traffic and unexpected edge cases; testers then run scripts to simulate actions like logging in, searching, or purchasing—and see what lags, loads, and crashes.
Pros who spoke with IT Brew see AI tools helping with tedious aspects of the process, like finding testing cases and analyzing telemetry, but don’t envision those tools helping to run the test itself anytime soon.
A large language model is too unpredictable to be a “fire and forget” tool for load tests, Kevin McDonald, principal consultant at tech consultancy Hylaine, told us. “You have to have somebody there to monitor it. You have to have somebody there to make sure it’s doing what it should be doing, it’s testing the right things, and it didn’t go down a rabbit hole, or it didn’t start pulling in systems that don’t need to be involved.”
Where AI might pass the test. While today’s pros might not be ready for an LLM to provide the actual load test, some see AI removing some of the day-to-day burden from QA engineers:
Where AI might not pass the test. Kathari and McDonald both see limitations with AI-assisted load testing. The models can identify correlations without identifying the true cause, and may not know your context (like architecture, traffic patterns, and business requirements), Kathari noted.
Also, whenever you’re sending production logs and customer information to LLMs, that means cybersecurity and privacy controls must be considered. “I see AI as something that can help engineers investigate faster, but its conclusions still need to be validated against real telemetry and engineering judgment,” Kathari wrote.
Though Kathari sees AI as a true “assistant” helping engineers generate tests, explore possible failure scenarios, summarize telemetry, and form guesses regarding why performance degraded, she would not want an AI model to be the final authority on whether a system passed a load test, she wrote.
What to do. As for as how to exactly use the AI in a load-test context, Kathari recommends that engineers: