“Add concurrent users and check the average response time” is popular load-testing advice. It's also incomplete. Without a realistic workload, meaningful limits, and a clear release decision, you can generate plenty of traffic and still learn very little about whether customers will be able to use your product.
A useful load test isn't a demonstration that your servers survived. It's an experiment that helps engineering leaders decide what demand the product can support, where it becomes unacceptable, and what evidence they still need before releasing.
Table of Contents
- Why Functional Tests Are Not Enough
- The Core Idea of a Load Test
- Load, Stress, Spike, and Soak Explained
- The Metrics That Actually Matter
- A Realistic Load Test Scenario
- Reading Results in a CI/CD Pipeline
- What a Load Test Cannot Tell You
- Frequently Asked Questions About Load Testing
Why Functional Tests Are Not Enough
A green functional test suite means the application produced the expected results under the conditions you tested. It doesn't automatically mean the application can produce those results when customers compete for the same resources.
Consider checkout. A functional test might confirm that a customer can submit an order and receive a confirmation. Under concurrent demand, that same flow may encounter database contention, exhausted connection pools, slow dependencies, or infrastructure limits. Research on realistic web-application load testing describes performance evaluation through virtual users, response time, throughput, and availability, rather than correctness alone.
The difference becomes visible when the team has working features but insufficient evidence about shared capacity.
Correctness and capacity answer different questions
A functional test asks whether an action produces the right outcome. A performance test adds conditions around how quickly and reliably that outcome must happen under demand.
For a CTO, that changes the release discussion:
- Correctness evidence: Can customers complete the intended business journey?
- Capacity evidence: Can they complete it acceptably at the demand you expect?
- Diagnostic evidence: If performance deteriorates, can engineers identify the limiting resource or dependency?
These are complementary requirements, not competing testing strategies. You shouldn't replace functional coverage with traffic generation, because a fast response can still contain the wrong result.
Release principle: Treat functional success as evidence of correctness, not a promise of capacity.
The practical management change is to ask for a capacity statement alongside the functional test report. That statement should name the journey, workload, environment, and acceptance conditions. “Checkout passed under the modeled peak workload” is useful; “performance looks good” leaves too much unstated.
The Core Idea of a Load Test
A coffee shop can serve a quiet queue comfortably and struggle when customers arrive faster than staff can complete orders. Adding baristas may help, but not if everyone must share a cramped counter or the same equipment. This is a hypothetical analogy, not a capacity formula.
The illustration shows why adding resources and handling more demand aren't always equivalent.
So, what is a load test? It's a controlled performance experiment that applies expected or deliberately increasing demand while measuring whether software remains responsive, reliable, and scalable.
A virtual user is a scripted participant that performs defined actions. It isn't necessarily a browser, and it isn't interchangeable with a request. Someone browsing slowly creates a different workload from someone repeatedly submitting expensive searches.
Define the experiment before running it
Start with these decisions:
- Workload profile: Specify the business transactions, their relative frequency, session behavior, and representative data. Describe what users do, not just how many are present.
- Service objectives: Agree on acceptable transaction latency, completed work, failures, and resource consumption before viewing results.
- Ramp strategy: Establish how demand increases, how long each stage remains stable, and when the team should stop the run.
Without these decisions, engineers can unintentionally choose favorable conditions or reinterpret an uncomfortable result afterward.
A useful test also declares its boundaries. If the script exercises APIs but doesn't render the interface, don't present its results as proof of browser responsiveness. If a payment dependency is replaced with a simplified substitute, document that limitation.
Use the accompanying video as another way to visualize the basic testing process.
The output should be a bounded conclusion: this application version met these objectives, under this workload, in this environment. That's more defensible than claiming the product can handle a universal number of users.
Load, Stress, Spike, and Soak Explained
Teams often call every traffic-generating experiment a load test. That shorthand becomes a problem when the test answers a different question from the release risk.
Load testing evaluates expected demand. Stress testing goes beyond intended capacity. Spike testing examines abrupt changes in demand, while soak testing looks for deterioration during sustained operation. These distinctions are described in the performance-testing guide covering workload objectives and test types.
Comparing the Four Core Performance Test Types
| Test type | Primary goal | Typical duration | What it catches |
|---|---|---|---|
| Load | Validate expected normal and peak demand | Long enough to observe stable behavior at each target stage | Unacceptable latency, failures, and capacity constraints |
| Stress | Identify behavior beyond supported capacity | Until failure behavior and subsequent recovery can be assessed | Failure modes and inability to recover cleanly |
| Spike | Examine a sudden increase or decrease in demand | An abrupt transition followed by observation | Problems triggered by rapid workload changes |
| Soak | Assess sustained operation | Extended steady operation | Gradual degradation, including memory leaks |
Duration follows the question. A stress test doesn't become a soak test because it runs longer, and a gradual ramp doesn't tell you how the application handles a sudden surge.
Choose the test from the release risk
For a routine release that changes an important API, start with expected-load validation. If the team needs to understand what happens after supported demand is exceeded, add software stress testing.
A campaign-driven release warrants attention to abrupt arrivals. A service expected to remain busy continuously needs evidence about sustained operation, even if it performs well during a shorter run.
Choose the failure question first. Then choose the traffic pattern that can expose it.
Avoid combining every objective into an indistinguishable run. You might discover a failure without knowing whether sustained demand, excessive demand, or a sudden transition caused it. Separate scenarios make the conclusion easier to interpret and the next engineering action easier to assign.
The Metrics That Actually Matter
An average response time can look acceptable while some customers wait far longer. Percentile latency describes that spread: p95 is the response-time boundary at or below which 95% of the measured requests completed, as represented in Microsoft's load-test results guidance.
Percentiles aren't maximums. They help teams see the slower end of the experience without allowing the fastest requests to dominate the conversation.
Read latency alongside the other measurements shown below.
Read the measurements together
In a reported Apache JMeter experiment, at 500 concurrent users, average response time reached 8,407.28 milliseconds while throughput fell to 38.52 requests per second; at 1,000 users, the average reached 10,860.18 milliseconds, the 90th-percentile response time reached 78,840.30 milliseconds, and errors reached 3.13%, according to the published JMeter experiment.
Those results describe that tested system, not a universal capacity benchmark. Their value is the relationship between measurements: demand increased, completed work declined, and slower requests took much longer than the average suggested.
- Latency: How long does a request or business transaction take?
- Throughput: How much work does the system complete over time?
- Error rate: What share of attempted transactions fails?
- Resource utilization: What's happening to CPU, memory, database capacity, disk activity, and network usage?
Resource telemetry supplies context for bottleneck analysis in performance testing. A latency chart identifies an unacceptable experience; correlated infrastructure measurements help engineers investigate why.
Write criteria the team can enforce
Use a specification like this, replacing each description with an agreed threshold:
| Criterion | Required decision |
|---|---|
| Tail latency | A p99 target and p95 ceiling for each critical transaction |
| Failures | Maximum acceptable transaction failure rate |
| Completed work | Required throughput under the target workload |
| Saturation | Resource limits for each relevant tier |
Don't borrow limits merely because another team uses them. Set them around customer expectations and business consequences, and keep transaction-specific results visible rather than burying checkout inside an application-wide aggregate.
A Realistic Load Test Scenario
Suppose your team is preparing an API-backed checkout release. This is an illustrative test design, not a report of measured results.
The critical journey is browse, add to cart, submit checkout, and confirm payment. Your first task is to define what successful completion means at each step, including the final business outcome. Receiving a response isn't enough if the order never reaches the expected state.
Build a representative workload
Give virtual users varied products, carts, and account data. Include pauses that reflect the intended session behavior, and distinguish shoppers who browse from those who purchase.
A useful scenario sheet might look like this:
| Test element | Checkout scenario |
|---|---|
| Business outcome | A valid order reaches payment confirmation |
| Workload basis | Expected launch demand, with assumptions documented |
| Transaction mix | Browsing, cart updates, and checkout |
| Load stages | Baseline, gradual increase, expected peak, ramp-down |
| Acceptance | Transaction-specific objectives agreed before execution |
| Stop conditions | Unacceptable failures or unsafe resource consumption |
Avoid inventing a precise concurrency target to fill the sheet. When demand is uncertain, record the uncertainty and test a range deliberately.
Observe the client and the system
During execution, align transaction timing with server-side telemetry. Microsoft's application load-testing guidance recommends correlating response time, throughput, and errors with infrastructure measurements such as CPU, memory, database, disk, and network usage.
If checkout latency rises alongside database saturation, investigate the data layer. If failures increase while infrastructure utilization remains normal, inspect application behavior and dependencies. These are diagnostic hypotheses, not proof of cause.
- During ramp-up: Check whether the application reaches each demand stage without departing from its objectives.
- During the hold: Look for sustained completion of the intended business transactions.
- During ramp-down: Observe whether the application returns to normal behavior after pressure decreases.
Finally, change the suspected cause and repeat the scenario under comparable conditions. A faster result is only meaningful if the team has not reduced workload difficulty, changed the data, or removed an important dependency.
Reading Results in a CI/CD Pipeline
A performance report becomes useful when someone can act on it before a release. My recommendation is to integrate tests according to the decision each pipeline stage can support, rather than running the largest scenario everywhere.
For teams establishing a CI/CD pipeline, the key is to balance quick feedback with deeper release evidence. A lightweight check can flag an obvious problem, but it shouldn't be presented as full capacity validation.
The pipeline should connect automated checks with observation after deployment.
Assign a purpose to each stage
| Pipeline stage | Recommended check | Decision it supports |
|---|---|---|
| Pull request | Lightweight workload against the affected flow | Whether an obvious regression needs investigation |
| Merge | Broader scenario in a controlled staging environment | Whether integrated changes preserve expected behavior |
| Release | Expected-peak validation and relevant resilience scenarios | Whether the available evidence supports deployment |
Keep workload definitions and acceptance criteria in version control alongside the application changes they evaluate. Store the environment configuration and results with them so engineers can reconstruct a comparison.
Separate a regression from a changed experiment
When a run deteriorates, ask what changed besides the code. Was the environment smaller? Was the dataset different? Did a dependency behave differently? Did the script perform more expensive work?
Compare against an accepted baseline, but don't let baseline comparison replace customer-facing objectives. A new build might resemble the previous build and still fail the agreed release requirements.
Gate design: Block releases on meaningful, reproducible breaches of agreed objectives, not unexplained dashboard differences.
An inconclusive run deserves an explicit status and an owner. Don't label it a pass, and don't leave the release team with a red indicator nobody can explain.
The release summary should identify the affected journey, unmet objective, supporting evidence, and next action. That gives a CTO a decision to make: fix the problem, investigate further, or approve a documented exception with appropriate rollout safeguards.
What a Load Test Cannot Tell You
A passing load test does not prove production readiness. It establishes that the application met defined requirements within a modeled scenario.
Production can differ in traffic patterns, data distribution, cache behavior, dependency behavior, devices, and networks. Guidance on testing in production treats controlled production validation as complementary evidence when pre-production environments can't faithfully reproduce those conditions.
The important question isn't whether staging is perfect. It's whether the team understands which differences could invalidate its conclusion.
Match rollout safeguards to the uncertainty
Use controlled production validation when a critical behavior depends on conditions staging can't adequately reproduce. Choose safeguards around the uncertainty being investigated:
- Canary releases: Limit initial exposure while observing the changed version under live conditions.
- Feature flags: Keep a changed behavior controllable so the team can disable it when necessary.
- Shadow traffic: Evaluate representative request patterns without making the experimental path responsible for the live outcome.
- Synthetic checks: Continue checking critical journeys after deployment.
These practices require observability, clear stop conditions, limited exposure, and a rehearsed rollback. They aren't permission to discover basic capacity failures on customers.
Preserve the distinction between evidence sources
Pre-release testing helps establish capacity limits in controlled conditions. Production monitoring helps determine whether actual user experience matches the assumptions behind those tests.
A smaller, carefully observed experiment may therefore be more informative than a large staging run with unrealistic behavior. That doesn't make traffic volume irrelevant; it makes representativeness essential.
For a release decision, record both the capacity evidence and the remaining production uncertainty. Deployment approval should include who watches the rollout, what triggers intervention, and how the team will respond.
Frequently Asked Questions About Load Testing
How long does the first load test take?
There's no universal schedule. Scope depends on journey complexity, usable test data, environment readiness, and available monitoring.
Can load testing break production?
It can create harmful demand. Start in a controlled environment and require explicit authorization, stop conditions, and rollback preparation before any live validation.
What concurrency should we start with without traffic history?
Begin conservatively and document assumptions about expected demand. Increase load deliberately rather than treating an arbitrary user count as a forecast.
Should we use synthetic users or real traffic?
Use synthetic users for controlled, repeatable experiments. Representative production patterns can help refine the model when appropriate safeguards exist.
Give your QA lead this starting checklist:
- Select: A business-critical journey and its success conditions.
- Prepare: Workload assumptions, data, telemetry, and safety controls.
- Agree: Acceptance criteria and the owner of the release decision.
- Deliver: A repeatable test and an actionable results summary.
AskYourQA offers performance testing and complete automation systems, including critical-journey mapping, bottleneck analysis, and CI/CD integration. Visit the team to discuss a first load-test cycle built around your release risks, realistic demand, and measurable acceptance criteria.