Most systems are tested at low concurrency during development. One developer, one browser, one request at a time. The behavior at 1 request per second and the behavior at 1,000 requests per second are often very different - different performance characteristics, different failure modes, sometimes entirely different behavior.

Load testing simulates the traffic your system will receive in production, before it receives it. The goal is not to prove that the system can handle load. It’s to find the point where it can’t, understand why, and either fix it or make an informed decision about the tradeoff.

What Load Tests Actually Find

The valuable findings from a load test are not usually “the CPU hits 100% at 500 req/s.” They’re more specific:

The database connection pool exhausts first. The application servers have capacity but they’re all waiting for database connections. Fix: tune the pool size, add read replicas, add caching.

A specific endpoint degrades disproportionately. Most endpoints handle load fine but one has an N+1 query that only shows up under concurrent access. Fix: add a join or batch query.

Memory grows without bound. At low load, a memory leak is invisible. Under sustained load, memory climbs until the process crashes or the OOM killer intervenes. Fix: find and fix the leak.

Response times degrade before throughput drops. At 400 req/s the system processes all requests. At 600 req/s, it still processes them all, but median latency doubles and the 99th percentile becomes unacceptable. Fix: might be the queue depth, might be lock contention, might be a slow dependency.

A cold cache behavior is catastrophic. The first request after a deployment or Redis restart triggers a cache stampede - thousands of simultaneous database queries for the same data. Fix: cache warming, stampede protection.

These are not hypothetical scenarios. They are the findings that come out of load tests on real systems.

Key Metrics

Throughput (RPS/TPS): requests or transactions per second. The capacity measurement - how much can the system handle.

Latency percentiles: p50 (median), p95, p99, p99.9. Averages are misleading - a p95 of 2 seconds means 5% of users wait more than 2 seconds even if the average is 200ms. Set targets for percentiles, not averages.

Error rate: what percentage of requests fail. Both 5xx server errors and timeouts. Should be near zero under normal load and have a defined acceptable limit at peak.

Saturation indicators: CPU, memory, database connections, thread pool utilization. When these approach 100%, you’ve found the bottleneck.

Load Test Types

Load test: ramp up to a target load level and sustain it. Validates that the system handles expected production load with acceptable performance.

Stress test: keep increasing load until something breaks. Finds the breaking point. Tells you what fails first and how it fails - graceful degradation or catastrophic collapse.

Spike test: sudden burst of traffic, then return to baseline. Simulates a flash sale, a viral post, a major event. Tests whether the system recovers after the spike.

Soak test: sustained load over a long period (hours to days). Finds memory leaks, connection leaks, and degradation that only appears with time.

Tools

k6 (Grafana): script tests in JavaScript, good scripting capabilities, runs from CLI or cloud.

// k6 script
import http from 'k6/http';
import { check, sleep } from 'k6';

export const options = {
    stages: [
        { duration: '2m', target: 100 },   // ramp up to 100 users over 2 min
        { duration: '5m', target: 100 },   // hold at 100 users for 5 min
        { duration: '2m', target: 0 },     // ramp down
    ],
    thresholds: {
        http_req_duration: ['p(95)<500'],  // 95% of requests under 500ms
        http_req_failed: ['rate<0.01'],    // error rate under 1%
    },
};

export default function () {
    const res = http.get('https://your-api.com/products');

    check(res, {
        'status is 200': (r) => r.status === 200,
        'response time < 500ms': (r) => r.timings.duration < 500,
    });

    sleep(1);
}

Artillery: YAML/JS configuration, good for API load testing.

Locust: Python-based, tests as Python code, useful for complex scenarios.

Apache JMeter: long-established, GUI-based, more setup required.

For most backend API load testing, k6 or Artillery are the current defaults - low setup cost and scriptable enough for real-world scenarios.

Designing Realistic Tests

The most common mistake in load testing: testing a simplified scenario that doesn’t represent real traffic.

Test realistic endpoints in realistic proportions. If 80% of your traffic is GET /products and 20% is GET /users, weight your load test accordingly. A load test that only hits one endpoint tells you about that endpoint, not the system.

Use realistic data. Database queries that hit the same cache-warm few rows behave differently from queries with a realistic distribution across millions of rows. Parameterize your test data to simulate real access patterns.

Test authenticated flows. If most of your endpoints require authentication, test them with authentication. A load test that only hits public endpoints misses the load on your auth layer.

Include think time. Real users don’t make requests as fast as possible. Adding a small sleep between requests in your test script produces more realistic concurrency behavior.

// More realistic: mix of endpoints with weights
export default function () {
    const endpoints = [
        { url: '/products', weight: 50 },
        { url: '/products/details?id=${randomId()}', weight: 30 },
        { url: '/users/me', weight: 15 },
        { url: '/cart', weight: 5 },
    ];

    // Pick endpoint based on weight
    const endpoint = weightedRandom(endpoints);
    const res = http.get(`https://api.example.com${endpoint.url}`, {
        headers: { Authorization: `Bearer ${__ENV.TEST_TOKEN}` }
    });

    check(res, { 'status 200': (r) => r.status === 200 });
    sleep(Math.random() * 2 + 0.5);  // think time: 0.5-2.5 seconds
}

Running the Test

Don’t load test production directly unless you have no other option and have done a careful risk assessment. Use a staging environment that mirrors production as closely as possible: same instance types, same database size, same caching configuration.

Monitor the system under test in real time. Application metrics, database metrics, infrastructure metrics. k6 outputs results to stdout; pair it with Grafana dashboards or your APM tool to see what’s happening inside the system as load increases.

Start below expected capacity. Ramp gradually. If you start at full load and the system immediately falls over, you haven’t learned where the threshold is.

Run multiple times. Results vary between runs. A single run might hit a cold cache, a GC pause, or a deployment restart. Multiple runs give you confidence in the results.

After the Test

The output of a load test is a set of findings: where the system degraded, what the bottleneck was, what the failure mode was under extreme load.

Some findings require a code fix (the N+1 query). Some require a configuration change (connection pool size). Some require infrastructure changes (horizontal scaling, read replica). Some require a conversation about product requirements (if 10,000 concurrent users is the target and the system handles 3,000, that’s a significant infrastructure investment).

The test didn’t fail. It told you something. The question is whether to fix it before launch or accept the risk. That’s an informed decision. The alternative - finding out in production - is an uninformed incident.