What a speed test measures, and the four things it cannot

What follows is an argument about broadband speed testing, and about where the received version of it stops being true.

How the measurement is constructed

A test opens several simultaneous connections to a server chosen for proximity and available capacity, then measures how much data moves in a few seconds. Parallel connections are used because a single connection to a distant server is limited by round-trip time rather than by line capacity. The result is therefore the best case: short path, uncongested server, several streams, short duration.

That is a legitimate way to measure the access line and a poor description of ordinary use.

What it leaves out

It says nothing about how the connection behaves when something else is already using it, which is the common real complaint. It says nothing about packet loss patterns, which destroy calls and games while barely affecting throughput.

It says nothing about performance to destinations further away or across a congested peering point. And it says nothing about consistency over hours, which is where contention shows up.

Your own network is part of the result

A test run over Wi-Fi measures the Wi-Fi as much as the access line, and on a distant device it mostly measures the Wi-Fi. Running the same test wired and wireless from the same spot separates the two immediately. The device itself can be the limit on older hardware at high line speeds, particularly on phones and cheap laptops.

The short version: browser-based tests add overhead that dedicated applications do not, which shifts results at very high speeds.

Latency figures deserve more attention

Idle latency tells you roughly how far away the test server is and how well your access technology performs. Latency measured while the line is saturated tells you whether the connection will remain usable during a large transfer. Some tests now report both, and the gap between them is the most actionable figure the test produces.

Jitter, the variation in latency, matters more than average latency for voice and video.

Provider tests and independent tests differ

A provider test often runs to a server inside the provider network, which excludes any congestion at the point where the provider connects to the rest of the internet. That is not dishonest — it isolates the part they control — but it can show excellent results while your experience is poor. Running both and comparing tells you which side of that boundary the problem lies on.

In practice, regulators in several countries define approved measurement methods for contractual purposes, and those are what a complaint should cite.

Figures here are typical rather than guaranteed — check the spec sheet for your part.

Testing usefully

Test wired, at several times of day, to more than one server, and record the results rather than trusting memory. Test loaded latency separately from throughput, because they diagnose different faults. Repeat any test that produces a surprising result; transient conditions are common and single measurements are weak evidence.

For an intermittent fault, a continuous monitoring tool that samples over days is worth far more than any number of manual tests.

The takeaway

The headline number is a best case to a nearby server. The loaded latency figure describes your actual day.

Understanding the failure mode tells you more than the feature list does.