← Blog

Why Two Speed Tests Disagree

Run two speed tests back to back and you get two different numbers. Neither is lying. Here is what each one actually measures, why parallel connections read higher, and which number answers which question.

Run a speed test. Write the number down. Run a different one thirty seconds later. The second number will almost certainly disagree with the first, sometimes by a factor of two.

The usual conclusion is that one of them is broken. Neither is. They are measuring different things and reporting both results under the same word, “speed,” which is where the confusion comes from.

A single connection has to warm up

TCP does not begin at full speed. It begins cautiously and accelerates, a mechanism called slow start: the sender transmits a small amount, waits for acknowledgement, then roughly doubles what it sends in flight, repeating until it hits a limit or starts losing packets. This is specified in RFC 5681, the TCP congestion control standard.

The practical effect is that a single TCP connection spends its opening moments well below the capacity of your link. On a slow connection the ramp is a rounding error, because the ceiling arrives quickly. On a fast connection the ramp is a meaningful fraction of a short test. Measure a 500 Mbps link with one connection for five seconds and a real part of that window is spent accelerating rather than at full rate.

So a single-connection test reports a number that includes its own warm-up. That is not an error. It is what one stream of data actually achieves, which is also what a single large file download achieves.

Parallel connections skip the problem

Open eight connections at once and the picture changes. Each one is slow-starting independently, but they are doing it simultaneously, and their combined throughput reaches the capacity of the link much sooner. Add the eight streams together and you get a figure much closer to the theoretical maximum of the connection.

This is why most consumer speed tests use several connections. It answers the question “what is this link capable of in total,” and it answers it fast enough for a web page. RFC 6349, the framework for TCP throughput testing, treats the number of parallel connections as a core parameter of any test rather than an implementation detail, precisely because it changes the result.

The two approaches are both valid and they answer different questions:

  • Parallel streams answer: how much capacity does this link have?
  • A single stream answer: how fast will one download actually go?

If you are checking whether you are getting the bandwidth you pay for, you want the first. If you are estimating how long one large file will take, the second is closer to your lived experience.

The server matters as much as the method

Distance and routing also move the number. A test server one network hop away flatters your connection. A server three countries away measures the internet between you and it as much as your own link. Different tests choose different servers, and some choose a nearby one deliberately in order to isolate your local connection from everything downstream.

Cloudflare’s speed test runs against their own network, which is unusually widely distributed, and they have written about the methodology and what it is designed to surface. Netflix’s fast.com measures against Netflix’s own delivery infrastructure, on the reasoning that what a viewer cares about is throughput to the service they are actually using. Both are defensible. Both will give you a different number.

Latency is the number people ignore

Throughput gets the big font, but round-trip latency is what you feel in a video call, a game, or a terminal session. A 900 Mbps connection with 120 ms of latency feels worse for interactive work than a 100 Mbps connection with 8 ms.

Latency also has its own measurement subtleties. The figure you see is usually the best of several probes rather than an average, because the fastest round trip is the closest thing to the true path delay, with queuing and scheduling noise stripped out. That is a reasonable choice, and it is worth knowing it was made.

What BandBar does, stated plainly

We build BandBar, a free menu bar speed test for macOS, and it makes specific choices that you are entitled to know about before you compare its output to anything else:

  • Download is a single HTTP stream from Cloudflare’s public speed test endpoint, timed from the first byte to the last. One stream, with its slow start included.
  • Upload posts a payload to the same endpoint and is timed the same way.
  • Ping is an HTTP HEAD request to 1.1.1.1, to 8.8.8.8 and to Cloudflare, reporting the fastest of the three round trips.
  • Connection details come from the system: type, interface and local address via the standard network path APIs (NWPathMonitor), and the external address read from Cloudflare’s trace endpoint.

The consequence is honest and worth stating in advance: on a fast link BandBar will usually read lower than a multi-connection test. That is the single-stream figure, and it is the one that predicts how fast one download will go. If you want the capacity figure, run a parallel-stream test, and now you know why the two disagree.

Results live in memory while the app is running and are cleared when you quit. Nothing you measure is written to disk.

The short version

Do not average two speed tests, and do not assume the higher one is the accurate one. Ask what each was measuring. A number without its method is a number you cannot use, which is why the method belongs in the open rather than buried in a support article.


Sources: RFC 5681, TCP Congestion Control · RFC 6349, Framework for TCP Throughput Testing · Cloudflare speed test · Cloudflare, on testing home network performance · fast.com · Apple, NWPathMonitor