Why we publish our p99, not our p50
Median latency is a marketing number. The tail is what pages your on-call. Here is every percentile we measure and how we collect it.
Every vendor page quotes a median. It is the most flattering number available, and it tells you almost nothing about the experience of running something in production.
The median hides the failure
If half your calls return in 40ms and one in a hundred takes four seconds, the median still reads 40ms. Your users experience the four seconds. Your on-call engineer is paged about the four seconds. Nobody has ever been woken up by a p50.
What we measure
Latency is recorded at the gateway for every call, not sampled. We keep p50, p90, p99, and the maximum for each API, each day. The number on a listing is the median because that is what people compare on, but the detail page shows the spread, and the status page shows the bad days rather than hiding them.
What we do not do
We do not exclude errors from latency. A request that failed slowly still cost you the wait. We do not measure from inside our own network either — the figure includes the time to reach us.