The support ticket is part of the product

We evaluate providers on documentation, latency under load, and support responsiveness. Of the three, support responsiveness has turned out to be the strongest predictor of whether a listing is any good, and it is the one nobody puts on a comparison page.

What we measure

Twice a quarter, we send each provider a genuine technical question through their normal support channel, from an account they cannot identify as ours. Not a trivial one — something requiring a look at the actual behaviour, like what happens to an in-flight batch when a rate limit is hit halfway through.

We record time to first human response, and whether the answer was correct.

What it correlates with

Almost everything. Providers who answer a real question within a working day also tend to have accurate documentation, honest rate limit behaviour, and changelogs that mention breaking changes before they ship.

Providers who take a week, or answer with a link to the FAQ that does not address the question, tend to have surprises in every other dimension too. The support queue is where an organisation's actual attitude to its users is visible, because it is the part that cannot be written by a marketing team.

The one that failed

We delisted a provider last year over this. Their API was fine — good latency, sensible errors, competitive price. Three consecutive test questions went unanswered for more than two weeks, and a customer incident took eleven days to get a reply that turned out to be wrong.

An API that works ninety-nine per cent of the time is a support relationship one per cent of the time. If that one per cent is unreachable, the ninety-nine does not matter.

Ours

We hold ourselves to the same measure and publish it: median time to first human response on a technical ticket, updated monthly, on the status page. It is currently just over three hours during working days.

It is a number we are occasionally embarrassed by, which is the point of publishing it.