Moving the gateway to HTTP/2
Multiplexing removed a class of head-of-line blocking we had been working around for two years. It also broke our connection pooling assumptions.
The gateway now speaks HTTP/2 to any client that offers it. The specification has been an RFC since 2015 and library support finally caught up enough to make this uneventful. Mostly.
What we got
Multiplexing is the whole reason to do this. Under HTTP/1.1, a client making twelve concurrent calls opens six connections and queues the rest, because that is the per-host limit browsers and most clients settle on. Those queued requests wait behind whichever of the six is slowest.
We had customers working around this by sharding calls across several hostnames, which is a hack the protocol forced on them. With one connection carrying all twelve streams, that hack is unnecessary and we have told them they can stop.
Header compression matters more than we expected. Our requests carry an auth header, a request ID, and a handful of client hints on every call. Under HPACK those are sent once and referenced afterwards. For chatty clients making small requests, the saving is a meaningful fraction of the bytes.
What broke
Connection pooling assumptions, everywhere. Our internal metrics counted connections as a proxy for concurrency, which was roughly true under HTTP/1.1 and is nonsense under HTTP/2 — one connection can carry a hundred in-flight streams. Several dashboards showed traffic collapsing on the day we shipped. Traffic had not collapsed.
We also found a client library that advertised HTTP/2 support, negotiated it, and then serialised every request onto a single stream anyway. It was slower than HTTP/1.1. We now test the protocol behaviour of the libraries we recommend rather than trusting the changelog.
What we did not get
It is not faster for a single request. If you make one call and wait, HTTP/2 does nothing for you beyond the header compression. The wins are all about concurrency, and a customer whose workload is one call per user request will see no change at all.
We turned it on with ALPN negotiation, so HTTP/1.1 clients are unaffected and nobody has to do anything.