API keys that retire themselves
The average key in our system had not been rotated since it was created. We made rotation the default rather than a chore.
We audited key age last month. The median key had been in use for fourteen months. The oldest was three years old and had been created by an employee of a customer who left that company in 2017.
Nobody involved was careless. Rotation was a task with no deadline, which is a task that does not happen.
Why rotation is hard
Rotating a key means finding every place it is deployed, updating them all, and confirming nothing broke — and the failure mode is an outage in production. The safe version requires two keys valid at once, which most systems do not support, so people do the unsafe version at the worst possible moment or they do not do it at all.
What we built
Keys can now have an expiry date, set at creation. The default for new keys is twelve months.
A key approaching expiry generates a notification at thirty days, at seven, and at one, to the account owner and to whoever created it. The dashboard shows age and last-used time on every key, so a key that has not been used in six months is visible as an obvious candidate for deletion rather than a row nobody reads.
Most importantly, an account can hold several active keys at once. Rotation is: create the new key, deploy it, confirm traffic has moved using the per-key usage chart, revoke the old one. Nothing is ever down, and each step is reversible.
The part that surprised us
The last-used column found more problems than the expiry did. Roughly a fifth of active keys across all accounts had not made a request in ninety days. Those are pure risk with no benefit — credentials that work, sitting in a config file or a wiki page, protecting nothing because nothing uses them.
We now surface unused keys in the dashboard with a suggestion to revoke. Adoption of that suggestion has been much higher than adoption of rotation ever was, presumably because deleting something unused is easy and rotating something load-bearing is frightening.
Existing keys are not expiring automatically. We are not going to break anyone's production to make a point.