One Missed Shearing. One Very Happy Wolf.

One Missed Shearing. One Very Happy Wolf.

I have been saying this for thirty years. Thirty. Years.

Rotate your credentials. It is not complicated. It is not glamorous. It does not require a subscription to some bloated Sky Pasture service that charges you per "insight." It requires a calendar, a policy document, and a flock that actually reads their memos.

Apparently, Grafana did not get that particular memo.

Here is what happened, stripped of the usual corporate euphemisms: The TanStack npm supply chain was compromised last week. A parasite got into the feed. Grafana, to their credit, responded and began rotating their GitHub workflow tokens. Standard procedure. Good instinct.

They missed one.

One token. Sitting there, unsheared, like a lamb in July. The wolf found it, of course. The wolf always finds the one you forgot. That is literally the wolf's entire job. I have been watching wolves operate since the days when your entire network topology fit on a magnetic tape you could hold in one hand, and I assure you, they have not gotten lazier.

What we are looking at here is a textbook secondary exploitation scenario. The TanStack incident was not just an attack. It was reconnaissance. It shook the fence to find the holes. And Grafana, bless them, left a hole.

The Sky Pasture ecosystem has made the flock complacent. Everyone assumes some automated tool is handling rotation. The shepherds in the C-suite approve a "cloud-native credential hygiene platform" at a conference, declare victory, and go back to their lunch. Nobody actually checks the spreadsheet.

In my day, you checked the spreadsheet. You printed the spreadsheet. You laminated it and put it on the wall next to the terminal. When a token was rotated, you crossed it off with a pen. A physical pen. No token survived unsheared because the list was right there, in front of your face, next to the coffee machine.

Modern tooling has made us soft. Dangerously, embarrassingly soft.

Remediation

This one is almost insultingly simple, which is exactly what makes it so infuriating.

First: After any supply-chain incident in your dependency graph, treat ALL tokens in connected workflows as compromised. Not "probably fine" tokens. All of them.

Second: Maintain an explicit, auditable inventory of every workflow token, service account, and automation credential. A real list. One a human has signed off on.

Third: Your rotation process needs a verification step. Someone confirms completion. Not a dashboard. A person.

Fourth: Assume the wolf already found what you forgot. Audit your logs for access patterns on those credentials immediately.

The vulnerability here was not technical. It was procedural. And procedural failures are the ones that keep me up at night, which is saying something, because I never sleep anyway.

Stay paranoid, stay sheared.


Original Report: https://www.bleepingcomputer.com/news/security/grafana-breach-caused-by-missed-token-rotation-after-tanstack-attack/