When Fireworks AI published its October incident bulletin, one line mattered more than the rest. If you got a notice, rotate the third-party credentials with their issuing providers, because "deleting the stored secret doesn't invalidate the credential."
Read that twice. Plenty of teams hear "a vendor leaked our env vars" and respond by deleting the variable in the dashboard. The key still works. The attacker still has it.
What actually happened
Fireworks found unauthorized use of its own internal credentials. A third party accessed environment variables used in evaluation jobs for a limited number of customer accounts. That likely included API keys and tokens stored as secrets. The company says it found no evidence that customer models, datasets, training data, or inference traffic were touched. Detection started October 6, and affected customers were notified between October 7 and 9.
It wasn't alone. Wiz's October 30 roundup linked at least nine AI developer platform incidents, including Fireworks, Railway, Zeabur, Lightning AI, Warp and Northflank. Wiz is careful to say it hasn't shown one actor behind all of them, and the entry points differ. Zeabur reported plaintext admin credentials. Warp pointed at a third-party vendor dependency. What they share is the target: stored environment variables and AI provider keys.
That makes sense. A dev platform is a vault with a nice UI. Break into the control plane and you collect everyone's downstream credentials in one trip.
How fast stolen keys get used
Wiz also described two AWS Bedrock cases where leaked credentials were abused. One saw about 800,000 requests in 30 hours. The other saw about 1.7 million requests in 13 hours, with more than a million succeeding. Combined usage was estimated at over $100,000 at list prices. The first unauthorized call landed within 30 minutes of the reported credential access.
Thirty minutes. Your billing alert, which fires on a daily rollup, is not going to save you. And the attacker's early moves were quiet: a low-volume probe, a few days of rechecks, then the big burst once they'd mapped models, regions and quotas.
The rotation habit that works
Here's what we'd tell any team we build with.
- Rotate at the issuer, not the host. Revoke the key in the AWS, OpenAI, Anthropic or Stripe console. Replacing it only in a hosting dashboard leaves the original valid.
- Verify the old key is dead. Call it after rotation and confirm it fails. Most teams skip this and assume.
- Scope keys to what they need. One model family, one region, one environment. A stolen staging key shouldn't touch production quota.
- Prefer short-lived workload identities over long-lived static keys wherever your cloud supports it.
- Alert on request and token rates, not just spend. Rate alerts fire in minutes. Billing alerts fire in days.
- Keep an inventory. Old preview environments and forgotten side projects hold live keys too.
None of this is clever. It's the boring kind of work that doesn't get a sprint ticket until the invoice arrives.
The takeaway
You can't stop a platform vendor from getting breached. You can decide how much a breach is worth. A key that is scoped, short-lived, and rotated at its source is worth very little to someone who steals it. A forever-key with account-wide permissions sitting in an env var is worth a six-figure inference bill.
We're here to help founders and teams design and build digital products that are built to scale with you, not slow you down. If you're looking to build something, get in contact with us today!