Breaches at AI suppliers are now hitting their customers


When a company plugs an AI provider into its products, that provider ends up holding customer data, shared datasets and access keys. This week three of Fireworks AI’s customers found out what that means when the provider gets breached.
On 13 November, Uber confirmed to TechCrunch that thousands of its US drivers had personal and driver’s licence information stolen through a third-party vendor, which TechCrunch identified as Fireworks AI, an inference platform that gives developers access to open AI models. The same day, Cursor said a Fireworks-exposed key was the starting point for the attack that planted a backdoored copy of the Claude Code extension in its marketplace. Notion had already confirmed that some of its customers’ AI chat transcripts were taken.
How it unfolded
Fireworks AI: two incidents in five weeks
Fireworks published its first security bulletin on 9 October. Attackers had used internal credentials to reach customer accounts, and the company linked the activity to a financially motivated group that resells stolen access to large language models (“LLMjacking”). In an update on 9 November, Fireworks widened the impact of that first incident to account and billing information (addresses, invoice and payment history, balances and credit usage), plus secrets, datasets and incomplete models for a limited set of accounts. It said it does not store payment card numbers.
The same update disclosed a second incident. A cloud provider hosting Fireworks’ research machines was compromised on 23 October, and the next day an attacker used that foothold to reach Fireworks development machines and run an unauthorised workload, causing further customer data exposure. Bloomberg reported that Fireworks brought in Palo Alto Networks to investigate and notified law enforcement.
Uber: driver records
In its notice to affected people, Uber said a service provider told it on 9 October, the same day Fireworks published its first bulletin, that an unauthorised party had used compromised internal credentials to access Uber’s account with that provider. By 13 October, Uber had confirmed that its data had been accessed. It is offering free credit monitoring and fraud assistance through Cyberscout. Breach filings with US state attorneys general, reported by TechCrunch, list 2,567 people in Massachusetts, 3,430 in South Carolina and 1,873 in Rhode Island, with Illinois and California also notified. Uber has not given a total and referred questions to Fireworks.
Notion: AI chat transcripts
Notion said Notion AI chat transcripts used for its “Insights” feature were stolen, affecting fewer than 1% of Notion workspaces. It cut all connections to Fireworks on 3 November and deleted its data from Fireworks’ systems the next day.
Cursor: from a vendor key to a backdoored extension
In a 13 November update, Cursor said an attacker obtained an access key it had issued to Fireworks, and that the key was exposed by Fireworks. The attacker used it to copy coding-agent traces, mostly from 2025 sessions not run in Privacy Mode, from a storage bucket Cursor shared with Fireworks. Some traces contained credentials, which led the attacker into some of Cursor’s SaaS tools and, from there, to an internal credential.
On 29 October that credential was used to place
tampered copies
of Anthropic’s Claude Code extension (versions 2.1.312 and 2.1.313) in the
cache behind Cursor’s extension marketplace. For about 11 hours, up to 5,000
users could have received the malicious build.
According to the security researchers,
it added an attacker SSH key labelled svc-telemetry, set up persistence on
Windows, macOS and Linux, tried to open a reverse SSH tunnel and sent host details to
an attacker server. Cursor said Anthropic did not publish the tampered copy,
the issue was not a flaw in Anthropic’s product, and the Open VSX registry was
never affected. Cursor has since moved all of its inference traffic off
Fireworks.
What AI suppliers hold
Uber, Notion and Cursor were each exposed through a supplier they had trusted with data or credentials. The prompts are only part of it. Billing records, fine-tuning datasets, model files, shared storage buckets and API keys all sit with the provider.
Cursor’s case shows a second risk. Developers had pasted secrets into coding sessions, the traces of those sessions sat in a bucket shared with Fireworks, and the attacker used them to move further in.
Steps to take
- Make a list of your AI suppliers and what each one holds. Include inference, evaluation and fine-tuning providers, and any storage or datasets you share with them.
- Rotate the keys you have issued to AI vendors and narrow what they can reach. Short-lived, tightly scoped credentials with network restrictions are safer, and anything a vendor no longer needs should be revoked.
- Keep secrets out of prompts and traces. Scan and redact logs, traces and datasets before you share them, and use zero-retention or privacy modes where a provider offers them.
- Cursor users: check whether version 2.1.312 or 2.1.313 of the Claude
Code extension was installed from Cursor’s marketplace on 29 October, and
look for a
svc-telemetryentry in~/.ssh/authorized_keys. Then follow Cursor’s remediation steps, because uninstalling the extension may not remove the backdoor. - Fireworks customers: read your incident notification and rotate any secrets you stored on the platform.
- Uber drivers who get a notice: expect phishing that uses your licence details, and take up the free credit monitoring offered in Uber’s letter.