AI labs have a data trust problem that their policies haven't solved

Nvidia won't even trust Anthropic with sensitive work despite being an investor

For companies worried about their intellectual property, that's a dealbreaker. Nvidia now uses Fable only for less sensitive tasks like open-source projects, according to The Information. For internal work like AI-powered supply chain monitoring, the company runs its own Nemotron models instead. "As a company, you know, we believe ZDR [Zero Data Retention] should be on by default," Justin Boitano, Nvidia's VP of Enterprise AI, told The Information. Nvidia has invested in Anthropic, reportedly plans to continue doing so, and supplies the company with hardware for model development.

Booz Allen Hamilton, one of the earliest users of Anthropic's Mythos model, has banned employees from using Fable for work on proprietary cybersecurity software, according to The Information. CTO Bill Vass said, "We worry a little bit that [Fable] might be learning from some of our code."

Palantir is blocking Fable deployment through its own software to customers until Anthropic grants irrevocable zero-data-retention guarantees, The Information reports. CEO Alex Karp said at a customer event that companies are tired of being "exploited" by AI labs. Karp has been vocal about his distrust before. Palantir's stance is also self-serving, since the company wants customers running AI models through its supposedly secure platform rather than going directly to providers.

After customer pushback and OpenAI's August move to let GPT-5.6 Cyber customers store security logs on their own servers, Anthropic followed suit with a similar program rolling out to select customers this fall.

Zero data retention still leaves gaps

Even with zero data retention, labs can learn from how their services get used. Both OpenAI and Anthropic collect metadata and technical usage data from enterprise customers, according to The Information. OpenAI calls this data "de-identified," meaning it's stripped of information that could be traced back to individual customers.

OpenAI states on its website that it runs business data through automated classifiers and security tools "to better understand how our services are used." The resulting classifications are metadata about the business data "but do not contain any of the business data itself," the company writes. Some customers aren't sure what exactly that metadata covers, according to The Information, and don't think the current transparency is enough.

Training on user data happens in ways companies won't talk about

John Schulman, OpenAI co-founder who briefly worked at Anthropic and now works at Thinking Machines, recently laid out the different ways AI companies can train on user data. The spectrum runs from direct pretraining on user data, which carries a high risk of reproducing content, to distilling large models into smaller ones, to building reinforcement learning tasks from "user traces."

That last approach has a low risk of content reproduction but can still extract customer IP. It ranges from harmless ("use explicit user feedback in reward model training") to invasive ("upload user's coding environment and commit history to turn into rl envs"), Schulman says. "De-identification is weak," he adds, and users can be traced back "with just a small number of bits" and it doesn't protect against IP leakage.

AI researcher Sarah Hooker, who previously worked at Cohere and Google DeepMind, describes a similar loophole. There are "clever synthetic data techniques that can generate distributional equivalent data while preserving privacy." In other words, even if an AI lab doesn't use original data directly, it might be able to extract statistical patterns that get the same job done.

Hooker warns companies, "If you are a company with IP you have a limited window to build your own intelligence that leverages your IP. Otherwise you are fueling a frontier lab which will encroach on your vertical sooner or later."

In a follow-up post, Schulman walked that back somewhat. Training on user data is "exceedingly unlikely" to contribute much to frontier capability gains, he said. Those gains come primarily from scaling pretraining and reinforcement learning. User data is more useful for finding failure modes or situations that are hard to replicate with paid annotators.

He added, though, that "model companies vary in how aggressively they train on user data (and uploading repos isn't hypothetical)." That's likely a nod to AI coding tools like Codex or Cursor, where users connect their code repositories directly to the services. Schulman called for "stronger norms around disclosing how companies train on user data."

The Buckmaster case made the trust problem real

Schulman weighed in after mathematician Tristan Buckmaster leveled serious accusations against OpenAI. Buckmaster and his co-author Levent Alpöge had used AI models to make progress on the Navier-Stokes equations, uploading their drafts through OpenAI's Codex. Shortly after, OpenAI presented its own breakthrough using the same unusual solution path.

OpenAI initially acknowledged that it could not rule out that anonymized data derived from their use of our products contributed to improving our models. After an internal investigation, the company updated its post, saying Buckmaster's Codex prompts from the two months before the September 8, 2026, publication "could not have influenced the system in any way, including through training." Still, the case showed how fragile the trust between business, academia, and the AI labs really is.