AI agent makers are promising privacy — will they deliver?

At this year’s OpenAI DevDay, CEO Sam Altman unveiled the company’s new AI agent Dots — and told the crowd that the company wants to “set a new standard for privacy in frontier AI.” OpenAI would spend the day taking veiled shots at Meta’s Muse, its primary competitor, for failing to keep users’ data safe. Yet Muse itself, a couple of months earlier, had launched as a supposedly safer alternative to predecessor OpenClaw — with CEO Mark Zuckerberg promising it was “built from the ground up for privacy and security.”

In an age when companies hoard customers’ personal data and cyberattacks are a dime a dozen, AI labs are trying to convince users to share even more information with their agents. Their latest strategy for success is one-upping the competition by promising that unlike their rivals, they’ll keep user data away from prying eyes. The question is whether companies can keep these promises.

Nat Friedman, head of product at Meta Superintelligence Labs, wrote on X that the company’s “goal with muse was to build something like openclaw that we could make safe and secure and easy to use and scale to billions of people.” User data is stored on a secure VM, what Zuckerberg described as an “isolated linux computer with a browser, CPU, memory, and storage.” The company said that most of the effort in building Muse was related to “careful design and engineering to operate [it] more safely.” Meta’s blog post continued, “Muse can and will still make mistakes, but we expect they’ll be much less frequent and cause much less damage due to the safety systems we’ve built in.”

But though Muse topped the App Store charts and, per Apptopia, gained 600,000 daily active users in the US within weeks, Meta’s promises seem to have fallen short. Although data is isolated from other users, and the company plans to introduce a way “to cryptographically and verifiably prevent Meta from accessing data in your VM” later this year, Meta itself can still access the data. A security researcher quickly exposed a zero-day vulnerability that could allow someone to take control of Muse (which has since been patched), raising fears of outside attacks. Multiple serious security issues reportedly cropped up at the last minute before launch, one of which could have allowed users to access Meta’s own internal databases, per 404 Media.

Muse also appears to liberally collect (and in some cases, hand out) data. It defaults to allowing Meta to train models on what users put into it, though it’s possible to opt out. An Inc. reporter complained that Muse uploaded and read his private messages without him asking it to, and a YouTuber said it had offered his address to a stranger via Marketplace — in both cases, Muse was functioning apparently as intended, but the user didn’t realize how far it would go. The platform creates “detailed profiles of all your friends and family,” as Wired reported. None of this is necessarily unexpected from Meta, but it certainly doesn’t back up the idea that Muse is uniquely privacy- or security-conscious.

OpenAI seized on this fact when announcing Dots in late September. Alexander Embiricos, OpenAI’s Codex product lead, said onstage at DevDay that OpenAI is focused on having the “most trustworthy, safe, and secure assistant,” and CEO Sam Altman demonstrated ways that people could exert control over their individual Dots, such as setting a rule that it should never make a purchase over a certain dollar amount. “I think we’re in a different position to Meta in that they don’t have an AI product that has 1.2 billion users,” said Glen Coates, OpenAI’s head of app platform, adding that “launching something that makes those kinds of mistakes is something that we would try to take the care to avoid.”

Altman himself also pushed the privacy angle at DevDay. To court business customers, executives presented a framework allowing enterprises to have “stronger controls” over their data and zero data retention policy options (meaning no data is stored on OpenAI servers). Indeed, so far, there haven’t been many privacy scandals with Dots — although since it’s only available on the $100-and-up ChatGPT subscription tiers, there are also likely fewer people using it.

People may still not feel comfortable giving OpenAI their personal data, in part simply because AI agents require so much of it. For instance, The Verge’s Allison Johnson felt uncomfortable typing in her bank information when the bot asked as part of a relevant task (Muse has a Stripe integration to take care of that).

Not all companies are making privacy promises, of course. Instinct was publicly criticized for reportedly overly-broad terms of service that gave it unfettered access to data; since then, the company seems to have made adjustments. For now, AI labs seem to be counting on a three-part approach to popularizing AI agent use with the general public: Make them useful, make them cute and disarming to help offset the creepiness, and make promises about privacy — and hope they hold up.