Meta's Muse Is an Adorable Privacy and Security Dumpster Fire

from the move-fast-and-break-the-planet dept

Meta’s agentic AI product Muse has had a rocky few weeks since launch. The product, which features an animated avatar named Jolly (one presumes to make mass hyper surveillance seem adorable) is supposed to help you offload busywork like making restaurant reservations, paying bills, or ordering groceries.

Despite Meta having claimed repeatedly that Muse was built with a heavy focus on privacy and security, the AI agent launched with a nasty zero-day flaw that made it possible to spy on Mac users. When one tech YouTuber put Muse in charge of their Facebook Marketplace sales, it sold his stuff way below acceptable rates and doled out their home address (the user apparently didn’t understand the permissions he set).

Somebody else found that you could trick Muse into giving root access on the device it’s running on by simply pretending to be a Muse agent yourself. Others found that Muse software not only accesses people’s private messages without approval, it often ignores all permissions and uploads them to the cloud – even if you specifically tell it not to.

That last problem was bad enough that Apple needed to change its macOS privacy settings to stop third-party app developers from misusing them to access message histories:

“Friday’s announcement comes two weeks after tech columnist Jason Aten said that Meta’s new general-purpose AI agent Muse sent him an unsolicited notification referencing a thread between him and a co-worker over Apple Messages. Aten said he never granted Muse permissions to read his messages and had assumed they were off-limits. Social media last week blew up with masses of people who agreed and said the incident showed that AI assistants given access to calendars, emails, messages, shopping accounts, and other resources are akin to a skill saw or other power tool. While potentially useful, they can do real damage if not used carefully.”

Meanwhile, Wired found that Muse consistently creates detailed profiles of all your friends, family, colleagues, “collaborators,” and people you “follow.” Obviously much of that information is necessary for the agent to get to “know” you, but this being Meta, people are understandably uncomfortable with this sort of massive ramp up of data collection in a country, under authoritarian control, that’s too corrupt to pass a privacy law or regulate data brokers:

“These [AI assistant] tools are actively soliciting users to plug their whole lives in—their emails, calendars, financial institutions, everything in order to be helpful assistance,” Bogen says. “That’s dramatically more information than people might have otherwise given to some of these companies. The breadth of access to information that these tools have will lead to a ballooning of what they know about users.”

But wait, there’s more! 404 Media found that in the weeks before launch, Meta was in a mad dash to quickly and sloppily fix multiple other vulnerabilities but refused to delay Muse’s launch to actually make sure the product was secure:

“The Meta source said they felt security teams were asked to push hot fixes to these bugs as quickly as possible and in a way that wouldn’t delay Muse’s launch, leading to what they described as “half-baked protections being rushed out to enable the launch. Many senior engineers believe it’s inevitable we’re going to have a massive data breach as a result of Hatch.” Muse is called “Hatch” internally and in Meta’s codebase.”

Many folks build their own agentic AI solutions with cobbled together open source hardware and software to ensure they have clear understanding of, and control over, what’s actually happening in their name. Meta, a glorified ad monopoly with a history of ethical “lapses,” not only wants to dominate the space, it will spend a lot of time in the new year lobbying against on device, open source, foreign, and/or open weighted alternatives to Meta.

Despite Meta’s claims that privacy and security would be a priority for Muse that’s clearly not the case; it’s also extremely clear that Meta doesn’t fear any meaningful government accountability or oversight, or the product would have spent significantly more time in the oven. Tech giants that had already clearly abandoned quality control at impossible scale are engaged in a mad dash to the trough with new product launches before their funny math causes a market correction.

Such is life in a country that has had most consumer protection regulators (and cybersecurity standards) lobotomized by corrupt authoritarians. Authoritarians guys like Mark Zuckerberg enthusiastically supported because they didn’t like paying taxes — and hated former FTC antitrust boss Lina Khan.

Muse’s early privacy and security problems are the kind of stuff anybody with a head on their shoulders could see coming miles over the horizon. And it’s all inevitably going to get more dangerous — and ridiculous — over the next year as unethical tech oligarchs fully exploit their successful lobotomization of the federal regulatory state. You know, for the love of innovation.

Filed Under: agents, ai, authoritarian, automation, consumers, mark zuckerberg, muse, polly, privacy, security, surveillance
Companies: meta