Tencent's Gander aims to keep talking while it works in the background

Tencent's research model Gander combines real-time conversation with AI agent capabilities. A "cerebellum" handles the conversation while a swappable "brain" tackles complex tasks. Users can interrupt at any time, but tests show a tradeoff between conversational timing and task accuracy.

Tencent's Hunyuan Speech team and researchers at several universities have introduced Gander, an AI model designed to keep chatting while it handles complex tasks. According to the technical report, it takes in speech, images, and text at the same time.

Today's voice assistants mostly take turns with users, the researchers argue. In real conversations, people interrupt each other, give quick feedback, and listen while talking. Gander is designed to handle that back-and-forth by continuously processing video, speech, and text, even while it speaks. Users can jump in at any time, and the model can ask follow-up questions or give progress updates without being prompted.

Gander keeps the conversation going while its "brain" works

Conversation needs quick responses, but searching files or writing code takes time to plan. The researchers argue that a single model has to balance speed against reasoning ability, so Gander splits the work between two roles.

Borrowing from human anatomy, they call these the "cerebellum" and the "brain." The cerebellum handles real-time conversation while the brain tackles reasoning and complex tasks in the background.

The brain can be swapped out for agent systems such as Codex or Claude Code without retraining the conversation model. In tests, an unspecified model from OpenAI's GPT-5.6 family filled that role. As the underlying model improves, the whole system benefits.

Gander gets the timing right but trails on task accuracy

Gander breaks conversations into one-second segments so the cerebellum can decide when to listen, speak, or stop if the user interrupts. It makes those decisions without a separate module to detect when speech starts and stops, using roughly the last two minutes of conversation as memory.

With no dedicated test for models like Gander yet, the researchers turned to established benchmarks. The report says Gander had the best timing on Full-Duplex-Bench v3, which tests voice assistants across different task scenarios.

Gander starts speaking at the right moment in all 100 scenarios and interrupts users in 8 percent of cases. That compares with 13.5 percent for GPT-Realtime and nearly 48 percent for the weakest competitor. According to the report, Gander uses a relatively small model to compete against commercial systems including GPT-Realtime, Gemini Live, and Grok.

Gander falls slightly behind on task accuracy. The researchers say that's partly because the test scores the whole system, so speech recognition and output errors count against it. The brain scores much better when given text directly.

Video and audio understanding also suffer: Gander performed worse than its base model in one test, which the researchers blame on training that favors fluid conversation over precise perception. That includes tasks such as counting objects and locating them in an image.

Tencent plans to release Gander's weights and training data

Gander was trained on about 2.7 million examples, according to the report. Some teach it to stay quiet when there's background noise or nobody in a group is addressing it.

The researchers say the work is still early. How to scale Gander up remains an open question, and there's no standard way to evaluate systems like it.

The team plans to publish the weights and training data once it completes "the open source release process." A GitHub repository for the code already exists, with demos on the project page.

More companies are splitting agent work across models

Gander follows Tencent's July release of Hy3, an open language model that reportedly narrowed the gap with rivals, especially on agent tasks. Hy3 already runs in WorkBuddy, Yuanbao, and WeChat. The company is also negotiating to take the largest stake in agent startup Manus after Beijing blocked Meta's acquisition. The company sees the deal as a fit for its own plans, including an agent embedded in WeChat.

Other companies are using orchestrators to delegate tasks across models. OpenAI's GPT-Live separates conversation from reasoning, handing web searches and agent tasks to a background model while the chat continues. Sakana AI's Fugu is a separate language model that calls others from an expandable pool. OpenAI is also testing proactive agents that create follow-up tasks and contact users without being asked.

Handling interruptions and avoiding delays remain practical concerns. An Anthropic analysis found that experienced users interrupt Claude Code in about 9 percent of work steps, compared with roughly 5 percent for newcomers. Teams building conversational voice and chat agents report latency problems particularly often, according to a survey.

AI News Without the Hype – Curated by Humans

Subscribe to THE DECODER for ad-free reading, a weekly AI newsletter, our exclusive "AI Radar" frontier report six times a year, full archive access, and access to our comment section.

Subscribe now