Search: “speech-to-text”

Intelligent transcription with Gemini 3.5 Transcribe

Google DeepMind launched Gemini 3.5 Transcribe, a speech-to-text model offering more intelligent transcription as part of the Gemini family.

Google DeepMind announced Gemini 3.5 Transcribe, a new speech-to-text model described as delivering more intelligent transcription. The blog post provides limited technical detail in the available text, with no benchmarks or model sizes given. The release adds a dedicated audio transcription model to the Gemini family.

Google DeepMind·Aug 26, 17:01 UTC · 21d agoModel release

Show HN: Nari Qwen3-TTS and Qwen3-ASR – High accuracy, low latency and cost

Nari Labs claims top Coval voice AI benchmark rankings with low-latency, low-cost Qwen3-ASR and Qwen3-TTS inference endpoints.

Nari Labs says its Qwen3-ASR Fast endpoint ranks #1 in Coval's time-to-final-segment latency (p50 44 ms) with 3.6% WER at $0.12/hour, behind only AssemblyAI Universal 3.5 Pro on accuracy. Its Qwen3-TTS Fast ranks #2 in time-to-first-audio (p50 63 ms) and #1 in WER at 3.8%, priced at $10 per 1M characters. The company reports beating the official Qwen3 TTS Flash Realtime endpoint (8.8% WER, 692 ms median TTFA) and Baseten's dedicated endpoint (6.0% WER, 101 ms). Public beta APIs are moving to paid general availability with $20 in credits for existing accounts.

Hacker News · security·Sep 14, 16:07 UTC · 2d agoAI tools & infraHN 21↑ · 7 comments

Building and Evaluating Fixed-Voice Thai TTS from Synthetic Speech

Researchers distill a compact 82M-parameter Thai TTS from synthetic OmniVoice data, enabling on-device fixed-voice synthesis without reference audio.

The paper uses a large voice-cloning model (OmniVoice) as a synthetic data source to train Wayu-Paxa-TTS-Edge, an 82M-parameter fixed-voice Thai TTS student. The model achieves 68.2% Challenge-Set Keyword Accuracy (85.5% of Gemini 3.1), 91.4% pause precision, and CERs of 3.7% on Thai and 1.1% on English. It outperforms its teacher on pause placement and is open-sourced with its evaluation framework.

Hugging Face daily papers·Sep 2, 20:00 UTC · 14d agoAI research

Watch out: Apple timepiece can grab snippets of conversation without both speakers' consent

Apple's Watch Series 12 Live Rewind and Siri Recap transcribe nearby conversations without bystander consent, drawing EFF criticism over all-party-consent laws.

Apple Watch Series 12's Audio Intelligence features on the S11 chip include Live Rewind, which transcribes the last 15 seconds of a conversation after a Digital Crown double-press, processing audio in a Secure Exclave and routing it to a nearby iPhone. Siri Recap generates AI summaries of daily conversations without retaining raw audio or attributing speakers. Apple says an audible chime and visual cue alert bystanders, but privacy advocates including the EFF note that 11 US states require all-party consent for recording and that bystanders have no practical way to opt in or decline.

The Register · Security·Sep 10, 19:50 UTC · 6d agoAI safety & security

Do speech foundation models really learn words?

Researchers show via residualization that later layers of HuBERT and wav2vec 2.0 encode word identity and semantics independently of phonetic content.

The study argues that discriminative ability on words does not imply specialized word representations, since good word discrimination can be explained by phoneme encoding alone. By partialling out phoneme information using residualization, the authors show that later layers of HuBERT and wav2vec 2.0 encode words with reasonable fidelity independently of local phonetic content. Applying this disentanglement approach enhances higher-order linguistic information in word discovery tasks, informing analysis of speech foundation models used for recognition and speech tokens.

arXiv cs.AI / cs.LG / cs.CL·Sep 9, 16:47 UTC · 7d agoAI research

The latest AI news we announced in August 2026

Google's August 2026 AI recap includes launches of Gemini 3.7 Flash, Gemini 3.5 Transcribe, and the Pixel 11 series, plus 1 billion Gemini users.

Google's monthly recap covers the Gemini 3.7 Flash workhorse model for coding and agents, released three weeks after 3.6 Flash at half its per-million-token cost, and the Gemini app surpassing 1 billion monthly users. The Pixel 11 series launched with the Tensor G6 chip running Gemini Nano, alongside Gemini 3.5 Transcribe for real-time speech-to-text and Gemini Omni 1.1 Flash for studio-quality video generation. Other announcements include a free year of Google AI for college students, Gemma's 1 billion downloads, and AI weather forecasts for aviation contrail reduction.

Google · AI·Sep 1, 20:45 UTC · 15d agoAI industry

The Semantic Bottleneck: Leveraging Semantic Representations for Non-Invasive Speech Decoding

Researchers introduced Brain2Semantics2Text, decoding sentence meaning from non-invasive MEG brain recordings via a semantic bottleneck, improving on prior Brain2Text methods.

The paper proposes Brain2Semantics2Text, a non-invasive speech decoding method that maps sentence-level magnetoencephalography (MEG) responses into a semantic embedding space and inverts those embeddings into natural language. Motivated by evidence that high-level semantic representations are distributed across cortex and evolve on slower timescales, the approach targets meaning rather than phonemes or words, avoiding the need for word-level alignment. The authors report improved sentence-level results compared to prior non-invasive Brain2Text methods despite the low signal-to-noise ratio of neural recordings.

Hugging Face daily papers·Sep 8, 20:00 UTC · 8d agoAI research2

Build more natural voice experiences with GPT‑Live‑1 in the API

OpenAI releases GPT-Live-1 in the API, a full-duplex voice model that handles interruptions natively and delegates reasoning to backend models.

OpenAI launched GPT-Live-1 in the API, a single-model full-duplex voice system that listens and speaks simultaneously, replacing chained STT-LLM-TTS architectures. It improves Full Duplex Bench performance by 30 percentage points over GPT-Realtime-2.1 and ranks #1 on Tau3 when paired with GPT-6 Astra at medium reasoning effort. Early partner Speak reported nearly 80% fewer interruptions in language tutoring. The API release costs $0.05 per minute for the front-end voice layer and supports telephony, native ASR transcripts, keyword biasing, and expanded voice and language options.

OpenAI News·Sep 10, 00:00 UTC · 7d agoModel release1

Studying Image Tokenizers as Visual Languages in Unified Multimodal Models

A controlled autoregressive testbed shows validation losses must be analyzed per task, and image tokenizer choice affects joint multimodal text modeling.

Researchers built a pure-autoregressive testbed to study image tokenizers as the 'visual language' of unified multimodal models, tracking task-specific validation losses during multimodal continual pretraining across text, image, text-to-image (T2I), and image-to-text (I2T) prediction. They found that losses exhibit distinct scaling behavior per task and rank tokenizers differently, and that I2T loss over a shared text vocabulary gives a more consistent loss–performance signal than T2I loss. Better reconstruction does not necessarily yield lower task-specific losses or stronger downstream performance, and tokenizer choice can affect text modeling under joint optimization. Case studies examine the discriminator, semantic supervision, and vocabulary size design axes.

Hugging Face daily papers·Sep 7, 20:00 UTC · 9d agoAI research1

Kimsuky Builds Offline AI Stack to Boost Phishing and Automate Malware Development

The Hacker News·Aug 10, 13:21 UTC · Aug 10, 2026Malware

Nuha-Speech: Building General-Purpose Arabic Speech-LLMs

Nuha-Speech initiative builds general-purpose Arabic speech-LLMs using a 1.5M-sample speech QA corpus and fine-tuned Qwen-Omni variants.

The paper introduces Nuha-Speech, an initiative covering dataset construction, model training, and evaluation for Arabic speech large language models. The authors built an Arabic Speech Question-Answering corpus of over 1.5 million training samples and used it for supervised fine-tuning of Qwen-Omni model variants at multiple scales. A tailored evaluation framework with diverse tasks and metrics is designed to assess Arabic speech capabilities under limited resource constraints.

arXiv cs.AI / cs.LG / cs.CL·Sep 10, 17:50 UTC · 6d agoAI research1

AI for everyone in every language

Google says its AI now spans 300+ languages reaching 7 billion people, unveiling Gemini 3.5 Transcribe, Live Translate, and TranslateGemma models.

Google announced its technologies now support more than 300 languages spoken by 7 billion people, 86% of the global population. Gemini 3.5 Live Translate powers real-time spoken translation across 70 languages and 2,000+ language pairs, while Gemini 3.5 Transcribe is its most precise speech-to-text model. Its Universal Speech Model was trained on 12 million hours of audio using cross-lingual transfer learning, and TranslateGemma is a family of lightweight open translation models covering 55 languages that run on-device. Open-data partnerships include WAXAL covering 27 Sub-Saharan African languages and Project Vaani with 30,000+ hours of speech across 109 languages.

Google · AI·Sep 15, 16:00 UTC · 1d agoAI industry

Enhancing Accessibility of Medical Texts through Large Language Model-Driven Plain Language Adaptation

Study shows LLMs with Mixture-of-Agents and QLoRA finetuning effectively simplify medical texts into plain language while preserving content.

The paper evaluates Plain Language Adaptation (PLA) using GPT-4o-mini, Gemini-1.5-pro, and LLaMA in zero-shot and few-shot settings. It compares prompting strategies, QLoRA finetuning across models, and integrates Mixture-of-Agents (MoA) techniques for robustness. Results demonstrate LLM-driven PLA makes healthcare texts more comprehensible while retaining essential content.

arXiv cs.AI / cs.LG / cs.CL·Sep 15, 16:33 UTC · 1d agoAI research

AuK Technical Report: An Open-Source Foundational Model for Speech Generation and Editing

Open-source speech foundation model AuK unifies generation and editing, trained on 1.95 million hours, with distilled AuK-Flash achieving 4.5x speedup.

AuK is an open-source foundational model that unifies speech generation and editing through natural-language instructions and audio context, trained on approximately 3.03 billion instruction-audio instances and 1.95 million hours of supervision across five task families including generation, content editing, and acoustic editing. It combines a multimodal LLM for semantic conditioning, a VAE jointly trained on speech, general audio, and music, and a hybrid rectified-flow Transformer using dual-stream MMDiT blocks followed by unified single-stream DiT blocks. Post-training applies human-feedback preference optimization for editing and reward-based reinforcement learning for generation, and the distilled AuK-Flash performs 4-step inference without classifier-free guidance at a 4.5x wall-clock speedup. Source code and model weights are released.

Hugging Face daily papers·Sep 7, 20:00 UTC · 9d agoModel release2

Kraken: LLM-based Speech-to-Speech Translation via Low-bitrate VQ and Dual-path Source Conditioning

Kraken speech-to-speech translation model builds on Qwen3-8B with low-bitrate vector quantization and source-conditioned vocoding.

Kraken augments a pre-trained Qwen3-8B LLM with speech feature inputs and low-bitrate single-layer vector-quantized tokens trained to reconstruct SSL features. A separate token-to-waveform decoder, Autowave-X, is conditioned on source speech to improve non-linguistic transfer. Training used 150k hours of multilingual and multitask speech data. The model reportedly beats SeamlessM4T-Large v2 and Qwen2.5-Omni in translation quality and speaker/prosody transfer.

arXiv cs.AI / cs.LG / cs.CL·Sep 11, 16:41 UTC · 5d agoAI research1

Climate-ModernBERT: Revisiting Corpus Composition for Domain-Adaptive Continued Pretraining

Climate-ModernBERT domain-adapted encoders reach 76.3 average F1 across nine climate benchmarks, 2.8 points above vanilla ModernBERT-Base.

The authors continue pretraining ModernBERT-Base on three climate corpora - academic text, climate-filtered web data, and synthetic documents - and compare joint mixtures against parameter-space merging of specialized checkpoints. The best model achieves 76.3 average F1 across nine climate NLP benchmarks, a 2.8-point improvement over the vanilla baseline. Academic climate corpora provide the strongest adaptation signal, and parameter-space merging outperforms joint multi-source training while preserving complementary corpus information; all variants are released.

arXiv cs.AI / cs.LG / cs.CL·Sep 7, 17:41 UTC · 9d agoAI research

What Did I Just Say? Self-Listening for Full-Duplex Speech Models

Researchers propose Self-Listening, a full-duplex speech approach feeding realized model speech back as input to improve interruption recovery.

Full-duplex spoken language models can listen and speak simultaneously, but asynchronous text generation, speech synthesis, and playback cause mismatches between what a model believes it said and what the user heard. The paper defines the resulting recovery problem as anchor interruption and proposes Self-Listening, which interleaves user speech, model text, and played speech as input streams. The authors also release AnchorSpeech, a benchmark with homogeneous training and test splits tracking which ordered response items were actually spoken. Experiments show self-listening models achieve better anchoring performance than full-duplex baselines.

Hugging Face daily papers·Sep 3, 20:00 UTC · 13d agoAI research

Motion-Omni: End-to-End Joint Speech and Full-Body Motion for Spoken Dialogue

Researchers introduce Motion-Omni, an end-to-end model generating speech with synchronized full-body motion, responding 5.4x faster than cascade pipelines.

Motion-Omni is an end-to-end framework in which a spoken dialogue model outputs facial expressions and hand, upper-body, and lower-body motion directly from the hidden states that produce speech, replacing two-stage cascade pipelines. Trained on 422,856 quality-ranked pseudo-labeled pairs (1,402 hours) with a Qwen2.5-7B-Instruct backbone, Motion-Omni-Q7 matches its teacher cascade within 2% on reference-free motion metrics, achieves a 2.62% word error rate, and runs faster than real time (RTF=0.78). The authors also release the SwDA-500 dataset and the first public evaluation protocol for stochastic open-ended full-body spoken dialogue.

Hugging Face daily papers·Aug 27, 20:00 UTC · 20d agoAI research1

ECHO: A Matched-Contrast Benchmark for Context-Sensitive Turn-Taking in Full-Duplex Dialogue

ECHO benchmark pairs identical-overlap Chinese dialogue examples with contrasting contexts, revealing most full-duplex systems bias toward yielding the floor.

ECHO is a paired diagnostic benchmark for Chinese full-duplex turn-taking that matches examples with identical overlap transcripts but contrasting preceding multi-turn contexts, one requiring Yield and the other Keep. It also includes off-talk examples for diagnosing unnecessary yielding and introduces pair accuracy, which grants no credit to constant-action policies. Experiments across multiple full-duplex systems show most exhibit a pronounced Yield bias, performing substantially better on interruptions than backchannels. The benchmark and metadata will be publicly released.

arXiv cs.AI / cs.LG / cs.CL·Sep 15, 15:58 UTC · 1d agoAI research

Verifiable by Construction: Claim-Level Evaluation of Verbatim Citation in Clinical Question Answering

Evaluation of twelve LLMs on 222 clinical questions shows verbatim quotes rarely substantiate claims; claude-opus-5 fully substantiates only 37.1%.

The authors build a standardized harness over four clinical practice guidelines and evaluate twelve LLMs on 222 synthetic clinical questions, measuring citation attachment, verbatim quote production, and claim substantiation. Most models attach verbatim quotes to over 90% of claims from prompting alone, though lightweight models like claude-haiku-4.5 struggle. Quotes frequently fail to substantiate claims: claude-opus-5 quotes 98.0% of claims but fully substantiates only 37.1%, exposing a capability gap for verifiable clinical QA.

arXiv cs.AI / cs.LG / cs.CL·Sep 14, 17:53 UTC · 2d agoAI research

VoT: Vision-of-Thought for Unified Multimodal Representation Alignment

Researchers propose Vision-of-Thought (VoT), a discrete visual-planning token layer between VLMs and diffusion transformers improving text-to-image semantic alignment.

VoT introduces a discrete visual-thinking layer between vision-language models and diffusion transformers, letting the VLM act as a multimodal planner that emits tokens describing objects and layouts before pixel generation. A specialized VoT tokenizer is trained with VLM alignment, feature reconstruction, and vector-quantization losses. Experiments show improved semantic alignment and a structured, interpretable interface for controllable generation.

arXiv cs.AI / cs.LG / cs.CL·Sep 7, 17:53 UTC · 9d agoAI research

Training and Finetuning Multi-Vector Embedding Models with Sentence Transformers

Hugging Face published a tutorial on training and finetuning multi-vector embedding models using the Sentence Transformers library.

Hugging Face's blog walks through training and finetuning multi-vector embedding models with Sentence Transformers. Multi-vector approaches store multiple vectors per document to support late-interaction retrieval. The post is a practical guide for developers building retrieval pipelines with the library.

Hugging Face Blog·Aug 26, 00:00 UTC · 22d agoAI tools & infra1

BreezeBlue/Breeze-TTS-2 — new model trending #19 on Hugging Face

BreezeBlue open-weights Breeze TTS 2, a bilingual text-to-speech model it ranks #1 among open-weight models on the Artificial Analysis TTS leaderboard.

BreezeBlue released open weights and Apache 2.0-licensed PyTorch inference code for Breeze TTS 2 on 2026-08-25. The text-to-speech model supports English and Chinese, voice cloning, reference-free voice design, voice direction, and inline vocal events like (laugh) and (sigh). Reported performance includes #1 open-weight ranking on the Artificial Analysis Elo leaderboard, under 40 ms time-to-first-audio, a 0.32 real-time factor on an NVIDIA H100, and about 7.7 GiB GPU memory for eager inference.

Hugging Face trending models·Aug 25, 16:39 UTC · 22d agoModel release

Studying Image Tokenizers as Visual Languages in Unified Multimodal Models

A controlled pure-autoregressive testbed shows task-specific validation losses rank image tokenizers differently, with I2T loss the most consistent signal.

Researchers built a controlled pure-autoregressive testbed and tracked task-specific validation losses during multimodal continual pretraining across text, image, text-to-image (T2I), and image-to-text (I2T) prediction. They find losses should be analyzed per task because they exhibit distinct scaling behavior and rank tokenizers differently, and that the loss-performance relationship depends on the predicted token space. I2T loss, computed over a shared text vocabulary, correlates consistently with both generation and visual understanding performance after supervised finetuning. Case studies revisit the discriminator, semantic supervision, and vocabulary size as tokenizer design axes.

arXiv cs.AI / cs.LG / cs.CL·Sep 8, 17:57 UTC · 8d agoAI research1

Build Low-Latency Multilingual Voice Agents: Open Weights & Full Deployment Control with NVIDIA Magpie TTS

NVIDIA's Magpie TTS open-weight multilingual speech model enables low-latency voice agents with full deployment control.

Hugging Face's blog highlights NVIDIA Magpie TTS, an open-weights multilingual text-to-speech model designed for building low-latency voice agents. The open licensing gives developers full deployment control, allowing self-hosted multilingual speech for agentic applications. The post walks through building voice agents with the model.

Hugging Face Blog·Aug 10, 16:25 UTC · Aug 10, 2026Model release

FLAT: Resampling Image and Text into 1D Flexible-Length Aligned Transmodal Tokens for Retrieval and Generation

FLAT jointly trains a multimodal encoder with text-to-image and image-to-text decoders, producing flexible-length tokens that hit 83.1 GenEval on T2I after fine-tuning.

FLAT (Flexible-Length Aligned Transmodal representations) is a pre-training framework that jointly optimizes a shared multimodal encoder with T2I and I2T decoders, combining contrastive alignment with bidirectional cross-modal generative objectives. It maps visual and textual inputs into a unified continuous 1D sequence space and uses nested dropout over prefix-K tokens for dynamic output lengths. A single pre-training stage supports cross-modal retrieval and generation (71.1 GenEval), with task-specific fine-tuning reaching 83.1 GenEval on T2I, 40.5 BLEU-4 and 138.6 CIDEr on MS-COCO captioning, and strong Recall@5 on MS-COCO and Flickr30K.

Hugging Face daily papers·Sep 14, 20:00 UTC · 2d agoAI research1

Technical Manual for a Toolkit for Measuring Contextual Individuation in Transformer Language Models

An open methodology toolkit measures whether transformer language models contextualize fixed word forms across domains using bridge forms and layer-wise silhouette analysis.

The manual documents an open toolkit built around 'bridge forms' - identical written words recurring across two or more subject domains with a different sense in each - to test whether transformer language models individuate word occurrences by context beyond the embedding layer. It covers declarative specification of bridge forms, Wikipedia corpus acquisition, occurrence localization, layer-wise representation extraction, domain-pairwise silhouette measurement, and visualization, justifying each choice against failure modes such as sense contamination and subword-tokenization misalignment. It is a methodological and implementation reference and reports no empirical results.

arXiv cs.AI / cs.LG / cs.CL·Sep 4, 16:38 UTC · 12d agoAI research

Measuring benchmark optimization in speech recognition

Hugging Face examines how much speech recognition systems overfit benchmarks and how to measure benchmark optimization in ASR.

A Hugging Face post on measuring benchmark optimization in automatic speech recognition, analyzing how model improvements on benchmarks reflect genuine capability gains versus overfitting. It is evaluation methodology research with no direct security impact.

Hugging Face Blog·Aug 21, 00:00 UTC · 27d agoAI research

The 2026 PNPL Competition: Word Classification and Efficient Cross-Subject Generalisation in LibriBrain100

The 2026 PNPL competition releases LibriBrain100, a MEG speech dataset with 32 extra subjects, targeting word classification and cross-subject BCI generalization.

The 2025 PNPL competition on non-invasive speech decoding from MEG achieved F1-macro scores of 95.6% for speech detection and 73.6% for phoneme classification, built on LibriBrain's ~50 hours of single-subject data. The 2026 edition extends this with LibriBrain100, adding 32 subjects (~40 minutes each) plus ~80 hours of within-subject data. Two tracks target within-subject word classification at scale and cross-subject generalization with subject-specific fine-tuning shrinking from ~40 to ~20 to ~10 minutes, aiming at clinically feasible non-invasive BCIs for people with profound paralysis.

Hugging Face daily papers·Sep 2, 20:00 UTC · 14d agoAI research

MP-Bench: Evaluating Voice Agents as a Multiparty Conversation Participant

MP-Bench is the first benchmark for voice agents in multiparty conversations, finding real-time agents near chance on turn-taking.

MP-Bench is the first benchmark designed to objectively evaluate conversational speech systems as active participants in multi-party conversations. It assesses agents on turn-taking awareness and response appropriateness, with comprehension-based question-answering as a complementary evaluation. Benchmarking 12 voice agents shows real-time agents score at or below 22% on multiparty comprehension and remain near chance on multiparty turn-taking.

arXiv cs.AI / cs.LG / cs.CL·Sep 11, 17:11 UTC · 5d agoAI research2

Putting sign language AI into users’ hands

Google DeepMind introduced SL2T, a sign-language-to-text model powering new accessibility features for Deaf and hard-of-hearing users.

Google DeepMind announced SL2T, a sign-language-to-text model described as a breakthrough for sign language understanding. The model powers new sign language features aimed at Deaf and hard-of-hearing users. Details on benchmarks and model size were not provided in the announcement.

Google DeepMind·Aug 12, 14:01 UTC · Aug 12, 2026Model release

I Am No One: Style-Aware Paraphrasing for Text Anonymization

Prompt-driven style-aware paraphrasing with LLMs cuts authorship attribution F1 by 60-70% while preserving content quality.

The paper proposes a style-aware, prompt-driven anonymization approach using pretrained LLMs to build compact stylistic profiles from minimal samples and rewrite text to suppress identifiable style markers while preserving meaning. It addresses stylometric re-identification risks in anonymized text, including ASR transcripts of meetings and call-center calls where leakage persists after acoustic anonymization. Across blog and review datasets, the approach reduces authorship attribution F1 by 60-70%, substantially outperforming both DP-based and non-DP baselines while maintaining readability.

arXiv cs.CR·Sep 11, 02:07 UTC · 6d agoResearch

Building a Production Greek-English Speech Recognizer

Engineering report details Sophea, a production Greek-English ASR reaching 4.26% WER on public English sets via ROVER ensemble and data-pipeline calibration.

Across 23 training iterations, two architectures, and nine production gates, no single data composition passed all gates; a three-model ROVER ensemble reached 9 of 9 gates and cut overlapping-speech WER from 53.35% to 37.87%. Calibrating an audio-quality filter against in-domain anchors reduced discarded scored Greek audio from 98.7% to 10.6%, and a pre-registered ablation traced a hallucination defect to one training-data package. The sophea/asr-k1 preview arbiter lists 4.26% average WER on eight public English test sets and 25.88% WER on live Greek noisy traffic; no weights or training data are released.

Hugging Face daily papers·Sep 10, 20:00 UTC · 6d agoAI research

Google Releases Gemini 3.8 Live and 3.8 Live Extended Thinking for Production Grade Voice Agents

Google launches Gemini 3.8 Live and Extended Thinking speech-to-speech models for production voice agents, topping speech-to-speech benchmarks.

Google released Gemini 3.8 Live and Gemini 3.8 Live Extended Thinking, native speech-to-speech models for real-time voice agents, available hosted via the Gemini Live API and AI Studio. Extended Thinking ranks #1 on Artificial Analysis' Speech-to-Speech Quality Index with 82.6, scores 68.6% on τ-Voice, 35.1% on Sierra's τ-Voice-banking, and 97.7% on Big Bench Audio. The models support asynchronous function calling, near-real-time visual context, alphanumeric precision, and 97 languages, priced at $0.005/min audio input and $0.018/min audio output. All generated audio carries Google DeepMind's imperceptible SynthID watermark.

MarkTechPost·Sep 15, 20:53 UTC · 1d agoModel release 3 sources

LACE: Layer-Wise Compression for Dynamic Frame Rate Codecs

LACE introduces layer-wise compression for dynamic frame rate audio codecs, cutting sequence lengths and speeding TTS inference while preserving quality.

LACE (Layer-Adaptive Codec Encoding) applies an independent compression step at each quantization layer of a neural audio codec, enabling layer-specific segmentation boundaries instead of shared ones. Union alignment and boundary anchor mechanisms keep durations consistent for downstream text-to-speech. On LibriTTS, LACE achieves a better rate-quality tradeoff than prior dynamic frame rate codecs and improves TTS inference efficiency at competitive synthesis quality. Code is released in the ESPnet3 codec recipe.

arXiv cs.AI / cs.LG / cs.CL·Sep 15, 17:46 UTC · 1d agoAI research1

Rosetta at AlexandriaX-2026: LoRA-Adapted NileChat for Context-Aware Dialectal Arabic Dialogue Translation

Rosetta ranks 4th and 5th in AlexandriaX-2026 dialectal Arabic dialogue translation using a LoRA adapter on NileChat-3B, finding limited pretraining benefit.

The Rosetta system for the AlexandriaX-2026 shared task fine-tunes a LoRA adapter on NileChat-3B for context-aware English-to-dialectal Arabic dialogue translation. The adapter was additionally pretrained on MADAR and PADIC dialect corpora for the unconstrained track. It achieved spBLEU 26.10 (4th, constrained) and 25.09 (5th, unconstrained). External dialect pretraining improved only two of thirteen dialects while slightly degrading overall performance, indicating negative transfer.

arXiv cs.AI / cs.LG / cs.CL·Sep 9, 16:15 UTC · 7d agoAI research1