ZeroHour

Search: “sign-language-to-text”

31 stories

Smart search ranks by meaning as well as keywords (one row per story, last 45 days).

Putting sign language AI into users’ hands

Google DeepMind introduced SL2T, a sign-language-to-text model powering new accessibility features for Deaf and hard-of-hearing users.

Google DeepMind announced SL2T, a sign-language-to-text model described as a breakthrough for sign language understanding. The model powers new sign language features aimed at Deaf and hard-of-hearing users. Details on benchmarks and model size were not provided in the announcement.

Google DeepMind · Aug 12, 2026Model release

Building AI to accelerate science and improve lives

Google highlights AI-for-science advances: AlphaGenome Atlas mapping 9 billion genetic variants, WeatherNext 3 weather model, and global health AI tools.

Google detailed AI advances across science and health, including AlphaGenome Atlas, which mapped all 9 billion possible single-letter genetic changes in the human genome and was made openly available. WeatherNext 3 delivers 50% more accurate precipitation forecasts a day or more ahead and is already in products. AlphaFold is used by 4 million researchers in 190 countries, TB chest X-ray screening has processed 25,000+ scans across six nations, and the diabetic retinopathy model has supported 1.15 million screenings. Google also released its AI & Economy ATLAS global usage insights.

Google · AI · 1d agoAI industry

Training and Finetuning Multi-Vector Embedding Models with Sentence Transformers

Hugging Face published a tutorial on training and finetuning multi-vector embedding models using the Sentence Transformers library.

Hugging Face's blog walks through training and finetuning multi-vector embedding models with Sentence Transformers. Multi-vector approaches store multiple vectors per document to support late-interaction retrieval. The post is a practical guide for developers building retrieval pipelines with the library.

Hugging Face Blog · 22d agoAI tools & infra1

Enhancing Accessibility of Medical Texts through Large Language Model-Driven Plain Language Adaptation

Study shows LLMs with Mixture-of-Agents and QLoRA finetuning effectively simplify medical texts into plain language while preserving content.

The paper evaluates Plain Language Adaptation (PLA) using GPT-4o-mini, Gemini-1.5-pro, and LLaMA in zero-shot and few-shot settings. It compares prompting strategies, QLoRA finetuning across models, and integrates Mixture-of-Agents (MoA) techniques for robustness. Results demonstrate LLM-driven PLA makes healthcare texts more comprehensible while retaining essential content.

arXiv cs.AI / cs.LG / cs.CL · 1d agoAI research

Studying Image Tokenizers as Visual Languages in Unified Multimodal Models

A controlled autoregressive testbed shows validation losses must be analyzed per task, and image tokenizer choice affects joint multimodal text modeling.

Researchers built a pure-autoregressive testbed to study image tokenizers as the 'visual language' of unified multimodal models, tracking task-specific validation losses during multimodal continual pretraining across text, image, text-to-image (T2I), and image-to-text (I2T) prediction. They found that losses exhibit distinct scaling behavior per task and rank tokenizers differently, and that I2T loss over a shared text vocabulary gives a more consistent loss–performance signal than T2I loss. Better reconstruction does not necessarily yield lower task-specific losses or stronger downstream performance, and tokenizer choice can affect text modeling under joint optimization. Case studies examine the discriminator, semantic supervision, and vocabulary size design axes.

Hugging Face daily papers · 9d agoAI research1

Training a coding model to paint watercolours with TRL and OpenEnv

Hugging Face tutorial trains a coding model with TRL and OpenEnv to paint watercolours through generated code.

A Hugging Face blog walkthrough uses the TRL reinforcement learning library and the OpenEnv environment framework to train a coding model. The target task is generating code that produces watercolour-style drawings, serving as a hands-on reinforcement learning training example. No article body was available in the feed, so specifics are limited to the title.

Hugging Face Blog · 14d agoAI tools & infra1

Technical Manual for a Toolkit for Measuring Contextual Individuation in Transformer Language Models

An open methodology toolkit measures whether transformer language models contextualize fixed word forms across domains using bridge forms and layer-wise silhouette analysis.

The manual documents an open toolkit built around 'bridge forms' - identical written words recurring across two or more subject domains with a different sense in each - to test whether transformer language models individuate word occurrences by context beyond the embedding layer. It covers declarative specification of bridge forms, Wikipedia corpus acquisition, occurrence localization, layer-wise representation extraction, domain-pairwise silhouette measurement, and visualization, justifying each choice against failure modes such as sense contamination and subword-tokenization misalignment. It is a methodological and implementation reference and reports no empirical results.

arXiv cs.AI / cs.LG / cs.CL · 12d agoAI research

Intelligent transcription with Gemini 3.5 Transcribe

Google DeepMind launched Gemini 3.5 Transcribe, a speech-to-text model offering more intelligent transcription as part of the Gemini family.

Google DeepMind announced Gemini 3.5 Transcribe, a new speech-to-text model described as delivering more intelligent transcription. The blog post provides limited technical detail in the available text, with no benchmarks or model sizes given. The release adds a dedicated audio transcription model to the Gemini family.

Google DeepMind · 21d agoModel release

Right Tool, Right Job: Native-Language Evaluation, Tokenizer Sensitivity, and Methodological Findings from a French-Only BabyLM

French BabyLM entry METRON-FR (125M GPT-2, 92.47M words) shows tokenizer artifacts dominate child-scale zero-shot evaluation; proposes standard diagnostics.

METRON-FR is a 125M-parameter GPT-2 pretrained on 92.47M French words, submitted to the BabyLM 2026 Strict track, scoring 85.97% on the native Quebec-French QFrBLiMP benchmark and 62.80% on the BabyLM-weighted leaderboard. A cross-lingual GLUE protocol combining French task-data translation with rank-16 LoRA shows relational tasks gain while world-knowledge tasks regress. Bilingual Lexicon Induction reaches p@1 of 68.84%, 18x above chance, and ablations show single-token zero-shot scoring is dominated by tokenizer and template artifacts at child scale.

arXiv cs.AI / cs.LG / cs.CL · 1d agoAI research

Studying Image Tokenizers as Visual Languages in Unified Multimodal Models

A controlled pure-autoregressive testbed shows task-specific validation losses rank image tokenizers differently, with I2T loss the most consistent signal.

Researchers built a controlled pure-autoregressive testbed and tracked task-specific validation losses during multimodal continual pretraining across text, image, text-to-image (T2I), and image-to-text (I2T) prediction. They find losses should be analyzed per task because they exhibit distinct scaling behavior and rank tokenizers differently, and that the loss-performance relationship depends on the predicted token space. I2T loss, computed over a shared text vocabulary, correlates consistently with both generation and visual understanding performance after supervised finetuning. Case studies revisit the discriminator, semantic supervision, and vocabulary size as tokenizer design axes.

arXiv cs.AI / cs.LG / cs.CL · 8d agoAI research1

Bad Likert Judge: A Novel Multi-Turn Technique to Jailbreak LLMs by Misusing Their Evaluation Capability

Unit 42 details the Bad Likert Judge multi-turn jailbreak that abuses LLMs' evaluation capability, raising attack success rates over 60% across six frontier models.

Palo Alto Networks Unit 42 describes the Bad Likert Judge technique, a multi-turn jailbreak that asks a target LLM to act as a Likert-scale judge scoring the harmfulness of example responses. The highest-rated example in each scale can carry harmful content, bypassing the model's internal guardrails. Testing across six state-of-the-art text-generation LLMs showed an average attack success rate increase of more than 60% versus plain attack prompts, with tested models anonymized. The technique targets edge cases rather than typical use, and the article positions the work as guidance for defenders on potential jailbreak risks.

Palo Alto Unit 42 · Aug 17, 2026AI safety & security

LexFlip: A Dissociation Diagnostic for Legal Meaning Preservation Metrics

LexFlip releases 373 minimal perturbations of Quebec statutory French that reverse legal force while preserving tokens, exposing weaknesses in embedding-based meaning preservation metrics.

LexFlip provides 373 minimal perturbations of Quebec statutory French that reverse legal force while preserving 0.93 of tokens, creating dissociation items that break monotone token-overlap metric validation. The seven embedding and BERTScore metrics tested register only 0.022-0.039 of their identical-to-unrelated range on these edits, versus 0.670 for bidirectional NLI. Against FrJudge, with a measured human ceiling of r=0.597, a bare length feature outscores every semantic metric tested.

arXiv cs.AI / cs.LG / cs.CL · 12d agoAI research

New AI Attack Hides Malicious Instructions in Normal-Looking Text to Evade Safety Filters

Check Point researchers show crafted prose hides policy-violating instructions that bypass all tested LLM gatekeepers, including GPT-4o mini and Llama Guard 3.

A new prompt-crafting technique embeds malicious payloads inside grammatical, natural-looking text without Base64, invisible Unicode, or obvious encodings, defeating lightweight pre-screening gatekeepers. In testing, all four evaluated gatekeeper models—gpt-4o-mini-2024-07-18, gpt-oss-safeguard:20b, claude-3-haiku-20240307, and llama-guard3:8b—classified the crafted wrappers as safe at a 100% bypass rate across 23 obfuscated prompts. GPT-5 Thinking in high-reasoning mode recovered and acted on the hidden instruction in 17 of 18 tests (~94.4%), often spending over a minute and multiple Python executions. Researchers recommend paraphrasing untrusted input, hardening gatekeeper policies, and applying defense-in-depth controls for agentic deployments.

GBHackers · 6d agoAI safety & security 2 sources

Register Tokens for Bounded-State Reasoning in Diffusion Language Models

Register tokens let diffusion language models like LLaDA and Dream carry reasoning state across cleared chunks, gaining up to 19.5 points on code.

Researchers propose register tokens: dedicated fixed-position tokens whose continuous hidden states are trained to carry reasoning progress across generation chunks in masked diffusion language models. After decoding and clearing a chunk, the model continues from the prompt and the carried register state instead of retaining earlier text. On LLaDA and Dream, registers outperform discrete-text carry on every benchmark, with gains up to 8.5 points on math and 19.5 points on code. Registers are especially effective for bounded code generation and can be further refined with reinforcement learning on long-horizon reasoning tasks.

Hugging Face daily papers · 3d agoAI research

Measuring Language Transfer in Robot Policies: Adding Greek to a Cosmos3 Vision-Language-Action Policy

Researchers added Greek to the Cosmos3 vision-language-action policy using only machine-rephrased instructions, finding bilingual training reaches roughly two fifths of English performance.

The paper studies localizing the open Cosmos3 vision-language-action robot policy to Greek without architectural changes, using machine-rephrased instructions only. Bilingual training yields a consistent 6.7-7.1 point margin over controls on a 90-task, three-seed evaluation suite, while Greek-only training gains at most 2.7 points. Several common evaluation instruments, including color-histogram metrics and single-goal benchmarks, produced false conclusions, and results were dominated by seed variation. The authors recommend building guaranteed-null baselines and replicating low-resource-language results across seeds.

Hugging Face daily papers · 10d agoAI research

Augustinian BabyLM: What Ostensive Definition Can and Cannot Teach a Small Language Model

Study shows visually grounded token embeddings in a small masked LM persist through training and improve object-property knowledge, but escape standard BabyLM benchmarks.

The paper implements ostensive definition for a small DeBERTa masked language model trained on 10M words, seeding visually grounded tokens with embeddings derived from labeled image regions before training. Visual initialization leaves a persistent, seed-replicated advantage on object-property knowledge (COMPS) and a corpus-tailored Visual-Property Swap benchmark covering color, material, size, and shape, but has no effect on most BabyLM grammar benchmarks. Synthetic grounding of previously unseeded words causally transfers the advantage to exactly those words.

arXiv cs.AI / cs.LG / cs.CL · 6d agoAI research

VoT: Vision-of-Thought for Unified Multimodal Representation Alignment

Researchers propose Vision-of-Thought (VoT), a discrete visual-planning token layer between VLMs and diffusion transformers improving text-to-image semantic alignment.

VoT introduces a discrete visual-thinking layer between vision-language models and diffusion transformers, letting the VLM act as a multimodal planner that emits tokens describing objects and layouts before pixel generation. A specialized VoT tokenizer is trained with VLM alignment, feature reconstruction, and vector-quantization losses. Experiments show improved semantic alignment and a structured, interpretable interface for controllable generation.

arXiv cs.AI / cs.LG / cs.CL · 9d agoAI research

microsoft/VibeVoice-ASR-Streaming-7B — new model trending #27 on Hugging Face

Microsoft released VibeVoice-ASR-Streaming-7B, an open streaming ASR model with speaker attribution, custom hotwords, and support for 10 languages under MIT license.

Microsoft Research released VibeVoice-ASR-Streaming-7B on Hugging Face, a unified streaming speech recognition model that continuously transcribes who said what as speech arrives. The 7B model supports customized hotwords for domain-specific terms and 10 languages including Chinese, English, French, German, Italian, Japanese, Korean, Portuguese, Russian, and Spanish. Code is available at github.com/microsoft/VibeVoice with a live demo, and a technical report is on arXiv (2609.02812). The model is licensed under MIT.

Hugging Face trending models · 14d agoModel release2

The Semantic Bottleneck: Leveraging Semantic Representations for Non-Invasive Speech Decoding

Researchers introduced Brain2Semantics2Text, decoding sentence meaning from non-invasive MEG brain recordings via a semantic bottleneck, improving on prior Brain2Text methods.

The paper proposes Brain2Semantics2Text, a non-invasive speech decoding method that maps sentence-level magnetoencephalography (MEG) responses into a semantic embedding space and inverts those embeddings into natural language. Motivated by evidence that high-level semantic representations are distributed across cortex and evolve on slower timescales, the approach targets meaning rather than phonemes or words, avoiding the need for word-level alignment. The authors report improved sentence-level results compared to prior non-invasive Brain2Text methods despite the low signal-to-noise ratio of neural recordings.

Hugging Face daily papers · 8d agoAI research2

Robust Coverless Linguistic Steganography via Sentence Embedding Space with Global Resynchronization

Researchers propose a coverless steganographic framework encoding messages as hierarchical clustering paths in sentence embedding space with a Global Resynchronization Mechanism for robustness.

An arXiv paper proposes encoding secret messages as hierarchical clustering paths in the sentence embedding space rather than token space, improving decoding stability against word- and sentence-level textual perturbations. A Global Resynchronization Mechanism (GRM) reframes variable-length bitstreams as discrete symbols anchored to semantic subspaces to prevent bit-slippage. Experiments show substantial robustness improvements while maintaining embedding capacity and resistance to statistical analysis.

arXiv cs.CR · 12d agoResearch

Multi-Vector (Late Interaction) Embedding Models with Sentence Transformers

Hugging Face details building and using multi-vector late-interaction embedding models with Sentence Transformers for retrieval workloads.

Hugging Face published a guide on multi-vector, late-interaction embedding models (ColBERT-style) supported through Sentence Transformers. The post covers how practitioners can build and use these models for retrieval and RAG pipelines. It is a developer tooling and technique write-up, not a security advisory.

Hugging Face Blog · Aug 18, 2026AI tools & infra1

Building and Evaluating Fixed-Voice Thai TTS from Synthetic Speech

Researchers distill a compact 82M-parameter Thai TTS from synthetic OmniVoice data, enabling on-device fixed-voice synthesis without reference audio.

The paper uses a large voice-cloning model (OmniVoice) as a synthetic data source to train Wayu-Paxa-TTS-Edge, an 82M-parameter fixed-voice Thai TTS student. The model achieves 68.2% Challenge-Set Keyword Accuracy (85.5% of Gemini 3.1), 91.4% pause precision, and CERs of 3.7% on Thai and 1.1% on English. It outperforms its teacher on pause placement and is open-sourced with its evaluation framework.

Hugging Face daily papers · 14d agoAI research

How Far Can Synthetic Data Take Thai OCR?

Synthetic-only training adapts PaddleOCR-VL into Wayu-Paxa-OCR-Zero, cutting Thai printed-page CER from 6.64% to 1.24% without real labels.

The study disentangles which factors of synthetic OCR data transfer to real Thai documents, finding typeface diversity, 2D structure, and real handwriting glyphs matter most. Using 45,723 synthetic pages, the authors adapt the 0.9B-parameter PaddleOCR-VL-1.6 into Wayu-Paxa-OCR-Zero, reducing median CER from 6.64% to 1.24% on printed pages and from 74.87% to 20.55% on handwriting. The model outperforms Typhoon OCR v1 7B on all five evaluation sets.

Hugging Face daily papers · 14d agoAI research

The Open ASR Leaderboard Adds Its First Global South Language

Hugging Face's Open ASR Leaderboard added its first Global South language, expanding speech-recognition benchmark coverage.

The Hugging Face Open ASR Leaderboard, a community benchmark tracker for automatic speech recognition models, added its first Global South language. This expands evaluation coverage beyond the high-resource languages the leaderboard previously tracked. No further details were available in the announcement text.

Hugging Face Blog · 20d agoAI research

Cohere Releases North Small Translate: A 218B MoE Translation Model That Scores 83.6 on WMT26 Across 50 Languages

Cohere released North Small Translate, an open-weight 218B MoE (25B active) translation model scoring 83.6 on WMT26 across 50 languages.

Cohere and Cohere Labs released North Small Translate, a decoder-only sparse Mixture-of-Experts translation model with 218B total and 25B active parameters, 128 experts with 8 activated per token plus shared experts, and 16K-token input and output context. In Cohere's vendor-reported WMT26 evaluation, judged by GPT-5.6-Sol, it scores 83.6 averaged across 50 languages (84.36 in an agentic multi-pass mode), ahead of DeepL NextGen (81.37), Qwen 3.5 397B A17B (81.56), GLM 5.2 (76.50), and Google Translate (68.20). The model was built with RWS's Language Weaver team, post-trained specifically for translation, and reports 112 output tokens per second versus 81 for Gemma 4 31B, with long-document xCOMET-XL scores of 48.9 versus 21.3 for Google Translate. It is available free on Cohere's Chat V2 API until rate limits, with three self-hosting checkpoints including a 4-bit NVFP4 variant running on 1x B200 or 2x H100.

MarkTechPost · 6d agoModel release

PANORAMA: Panoptic Grounded Captioning via Mask Proposal Selection

PANORAMA grounds vision-language caption phrases in pixel-level masks via mask proposal selection, achieving state-of-the-art grounding on the new PanoCaps benchmark.

The paper introduces panoptic grounded captioning, requiring VLMs to describe foreground and background regions while grounding each phrase with pixel-level masks. Contributions include PanoCaps, a human-annotated benchmark built from panoptic segmentation datasets with entity-level image-text alignment, a phrase-mask matching protocol, and a generalized Panoptic Quality metric. PANORAMA conditions a pretrained segmenter on contextualized phrase representations to select masks, achieving the best overall grounding on PanoCaps; code, data, and models are released.

BreezeBlue/Breeze-TTS-2 — new model trending #19 on Hugging Face

BreezeBlue open-weights Breeze TTS 2, a bilingual text-to-speech model it ranks #1 among open-weight models on the Artificial Analysis TTS leaderboard.

BreezeBlue released open weights and Apache 2.0-licensed PyTorch inference code for Breeze TTS 2 on 2026-08-25. The text-to-speech model supports English and Chinese, voice cloning, reference-free voice design, voice direction, and inline vocal events like (laugh) and (sigh). Reported performance includes #1 open-weight ranking on the Artificial Analysis Elo leaderboard, under 40 ms time-to-first-audio, a 0.32 real-time factor on an NVIDIA H100, and about 7.7 GiB GPU memory for eager inference.

Hugging Face trending models · 22d agoModel release

Do speech foundation models really learn words?

Researchers show via residualization that later layers of HuBERT and wav2vec 2.0 encode word identity and semantics independently of phonetic content.

The study argues that discriminative ability on words does not imply specialized word representations, since good word discrimination can be explained by phoneme encoding alone. By partialling out phoneme information using residualization, the authors show that later layers of HuBERT and wav2vec 2.0 encode words with reasonable fidelity independently of local phonetic content. Applying this disentanglement approach enhances higher-order linguistic information in word discovery tasks, informing analysis of speech foundation models used for recognition and speech tokens.

arXiv cs.AI / cs.LG / cs.CL · 7d agoAI research

FLAT: Resampling Image and Text into 1D Flexible-Length Aligned Transmodal Tokens for Retrieval and Generation

FLAT jointly trains a multimodal encoder with text-to-image and image-to-text decoders, producing flexible-length tokens that hit 83.1 GenEval on T2I after fine-tuning.

FLAT (Flexible-Length Aligned Transmodal representations) is a pre-training framework that jointly optimizes a shared multimodal encoder with T2I and I2T decoders, combining contrastive alignment with bidirectional cross-modal generative objectives. It maps visual and textual inputs into a unified continuous 1D sequence space and uses nested dropout over prefix-K tokens for dynamic output lengths. A single pre-training stage supports cross-modal retrieval and generation (71.1 GenEval), with task-specific fine-tuning reaching 83.1 GenEval on T2I, 40.5 BLEU-4 and 138.6 CIDEr on MS-COCO captioning, and strong Recall@5 on MS-COCO and Flickr30K.

Hugging Face daily papers · 2d agoAI research1

AuK Technical Report: An Open-Source Foundational Model for Speech Generation and Editing

Open-source speech foundation model AuK unifies generation and editing, trained on 1.95 million hours, with distilled AuK-Flash achieving 4.5x speedup.

AuK is an open-source foundational model that unifies speech generation and editing through natural-language instructions and audio context, trained on approximately 3.03 billion instruction-audio instances and 1.95 million hours of supervision across five task families including generation, content editing, and acoustic editing. It combines a multimodal LLM for semantic conditioning, a VAE jointly trained on speech, general audio, and music, and a hybrid rectified-flow Transformer using dual-stream MMDiT blocks followed by unified single-stream DiT blocks. Post-training applies human-feedback preference optimization for editing and reward-based reinforcement learning for generation, and the distilled AuK-Flash performs 4-step inference without classifier-free guidance at a 4.5x wall-clock speedup. Source code and model weights are released.

Hugging Face daily papers · 9d agoModel release2

Amazon Prime Video’s new AI tech matches lips to dubbed audio

Amazon Prime Video launched AI lip-sync dubbing aligning actors' mouth movements with human-dubbed audio, starting with the English dub of Maxton Hall.

Prime Video's new feature combines AI and visual effects technologies to make lip movements match translated speech, currently available for seasons 1-2 of the German series Maxton Hall worldwide and planned for additional titles, with season 3 arriving December 9. Amazon previously piloted AI-aided dubbing in English and Spanish across 12 movies and shows last year. Meta and YouTube have recently launched similar AI auto-dubbing features with optional lip sync for creators.

The Verge · AI · 7d agoAI industry