Paper measures mismatch on-policy distillation can tolerate
Model-specific FP8 recipes kept quality and fit at least 1.93x as many KV-cache tokens as BF16.
Stories per day · 30 days
1 RSS
Relatedkv-cache3quantization3on-policy-distillation2abliteration1amd1deepseek1flashattention1hopper1huggingface1inference1language-models1llama1mi355x1open-weights1reinforcement-learning1uncensored-model1zyphra1
11 stories, newest first · one card per story
Stories tagged #fp8
Zyphra ZYFA FP8 FlashAttention for AMD MI355X up to 1.76x faster
ZYFA achieves up to 1.76x faster performance using FP8 FlashAttention on AMD MI355X.
DealignAI posts uncensored FP8 DeepSeek-V4.1-Flash-1017 on Hugging Face
Publisher says refusal was removed by a weight edit, not fine-tuning, and its MMLU score fell 3.86 points.
6
FuTransformer speeds Llama-style inference with FP8 prefill and INT4 decode
On one H100, prefill reaches 1.29 PFLOP/s and is up to 1.48x faster than compiled PyTorch FP8.
Towards Full Pipeline FP8 Reinforcement Learning for LLMs
A paper on Hugging Face's daily papers describes FP8 reinforcement learning challenges and proposes a solution to fix training instability in LLMs.
VC-Attention: Value Smoothing and Softmax Casting for Low-bit Attention
VC-Attention is a training-free low-bit attention method for diffusion transformers, achieving 1.46-1.59x kernel speedups on datacenter GPUs with higher fidelity.
Attention Quantization for Tabular Foundation Models
FP8 quantization of attention queries, keys, and values speeds tabular foundation model inference up to 1.7x with no accuracy loss.
DeepSeek v4.1 Flash Uncensored
Hugging Face user dealignai published an uncensored FP8-quantized variant of DeepSeek v4.1 Flash, drawing moderate Hacker News attention.
ZGCM-1: A Fully Open and Extremely Efficient Foundation Model for Math and Agentic Search
ZGCM-1 is a fully open 7B foundation model with 256K context that stays competitive with frontier models on math reasoning and agentic search.
Training a 3.8B LLM to 0.384 CORE for $998 – Hugo Vergnes
Independent developer Hugo Vergnes trained a 3.8B-parameter Llama-style model to 0.384 CORE on 65B tokens for $998 in 43 hours on rented B200s.
4
dealignai/GLM-5.3-CYBERSECURITY-FP8 — new model trending #13 on Hugging Face
dealignai releases GLM-5.3-CYBERSECURITY-FP8, a 753B MoE weight-modified variant cutting refusals on offensive-security prompts, trending #13.
6