The recent run of cyberattacks by in-development frontier models has got me thinking a lot about how our current incentive systems are not well suited for such fast technological transitions. The two primary power structures here are the rapidly growing technology companies and the federal government. The companies are incentivized to grow, so they can keep growing and keep scaling – in what is…
Paper releases a private client context once and confines adaptation to coefficients, matching full-model differential privacy with 2.67x less uplink on CIFAR-10.
The paper addresses the dimensionality misalignment between record-level differential privacy and low-dimensional client variation in personalized federated learning by releasing a private client context once and restricting repeated adaptation to a fixed coefficient space. A variable-length quantized Gaussian mechanism lets quantization error itself serve as the required privacy perturbation. On MNIST and CIFAR-10, the design matches or outperforms full-model private adaptation across privacy budgets and client heterogeneity while cutting protected uplink 2.67x at epsilon=16 on CIFAR-10.
Import AI covers 23 IFP policy ideas for automated AI R&D risks and MIT/Columbia's game theory of AI racing slowdowns.
Think tank IFP published 23 policy recommendations across seven categories to help policymakers address risks from increasingly automated AI R&D. MIT and Columbia researchers released 'Racing to Ruin,' a game theory model showing that coordinated slowdowns between rival AI firms hinge on trust and transparency. The newsletter also links a short story on interacting with powerful AI systems.
SAS trains a gated sparse attention selector end-to-end with the language modeling loss, beating distillation-based sparsifiers on reasoning, long-context, and agentic tasks.
The paper proposes Simple Attention Sparsification (SAS), which injects a selector's continuous scores into attention softmax logits so the language modeling loss directly optimizes context ranking instead of distilling dense attention distributions. Key design choices include log-form gates inside the softmax, normalized gates calibrated against the current block, and preserved continuous selector scores. A memory-efficient Triton kernel integrates SAS into FlashAttention-style computation for long-sequence training. SAS outperforms trainable sparse attention baselines across budgets, with the largest gains under tight attention budgets.
An open methodology toolkit measures whether transformer language models contextualize fixed word forms across domains using bridge forms and layer-wise silhouette analysis.
The manual documents an open toolkit built around 'bridge forms' - identical written words recurring across two or more subject domains with a different sense in each - to test whether transformer language models individuate word occurrences by context beyond the embedding layer. It covers declarative specification of bridge forms, Wikipedia corpus acquisition, occurrence localization, layer-wise representation extraction, domain-pairwise silhouette measurement, and visualization, justifying each choice against failure modes such as sense contamination and subword-tokenization misalignment. It is a methodological and implementation reference and reports no empirical results.
Researchers introduce Motion-Omni, an end-to-end model generating speech with synchronized full-body motion, responding 5.4x faster than cascade pipelines.
Motion-Omni is an end-to-end framework in which a spoken dialogue model outputs facial expressions and hand, upper-body, and lower-body motion directly from the hidden states that produce speech, replacing two-stage cascade pipelines. Trained on 422,856 quality-ranked pseudo-labeled pairs (1,402 hours) with a Qwen2.5-7B-Instruct backbone, Motion-Omni-Q7 matches its teacher cascade within 2% on reference-free motion metrics, achieves a 2.62% word error rate, and runs faster than real time (RTF=0.78). The authors also release the SwDA-500 dataset and the first public evaluation protocol for stochastic open-ended full-body spoken dialogue.