CATF framework improves context-aware time-series forecasting
Manager-Worker architecture with hard routing outperforms standalone models and Time-MoE on five context-rich datasets.
Stories per day · 30 days
3 RSS
Related3d-reconstruction1action-genome1attention-kernel1blackwell-gpu1computer-vision1disease-trajectory1doom1ehr1flamia1forecasting1geometry1github1graph-neural-network1hard-routing1midi1mimic-iv1mixture-of-experts1monte-carlo1
37 stories, newest first · one card per story
Stories tagged #transformer
FLAMIA kernels accelerate attention on NVIDIA Blackwell GPUs
FLAMIA achieves 1.10-1.40x speedup for long-context attention using MXFP8 on B200 GPUs with token-block scaling.
Solv3R looped transformer solves multi-view geometry
Solv3R uses a looped transformer for iterative refinement in multi-view reconstruction without task-specific modules.
Hacker News post turns DOOM into a transformer
The Show HN listing has 20 points and 5 comments and gives no technical detail.
Transformer sampler matches MCMC on the 2D XY model up to 96x96
A locality-informed autoregressive network reproduces energy, magnetization and helicity modulus without missing configuration modes.
DuET jointly transcribes piano notes and tracks beats for scores
On Gangqinpu, DuET cuts MUSTER mean error from 6.56% to 3.64% versus the strongest baseline.
Graph-Enhanced Disease Trajectory Discovery and Temporal Endpoint Forecasting from Longitudinal Electronic Health Records
A MIMIC-IV study finds graph branching motifs mainly improve long-range event timing, not endpoint discrimination.
RTAformer targets relation changes in video scene graph generation
On Action Genome, gains are larger on relation-transition frames than on non-transition frames with matched backbones.
QCATS: Query Context-Aware Transformer Slicing for Efficient Predictive Query Processing
QCATS slices transformer FFNs using query predicates, cutting in-database inference latency up to 4.42x.
[0day-rubbish] StreamSets Transformer 3.17.0 auth-mode none fallback and un-sandboxed ScalaDTransform execution to container root (8.1 primary)
Researchers disclose StreamSets Transformer 3.17.0 auth fail-open and unsandboxed code execution as container root.
Systematically Optimized CNN-Transformer with Focal Loss for Imbalanced Intrusion Detection on NSL-KDD
A tuned CNN-Transformer with Focal Loss reaches 98.73% accuracy on imbalanced NSL-KDD detection.
Deep Learning for Sleep Heart Rate Estimation from Accelerometers: Toward Population-Scale Cardiac Insight Without Optical Sensors
SeqSmoother estimates sleep heart rate from wrist accelerometry at 1.60 bpm participant-macro MAE.
Learning to Learn a Language
A 300M-parameter model trained only on synthetic priors learns to predict real languages in context.
Less Decoder is More Encoder: Geometric Representation Learning from Novel View Synthesis
SNAP, a self-supervised pose-conditioned transformer for novel view synthesis, learns transferable geometric representations competitive with geometry-supervised methods across five tasks.
Do Language Models Need a Trainable Input Embedding Table? Fixed Minimal Token Codes at 1.7B-Class Scale
Fixed 16-bit token codes let 1.7B decoder-only models learn substantial language ability without trainable input embeddings.
Looped Diffusion Transformer
Looped-DiT reuses shared Transformer blocks within each denoising step, letting a 260M-parameter text-to-image model beat one 6.5x larger at 4.9x lower compute.
Attention Function as an Intrinsic Inductive Bias: How Models' Behavior Diverges in Novel Contexts
A fixed softmax-sigmoid attention mix barely changes GPT-2 in-distribution loss but shifts out-of-distribution perplexity sharply.
CDMD: A Cross-Dataset Mixed-Type Diffusion Model for Tabular Data
CDMD trains one mixed-type diffusion model jointly across heterogeneous tabular schemas and beats single-dataset baselines.
Show HN: TurboGPT: train 22KiB transformer in 13s
Show HN project TurboGPT trains a 22-kilobyte transformer in 13 seconds.
Cropland PAtteRNS: Parallel Dimensional Attention Networks and Attention to Dataset Disparity for Crop Segmentation in Satellite Imagery Time Series Data
PAtteRNS factorizes temporal, spectral, and spatial attention for Sentinel-2 cropland segmentation.
LVMT: Video Mask Transformer for Long-term Video Segmentation
LVMT uses GRU memory and truncated query propagation to set a faster state of the art in long-term video segmentation.
WhiteMatter: All-to-All Cross-Layer Connections via KV Source Mixing
WhiteMatter mixes key-value sources across Transformer layers, matching deeper models with a smaller KV cache.
Your Transformer Can Hold Two Thoughts at Once: Evidence of Linear Superposition in LLMs
Transformers superpose next-token distributions from mixed text streams, and fine-tuning can restore that linearity.
AstroForge is putting AI in command of its next spacecraft
AstroForge will fly Solo, an in-house transformer stack, to autonomously control a spacecraft.
2