ZeroHour

Search: “ALiBi”

3 stories in the last 7d

Distance generalization in transformers: why bother with positional encoding?

arXiv study uses synthetic delay-copy tasks to show how RoPE, ALiBi, NoPE and training data diversity affect transformers' distance generalization.

The paper studies distance generalization in transformers: extrapolating when inter-token distances change between training and inference while context length stays fixed. Using two synthetic delay-copy tasks with finite source-recall distances, the authors test models on unseen delays. They investigate whether positional encodings such as RoPE and ALiBi outperform no positional encoding (NoPE), how the diversity of training distances affects performance, and when distance transfer learning is positive or negative.

arXiv cs.AI / cs.LG / cs.CL · 6d agoAI research2

Fake CAPTCHA Scams

Bruce Schneier examines fake CAPTCHA scams that abuse human-verification prompts as social engineering lures against users.

Bruce Schneier's blog post covers fake CAPTCHA scams, a social engineering technique in which attackers pose as CAPTCHA verification checks to manipulate users. The available page text is largely site navigation, and no specific campaign, victim, or malware family is named in the source.

Schneier on Security · 17h agoPhishing & fraud

Cliff Stoll’s DEF CON Talk

Schneier on Security posts about Cliff Stoll's DEF CON talk; the visible text provides no substantive details about its content.

Schneier on Security published a brief post referencing Cliff Stoll's talk at DEF CON. The available text consists almost entirely of standard blog navigation, author biography, and archive listings, with no concrete details about the talk itself. As a general security-community item, it carries no direct risk impact for defenders.

Schneier on Securityupdated · 5d agofirst · 5d agoIndustry 2 sources1