TurnTurk-GPT predicts Turkish turn-taking from unpunctuated text
The fine-tuned Turkish GPT-2 beats a multilingual detector, but ties a zero-shot model on unseen speakers.
Stories per day · 30 days
2 RSS
Relatedbilevel-optimization1dialogue1hyperparameter-optimization1meta-learning1newton-methods1speech-recognition1turkish1turn-taking1
13 stories, newest first · one card per story
Stories tagged #gpt-2
Newton bilevel methods drop the lower-level strong-convexity assumption
Experiments cover synthetic tasks, data hyper-cleaning, and GPT-2 reweighting, with claimed better complexity than GALET and sl-BAMM.
Inner Momentum for Differentially Private Muon
Averaging per-example Muon gradients over recent model snapshots before clipping bounds distortion and improves differentially private GPT-2 fine-tuning.
What Gradients Add to Text Leakage in Split Language Models, Counted per Token and per Document
Split-learning observers recover 97% of GPT-2 tokens; gradients lift exact document recovery from 14% to 38%.
Every Ablation Is a Dose: Counterweights and the Semblance of Self-Repair
Language-model self-repair is explained as pre-existing counterweights following an affine response to ablation dose.
Is Weight Tying Still Beneficial for Decoder-Only LLMs in Private Settings Under DP-SGD?
Untied embeddings beat weight tying for decoder-only LLMs under DP-SGD, raising accuracy and cutting memory by over 60%.
PassGPT+: Leveraging Linguistic Priors for Password Modeling
PassGPT+ adapts GPT-2 priors for passwords, recovering 22.53% of RockYou held-out passwords at 10^8 guesses.
Attention Function as an Intrinsic Inductive Bias: How Models' Behavior Diverges in Novel Contexts
A fixed softmax-sigmoid attention mix barely changes GPT-2 in-distribution loss but shifts out-of-distribution perplexity sharply.
Rethinking Circuit Evaluation: Do Circuits Explain Model Errors?
Circuits that match GPT-2's correct answers often fail to reproduce most of its errors.
Seal, Then Sample: Sampled Layerwise Proofs for Verifiable LLM Inference from GPT-2 to 70B
Researchers propose Sampled Layerwise Proofs to audit outsourced LLM inference through Llama-2-70B.
The Functionalizer: Lossless Functional Decomposition for Subword Tokenization
The Functionalizer losslessly factors casing, diacritics and character repetition into reversible opcode prefixes, cutting tokenizer vocabulary needs by up to 19.7 percent.
Right Tool, Right Job: Native-Language Evaluation, Tokenizer Sensitivity, and Methodological Findings from a French-Only BabyLM
French BabyLM entry METRON-FR (125M GPT-2, 92.47M words) shows tokenizer artifacts dominate child-scale zero-shot evaluation; proposes standard diagnostics.
Due to concerns about malicious applications, GPT2 will not be released (2019)
OpenAI's landmark 2019 GPT-2 post withheld the full 1.5B-parameter model over misuse concerns, releasing only a smaller variant and paper.
5