RoBERTa (AI model): news timeline & CVEs

Intrinsic-Extrinsic Coupling in Learning Dynamics

Researchers show replay can erase a classifier-head intervention's gains, formalizing intrinsic-extrinsic coupling in learning.

The paper formalizes intrinsic-extrinsic coupling, arguing that current observations do not fully determine a learner's response to further training. An executable classifier-head write preserves current logits while repairing specified historical margins. In a CLINC-derived class-incremental setting, replay changed that write's 32-update contribution from five correct predictions to zero. Nonzero interactions also appeared under output distillation, a RoBERTa backbone, and SGDW, where correct-count interactions were negative at 128 updates.

arXiv cs.AI / cs.LG / cs.CLSep 24, 17:30 UTC · 2d agoAI research1

Linguistic Features for Interpretable Textual Entailment

SLITE uses 17 linguistic features for textual entailment, reaching 83% on SICK and nearing RoBERTa.

The paper introduces SLITE, a hybrid explainable model for recognizing textual entailment that pairs structural-relational semantic features with distributional measures including entropy and transfer entropy. Logistic regression on 17 features reaches 83% accuracy on three-class SICK and 96% on SICK-CE, outperforming IsoLex by 4 points and landing within 2 points of RoBERTa at much lower cost. Ablation and SHAP analysis show structural-relational features drive most decisions, while distributional features help detect neutrality and contradiction.

arXiv cs.AI / cs.LG / cs.CLSep 21, 17:26 UTC · 5d agoAI research

RegionFed: Federated Learning for Personalized Query Understanding in Heterogeneous Retail Environments

RegionFed is a gradient-level federated learning framework enabling personalized retail query understanding while matching centralized accuracy with differential privacy.

RegionFed is an architecture-robust federated learning framework for personalized query understanding that operates at the gradient level, using the l2 conflict between regional and global gradients to diagnose heterogeneity and control personalization. Existing parameter-level personalized FL methods collapse on transformers, falling below 10% accuracy on T5, while RegionFed deploys unchanged on T5-Small, T5-3B, RoBERTa, and CNNs. RegionFed-Meta achieves 92.27% across Amazon ESCI, Amazon Reviews, and LEAF-FEMNIST, within 0.23 percentage points of the centralized upper bound, with epsilon-approx-0.60 differential privacy.

arXiv cs.AI / cs.LG / cs.CLSep 4, 17:50 UTC · 22d agoAI research1

RoBERTa

Timeline3 articles

Appears with

Intrinsic-Extrinsic Coupling in Learning Dynamics

Linguistic Features for Interpretable Textual Entailment

RegionFed: Federated Learning for Personalized Query Understanding in Heterogeneous Retail Environments