Training and Inference Dynamics of PLDR-LLMs: Row-Map Collapse, Renormalization, and Predictive Reduction
A monograph derives exact training and inference identities for power-law decoder representation language models.
The monograph gives a unified account of training and inference in Power Law Decoder Representation language models. Exact finite-work identities split changes in the absolute energy of the row-centered learned map into parameter contributions, signed interactions, and numerical defects. Experiments show observer and optimizer dependence, reject tested autonomous row-state candidates, and support finite conditional prediction rather than a thermodynamic critical class. The theory separates exact identities, conditional dynamical claims, and finite empirical findings, with proofs and compact numerical evidence.
- Exact identities decompose row-map energy into parameters, interactions, and defects.
- Predictive renormalization retains optimizer memory, data, schedule, and numerics.
- Experiments reject autonomous row-state candidates and a thermodynamic critical class.
- Scaling-law transfer to inference needs symmetry, head limits, and error budgets.
Full article204 words · extracted from huggingface.co · click to collapse
This monograph develops a unified account of training and inference in Power Law Decoder Representation language models (PLDR-LLMs). Exact finite work identities decompose changes in the absolute energy of the row-centered learned map into parameter contributions, signed interactions, and numerical observation defects. Positive affine blocking retains restarts at the row-constant face, while the augmented AdamW state supplies the complete dynamical description. Predictive renormalization acts on the complete conditional training law for a single pass over distinct corpus target blocks, retaining optimizer memory, remaining data, schedule, and numerical policy. Autonomous reductions require closure; approximate reductions carry successor and emission errors. Finite-population covariance, matched physical clocks, matrix fluxes, and signed temporal energy connect row dynamics to model-wide observations. Absolute row collapse, relative row concentration, operator stabilization, and predictive accuracy are distinguished. Experiments reveal observer and optimizer dependence, reject the tested autonomous row-state candidates, and support finite conditional prediction and state-specific operator reduction. Independent single-pass families exhibit moving finite fluctuation regions without establishing a thermodynamic critical class. Conditional symmetry, head limits, covariance flows, and readout error budgets specify assumptions needed to transfer scaling laws to inference. The theory separates exact identities, conditional dynamical claims, and finite empirical findings, with proofs, selected formal checks, and compact numerical evidence.
Text extracted automatically; images, tables and formatting may be missing. Original: https://huggingface.co/papers/2609.34130