Beyond the Parameter Monolith: Reconstructive Memories, Executable Skills, and Residual Assembly for Language Models
FEM-ASM separates model storage, executable skills, and neural coordination, with limited experimental results.
The paper proposes FEM-ASM, a finite-element-inspired design in which document states and deterministic executable skills submit typed proposals that a residual operator reconciles. An attention-free Multi-Mesh prototype learns causal language modeling but does not demonstrate competitive general capability. A versioned store contains 52,809 reconstructive memory elements near a 1.7-billion-floating-value budget, with incomplete reconstruction at about 75 percent token accuracy. The authors argue for separating storage, execution, and neural coordination while reporting limits in question-only retrieval, unrestricted generation, and end-to-end efficiency.
- FEM-ASM reconciles memory and skill proposals with a residual operator.
- A versioned store holds 52,809 reconstructive memory elements.
- Reconstruction reaches about 75 percent token accuracy.
- An attention-free prototype does not reach competitive general capability.
- Retrieval, unrestricted generation, and efficiency remain unresolved.
Full article193 words · extracted from huggingface.co · click to collapse
Language-model systems can separate contextual computation, persistent storage, and exact execution instead of updating all capabilities through one shared parameter system. We investigate FEM-ASM, a finite-element-method-inspired organization in which independently constructed document states and deterministic executable skills contribute typed proposals to a shared language-model state. An explicit residual operator reconciles proposals attached to common interface nodes. We evaluate this organization through controlled experiments and negative results rather than claiming a physical finite-element formulation of language. An attention-free Multi-Mesh prototype learns causal language modeling but does not establish competitive general capability. A versioned store contains 52,809 reconstructive memory elements near a 1.7-billion-floating-value budget; reconstruction is incomplete, with approximately 75\% token accuracy. Support-aware lexical indices make these elements addressable under provenance-controlled query construction. For executable arithmetic, positional result observations substantially improve neural rendering relative to a repeated global result vector, and output substitutions change the model's preferred answer. A bounded attachment demonstration further measures the effect of making selected evidence available, without establishing the utility of loading an entire multi-billion-value store. The results support a separation of storage, execution, and neural coordination, while identifying unresolved limitations in question-only retrieval, unrestricted answer generation, and end-to-end efficiency.
Text extracted automatically; images, tables and formatting may be missing. Original: https://huggingface.co/papers/2610.04012