Can gzip be a language model?
A developer shows gzip can continue text by beam-searching byte sequences that compress best against a primed corpus.
A developer demonstrates language modeling with gzip by scoring candidate byte continuations using DEFLATE compressed length. A corpus is placed in zlib's 32 KiB window, and beam search keeps the spans that compress best before committing them. Single-byte greedy selection fails because compressed length is an integer and often unchanged. Text primed on tiny Shakespeare is only partly coherent, and the author notes earlier compression-only generation performed poorly without beam search.
36