| Can gzip be a language model?(nathan.rs) | |
| 398 points by networked 22 days ago | 161 comments | |
tl;dr: Building on the "compression is prediction" equivalence, the author uses gzip (via zlib's DEFLATE) as a language model by scoring candidate continuations by their compressed length against a corpus-primed context. Naive next-byte selection fails due to integer-length quantization, so a beam search over multi-byte spans is used, with only a recent tail retained to avoid verbatim loops. Primed on tiny Shakespeare, it produces recognizably Shakespeare-ish (if incoherent) output—no weights, no training, pure standard-library Python. | |
HN Discussion:
| |