tl;dr: Compression and language modeling are fundamentally the same problem: both rely on predicting symbol probabilities, where better predictions yield fewer bits per symbol (Shannon entropy). Entropy coders like arithmetic coding already hit near-optimal compression given a probability distribution, so gains now come from better models—and LLMs happen to be state-of-the-art predictors, trained to minimize cross-entropy (the same math). LLMs can compress dramatically better than gzip (e.g., GPT-2 hitting 10% vs. 24% on sample text), but their multi-gigabyte size and compute cost make them impractical for everyday use like HTTP responses.