| I accidentally turned LLM memory into program analysis(pwning.systems) | |
| 300 points by matt_d 12 days ago | 83 comments | |
tl;dr: Frustrated by LLM agents losing track of established facts during long vulnerability investigations, the author built Lemmalog, a Datalog engine that maintains structured facts, rules, and derivations with provenance tracking—so when an observation is invalidated, dependent conclusions are automatically retracted. On LongMemEval and LoCoMo benchmarks, it's competitive with dedicated memory systems (0.463 and 0.533 F1) while using ~38x less context than full-history prompting, and notably tops the field on knowledge-update questions. The takeaway: much of "LLM memory" is really a database problem (incremental evaluation, retraction, temporal facts) that classical program analysis techniques already handle well. | |
HN Discussion:
| |