AI / Research / LLM · 2025
Open-source LLM memory benchmark — 5.45× recall/token at T=100.
MemoryLens is an open-source framework for rigorous comparison of LLM memory architectures across long conversations. Three backends — Naive (full context), RAG (semantic retrieval), and Cascading Temporal (hierarchical decay) — are evaluated against five content-based metrics at conversation lengths up to T=100. The headline result: Cascading Temporal achieves 5.45× recall per token over naive at T=100 with 78% lower cost. All results are surfaced in a Streamlit dashboard with automated CI via GitHub Actions.
3 memory backends: Naive full-context, RAG retrieval, Cascading Temporal
5 content-based evaluation metrics for memory quality scoring
Cascading Temporal achieves 5.45× recall/token vs naive at T=100
78% lower token cost vs naive full-context approach at T=100
Streamlit interactive dashboard for result exploration and visualisation
CI/CD automated testing and regression checks via GitHub Actions