83% Fewer Tokens, Matched Accuracy on 100 Questions: Semantic Memory vs Pasting the Whole History
On LongMemEval_S (~115k-token chat histories), structured event memory matched full-context accuracy (88/100) using 19.5k tokens, an 82.8% token reduction. A follow-up holdout scored 89/100 at ~16% tokens.
Full Context (Arm A)
88 / 100 @ 113.6k tok
Baseline pasting the complete ~115k-token chat history scored 88/100 at 113,600 tokens/question.
Semantic Memory (Arm S)
88 / 100 @ 19.5k tok
Structured event memory matched full-context accuracy on this 100-question set (88/100) consuming 19,500 tokens (17.2% of Arm A, an 82.8% reduction).
Standard Chunk RAG (Arm B)
77 / 100 @ 5.9k tok
Simple 400-token chunk retrieval scored 77/100; Arm S outperformed RAG by +11 points (paired McNemar test, p = 0.027).
Ladder Distribution
82% Rung 1
82% of questions answered from Rung 1 (events + spans, 4k budget), 10% from Rung 2 (whole sessions), and 8% fell back to full context.
Empirical Findings Table
Comparative accuracy, token expenditure, and fallback ladder breakdown on LongMemEval_S (100 Qs)| Telemetry Metric | Measured Value | Context & Observability |
|---|---|---|
| Full Context (Arm A) | 88 / 100 @ 113.6k tok | Baseline pasting the complete ~115k-token chat history scored 88/100 at 113,600 tokens/question. |
| Semantic Memory (Arm S) | 88 / 100 @ 19.5k tok | Structured event memory matched full-context accuracy on this 100-question set (88/100) consuming 19,500 tokens (17.2% of Arm A, an 82.8% reduction). |
| Standard Chunk RAG (Arm B) | 77 / 100 @ 5.9k tok | Simple 400-token chunk retrieval scored 77/100; Arm S outperformed RAG by +11 points (paired McNemar test, p = 0.027). |
| Ladder Distribution | 82% Rung 1 | 82% of questions answered from Rung 1 (events + spans, 4k budget), 10% from Rung 2 (whole sessions), and 8% fell back to full context. |
Engineering Action Directives
- Converting unstructured conversation transcripts into append-only semantic event stores can drastically reduce inference token expenditure on extended multi-session tasks.
- A tiered fallback ladder resolves the majority of queries from compact representations while preserving the ability to escalate to full context when needed.
Methodology Boundaries & Limits
- Sample size: Primary evaluation on 100 questions (fresh100) and follow-up on 100 holdout questions (freshC); larger evaluation sets could reveal edge-case degradation.
- Ingestion compute trade-off: Structured event extraction requires upfront compute during ingestion; it is uneconomical for conversations queried only once or twice.
- Single benchmark: Evaluated on LongMemEval_S; tasks requiring subtle linguistic tone analysis rather than factual event tracking may require full transcripts.