Elephantus: a Memory Layer for AI Apps

2026-10-04T00:00:00+00:00

RAG finds similar text. Memory tracks what is currently true</em> about a user.</strong></p> </blockquote>

Elephantus is a small memory engine for AI apps that runs on your own machine. You send it messages; it pulls out short facts about the user, links each new fact to the ones it already has, and keeps track of which facts are still true. You can use it from a REST API, from Claude Desktop over MCP, or from a terminal-style web UI.</p>

An elephant never forgets, but it does know which of its memories are out of date.</em></p>

</video>
A walkthrough of the web UI: chat, linking decisions, the memory graph, and the RAG-vs-memory comparison.</figcaption> </figure>
The problem: similarity isn’t truth</h2>
Suppose a user tells an assistant three things over a few weeks:</p>

“I love Adidas sneakers”</li>
“My Adidas broke after a month”</li>
“I’m switching to Puma”</li> </ol>
Later they ask, “What sneakers should I buy?”</strong></p>

Naive RAG</strong> stores the raw messages and fetches the ones most similar</em> to the question. “I love Adidas sneakers” is the closest match (it even contains the word sneakers</em>), so the assistant recommends Adidas</strong>. Similarity says nothing about whether a fact is still true.</li>
Memory</strong> turns each message into atomic facts and compares every new fact with what it already knows. “User is switching to Puma” updates</strong> “User loves Adidas sneakers”. The old fact is marked outdated: it’s kept for history but never retrieved again. The assistant recommends Puma</strong>.</li> </ul>
Most “memory” features in AI apps are really retrieval over a chat log. That works until the user changes their mind, and people change their minds all the time. I built Elephantus to make that difference concrete and measurable.</p>
How it works</h2>
The engine</h3>

Concept</th> What it does</th></tr></thead>

Documents vs memories</strong></td> Raw messages are stored and chunked (that’s the RAG baseline). An LLM extracts short, atomic memories</strong> from them.</td></tr>
Container tags</strong></td> Every document and memory belongs to a container_tag</code> (for example khan</code> or work</code>). Tags never mix, not even during linking.</td></tr>
Relationships</strong></td> Each new fact is compared with the most similar current</em> memories and labelled NEW</code>, UPDATES</code> (the old fact is retired and an edge is stored), EXTENDS</code> (both stay current, with an edge) or DUPLICATE</code> (skipped). Every decision is logged with a reason.</td></tr>
Kinds</strong></td> static</code> facts are long-term (where you study, lasting preferences). dynamic</code> facts are recent or ongoing (this week’s project).</td></tr>
Forgetting</strong></td> Time-bound facts (“exam tomorrow”) get an expiry at extraction time and drop out of retrieval once it passes. Facts can also be forgotten explicitly.</td></tr>
Hybrid search</strong></td> Embedding similarity plus SQLite FTS5 keyword search, merged with Reciprocal Rank Fusion</strong>. Only current, unexpired memories are searched.</td></tr>
Profile</strong></td> One call returns the static facts, the recent dynamic facts and, optionally, search results for a query: a ready-made context block for a prompt.</td></tr> </tbody></table> </div>
What happens when you add a message</h3>
"I'm switching to Puma" </span> │ 1. store document, chunk, embed chunks (RAG baseline data) </span> │ 2. LLM extracts facts → "User is switching to Puma sneakers" (dynamic, no expiry) </span> │ 3. embed fact, shortlist the 5 most similar CURRENT memories in this container </span> │ 4. LLM judges → UPDATES "User loves Adidas sneakers" </span> ▼ 5. insert new memory, mark old is_latest=0, add edge new→old, log the decision </span></code></pre> Linking is two steps on purpose. Embeddings give a cheap shortlist, and an LLM judge makes the precise call. The judge always sees the top 5 current memories, with no similarity threshold, so it can still catch an update between facts that share almost no words: “switching to Puma” and “loves Adidas” have none in common.</p> Architecture</h3> The engine (elephantus/engine.py</code>) is a plain Python library that knows nothing about HTTP, MCP or the UI. Three thin entry points sit on top of it:</p> a REST API</strong> (FastAPI), which also serves the web UI;</li> an MCP server</strong> over stdio, which Claude Desktop launches;</li> the web UI</strong>, a hand-written static page that only calls the public REST API.</li> </ul> All three share one engine and one SQLite file</strong>, so a fact saved from Claude Desktop shows up in the UI straight away. Behind the engine are:</p> an LLM provider of your choice:</strong> Anthropic, OpenAI, Azure AI Foundry, or Ollama for fully offline use;</li> local embeddings:</strong> bge-small</code> via fastembed, about 70 MB and downloaded on first use;</li> a single SQLite database</strong>, with FTS5 for keyword search.</li> </ul> The demo</h2> The web UI is a dark, terminal-style page with tabs for chat · memories · graph · profile · search · eval · log</strong>. A sidebar holds the container tag, a clock for simulating time, live counts, and the current memory list.</p> Chat: RAG and memory, side by side</h3> Each message shows its linking decisions like a coding agent's tool calls. Every question is answered twice with the same model and prompt, once from RAG and once from memory, so the only difference is the context.</figcaption> </figure> The RAG answer’s context is a list of past messages, including stale ones like “I love adidas sneakers”. The memory answer’s context lists only what is currently true, newest first.</p> Graph: how facts relate</h3> The red UPDATES</code> edge from "switching to Puma" retires "loves Adidas sneakers" (struck through). Blue EXTENDS</code> edges add detail while keeping both facts current.</figcaption> </figure> Profile: what the model actually sees</h3> The profile splits facts into static and dynamic, and shows the exact context prompt an app would inject. Outdated and expired facts are excluded.</figcaption> </figure> Log: every decision, with a reason</h3> Every linking decision is logged with the judge's reasoning, which makes the memory easy to audit and debug.</figcaption> </figure> Forgetting, with a simulated clock</h3> Send “I have an exam tomorrow”, then drag the Clock</strong> slider to +72h. The memory turns expired</em> and disappears from answers, the profile and search. The same time_offset_hours</code> parameter is accepted by the API, which is how the evaluation tests expiry without waiting days.</p> Using it from Claude Desktop (MCP)</h2> The MCP server exposes three tools:</p> Tool</th> What it does</th></tr></thead> memory(content, action="save" | "forget")</code></td> Save information (extract + link), or forget the best-matching memory</td></tr> recall(query, limit?)</code></td> Search current memories, plus a profile summary</td></tr> context()</code></td> The full profile, to inject at the start of a conversation</td></tr> </tbody></table> </div> In one chat say “Remember that I’m switching to Puma sneakers”</em>. In a brand-new chat, ask “What do you know about my shoe preferences?”</em>: Claude calls recall</code> and answers from the same store the web UI is showing.</p> REST API</h2> The core calls are add</code>, search</code> and profile</code>, with interactive docs at /docs</code>:</p> curl</span></span> -</span>s</span> localhost:8000/v1/add -</span>H</span> '</span>content-type: application/json'</span></span> \ </span></span></span> -</span>d</span> '</span>{"content": "I am switching to Puma", "container_tag": "khan"}'</span></span></span> </span> </span>curl</span></span> -</span>s</span> localhost:8000/v1/search -</span>H</span> '</span>content-type: application/json'</span></span> \ </span></span></span> -</span>d</span> '</span>{"q": "What sneakers should I buy?", "container_tag": "khan", "mode": "memories"}'</span></span></span> </span></code></pre> /v1/search</code> takes a mode</code>: memories</code> (hybrid search over current facts), documents</code> (the naive RAG baseline) or hybrid</code> (both). Other endpoints cover chat (answers twice and returns both contexts), forgetting, and listing a container’s memories, graph, log and documents.</p> Evaluation</h2> elephantus eval</code> runs 25 scripted scenarios</strong> in three categories:</p> Category</th> n</th> What it tests</th></tr></thead> knowledge_update</code></td> 11</td> A fact changes (city, job, phone, diet…), then a question about the current value</td></tr> extension</code></td> 7</td> Detail builds up across several messages (job → team → role → language)</td></tr> expiry</code></td> 7</td> A temporary fact (“exam tomorrow”, “in Tokyo this week”), then a question days later</td></tr> </tbody></table> </div> Each scenario starts a fresh container with 4 unrelated distractor messages, adds its own messages one simulated hour apart, then asks its question. Three retrieval modes return their top 3: RAG</strong> (chunk similarity), Memory</strong> (hybrid search over current memories) and Hybrid</strong> (memories plus chunks).</p> Recall@3:</strong> the fraction of expected current</em> facts found in the top 3.</li> Stale-fact rate:</strong> the fraction of scenarios whose top 3 contains an outdated or expired fact.</li> </ul> Results</h3> Run with Azure AI Foundry gpt-4o</code> and BAAI/bge-small-en-v1.5</code> embeddings, k = 3, about 2 minutes:</p> Category</th> n</th> RAG Recall@3</th> Memory Recall@3</th> Hybrid Recall@3</th> RAG stale</th> Memory stale</th> Hybrid stale</th></tr></thead> knowledge_update</td> 11</td> 100%</td> 100%</td> 91%</td> 100%</td> 27%</td> 82%</td></tr> extension</td> 7</td> 93%</td> 96%</td> 58%</td> n/a</td> n/a</td> n/a</td></tr> expiry</td> 7</td> 100%</td> 100%</td> 100%</td> 100%</td> 0%</td> 43%</td></tr> overall</strong></td> 25</td> 98%</td> 99%</strong></td> 84%</td> 100%</td> 17%</strong></td> 67%</td></tr> </tbody></table> </div> The same results in the UI's eval tab.</figcaption> </figure> What the numbers show:</p> Memory matches RAG on recall and cuts stale facts from 100% to 17%.</strong> RAG finds the right text every time, but it also returns the outdated version every time. That gap is what this project is about.</li> Expiry: 0% stale.</strong> Facts like “exam tomorrow” are filtered out at query time once they expire. RAG has no notion of time.</li> The LLM judge caught all 11 updates</strong>, including “switching to Puma” vs “loves Adidas”, which share no words.</li> Most of the remaining 27% on updates is a strict metric.</strong> The flagged memories describe</em> the change (“User’s Adidas sneakers broke after a month”) rather than restating the old fact. They mention the old keyword without a new one, so the keyword check counts them as stale. Judged by hand, none of them is outdated.</li> Hybrid mode is the weak spot at k = 3.</strong> Raw chunks, including stale ones, compete with memories for the same three slots.</li> </ul> An offline run swaps the LLM for a rule-based stand-in and hash embeddings. It reaches only 68% recall and a 33% stale rate, which shows how much of the quality comes from the LLM judge and a real embedding model.</p> Design decisions</h2> SQLite + FTS5 with brute-force NumPy vectors.</strong> One file and no services. At demo scale, brute-force cosine search is instant and avoids a vector-database dependency. WAL mode lets the API, MCP and UI processes share the file.</li> fastembed (ONNX) instead of sentence-transformers.</strong> The same bge-small</code> model without PyTorch, so the install is much smaller and faster.</li> Safe fallbacks for bad LLM output.</strong> Lenient JSON parsing, one corrective retry, then per-item validation. An unusable relation becomes NEW</code>, so a fact is never lost or wrongly retired.</li> Short aliases for the judge.</strong> Candidate memories are shown as m1</code>, m2</code>… instead of UUIDs, so the model copies IDs reliably.</li> Relative expiry.</strong> The LLM outputs expires_in_hours</code> rather than timestamps. Models handle “tomorrow ≈ 48h” more reliably than absolute dates, and the engine converts it using the message’s (possibly simulated) time.</li> Nothing is ever deleted.</strong> UPDATES</code> flips an is_latest</code> flag, expiry and forgetting are filters, and explicit forget is a soft delete. That keeps the full history for the graph and the decision log.</li> A fair side-by-side.</strong> Both chat answers use the same model and prompt, so only the retrieved context differs. The question is stored only after</em> answering, so the RAG baseline can’t retrieve the question itself.</li> One client for OpenAI, Azure and Ollama.</strong> All three expose an OpenAI-compatible endpoint, so one small class covers them.</li> A static web UI instead of Streamlit.</strong> Hand-written HTML, CSS and JS with no build step, served by the same FastAPI process. It uses only the public API, so it doubles as a working example of it.</li> </ul> Testing</h2> The pytest suite needs no API key or downloads: the LLM is replaced by a scripted fake. It covers storage and container isolation, chunking, extraction (including malformed output), the sneaker sequence (UPDATES</code> / EXTENDS</code> / DUPLICATE</code>), hybrid search, expiry with simulated time, the profile, forgetting, the REST API, the MCP tools, and the web UI, including a real-browser run of the sneaker flow with Playwright. An optional live test runs end to end against a real provider.</p> Limitations</h2> Small scale by design.</strong> Brute-force vector search and a single SQLite file suit demos and personal use, not millions of memories.</li> Quality depends on the LLM.</strong> Small local models may mislabel relations or miss expiry hints, as the gap between the offline and real evaluation runs shows.</li> One relation per fact.</strong> A fact that both updates one memory and extends another is simplified to a single label.</li> Expiry is set once</strong>, at extraction time. “The exam moved to Friday” creates a new fact that updates the old one, rather than rescheduling it.</li> The evaluation is keyword-based</strong> and checks retrieval, not answer quality, on a small hand-written dataset.</li> </ul> Stack</h2> Python 3.11+, FastAPI, SQLite (FTS5), NumPy, fastembed (BAAI/bge-small-en-v1.5</code>), the official MCP Python SDK, and Anthropic / OpenAI / Azure AI Foundry / Ollama as LLM providers. MIT licensed.</p>

Akram - agents

Elephantus: a Memory Layer for AI Apps

How it works</h2>

The demo</h2>
The web UI is a dark, terminal-style page with tabs for chat · memories · graph · profile · search · eval · log</strong>. A sidebar holds the container tag, a clock for simulating time, live counts, and the current memory list.</p>

Stack</h2>
Python 3.11+, FastAPI, SQLite (FTS5), NumPy, fastembed (`BAAI/bge-small-en-v1.5</code>), the official MCP Python SDK, and Anthropic / OpenAI / Azure AI Foundry / Ollama as LLM providers. MIT licensed.</p>`