I finally published the AI memory system I've been building.
It took longer than I wanted. Broke in a few different ways, drifted in a few others, spent too long in that miserable almost-done state where you fix the last thing and find the next last thing. But it's out.
Every AI chat starts cold. Blah,blah,blah. That friction compounding across a hundred sessions — that's what I was trying to fix.
The architecture is two layers.
The wiki is the curated layer: the best 5% of everything. Active projects, recurring concepts, key people, decisions worth remembering. The things your AI should be able to walk up to and understand without a briefing. An LLM helps write and maintain it. It stays small on purpose — 200 to 400 pages at steady state, not 10,000.
MemPalace @bensig @MillaJovovich is the other 95%: raw notes, session transcripts, research, old documents, the full long tail of everything you don't want to curate but can't afford to lose. Searchable. Semantic. There when you need it.
Wiki = curated 5%. MemPalace = searchable 100%.
The wiki layer started with Andrej @karpathy 's LLM-wiki idea — the observation that AI memory shouldn't live inside an app feature you don't control, but in plain files a model helps you maintain. That framing is right. I built on top of it.
What I added is the deeper memory layer and the bridge between them: HARVEST.
HARVEST is a daily loop. Each agent writes a session diary as it works. At 08:30, those diaries get mined into MemPalace — turned into searchable drawers and knowledge graph facts. At 09:00, HARVEST runs and asks: what did the agents learn since yesterday, and what's durable enough to belong in the shared wiki? The pieces that qualify get promoted. The wiki updates. Every agent on every machine reads the same wiki.
Here's what that actually means: one agent runs an overnight task on one machine at 3 AM and writes its findings into a session diary. By the time anyone opens a chat the next morning, those conclusions are already in the wiki — attributed, citable, searchable. Local session memory becomes shared durable memory without anyone manually copying a summary across. The system knows what it learned.
This is not a plugin. No install button, no "upload your notes and your AI is fixed." It's a pattern — a gist for the concept, a repo for the implementation, markdown files with a clear schema, and automation wired around all of it. Every piece is replaceable. The pattern is what travels.
Better models are nice. Better prompts are nice. But if the system doesn't remember your world between sessions, you're rebuilding context from scratch every time — regardless of how capable the model is.
That's the problem. This is the attempt to fix it.
Gist: gist.github.com/FuFicFac/8ddc6…
Repo: github.com/FuFicFac/ej-pa…
English
