Andrew

the way people are approaching this atm is sort of a horseless carriage. they want to take a bunch of ephemeral slack messages + other conversations and transform them into 'useful facts' without the broader context of the company ontology. it is really not that hard once you design your ontology and system of artifacts correctly. i personally haven't felt a need for this memory thing that people are talking about at all, once i designed my filesystem correctly and SOPs for mutating it: x.com/dremnik/status…

On memory harnesses for agents: 1. I remain unconvinced that the best “single-agent” memory harnesses will be external to the foundation model providers. Genuinely believe that both claude and codex will iterate heavily on this. Memory gives personality and is therefore too strategically important to out-source. The memory systems that survive will have to offer more than a single-agent solution. 2. As it stands today - the best memory harnesses are the ones with the best models doing inference. Virtually all “good” memory harnesses are doing a combination of retrieval + inference on retrieved facts to answer queries. A lot of the performance in the benchmarking of “good” memory systems is a direct result of the inference. Obviously the prompts and workflow matters too; but it is undoubtedly the case that much of the "problems" of semantic reasoning is being solved by the inference of a smart model. This paradigm is essentially “memory as an agent” and derives performance and resolves contradictions from the underlying intelligence of the model doing the inference. Suffice to say, you want the smartest model doing this. 3. Virtually all memory systems have settled on “consolidation” or “dreaming” as a core mechanism. It’s basically a process to correct contradictions in your facts and consolidate memory into hierarchical layers. -- If you're looking for a memory harness to implement and don't want to wait around for codex/claude to "solve memory", I've really come to like what Honcho is doing and you should check them out. They have an open-sourced repository that lets you plug and play any model for inference. It should go far enough for 99% of use-cases, but if you really want a hands-off solution... Their managed solution comes with a specialized, fine-tuned model. I think it gives them a huge edge because - specialized models beat generalized models at specific tasks and inference on facts to answer queries accurately and to be helpful is such an example of a specialized task.

@ashugarg lots of great points. another force worth noting is the active restructuring as people begin to realize that capturing traces is necessary for agent performance. competitive pressure alone will drive adoption of systems like the design i outline here: x.com/dremnik/status…

@andrewchen i think the more salient question is whether incumbents can actually structurally adapt enough to compete with the margins of smaller, AI-native startups that build systems like this from the outset to maintain their company world model: x.com/dremnik/status…

question being asked across every B2B vertical: Will incumbents incorporate AI faster than startups can disrupt their industries with AI? do the tools sold to incumbents win, or the full-stack players seeking to replace the existing players?

for anyone interested where jack’s ideas are likely coming from

@karpathy @garrytan i’ve been doing something similar, which motivated the idea of a compiler for markdown to do things like validate links and generate indexes / outlines at the top of files automatically for fast nav: github.com/dremnik/kdb

LLM Knowledge Bases Something I'm finding very useful recently: using LLMs to build personal knowledge bases for various topics of research interest. In this way, a large fraction of my recent token throughput is going less into manipulating code, and more into manipulating knowledge (stored as markdown and images). The latest LLMs are quite good at it. So: Data ingest: I index source documents (articles, papers, repos, datasets, images, etc.) into a raw/ directory, then I use an LLM to incrementally "compile" a wiki, which is just a collection of .md files in a directory structure. The wiki includes summaries of all the data in raw/, backlinks, and then it categorizes data into concepts, writes articles for them, and links them all. To convert web articles into .md files I like to use the Obsidian Web Clipper extension, and then I also use a hotkey to download all the related images to local so that my LLM can easily reference them. IDE: I use Obsidian as the IDE "frontend" where I can view the raw data, the the compiled wiki, and the derived visualizations. Important to note that the LLM writes and maintains all of the data of the wiki, I rarely touch it directly. I've played with a few Obsidian plugins to render and view data in other ways (e.g. Marp for slides). Q&A: Where things get interesting is that once your wiki is big enough (e.g. mine on some recent research is ~100 articles and ~400K words), you can ask your LLM agent all kinds of complex questions against the wiki, and it will go off, research the answers, etc. I thought I had to reach for fancy RAG, but the LLM has been pretty good about auto-maintaining index files and brief summaries of all the documents and it reads all the important related data fairly easily at this ~small scale. Output: Instead of getting answers in text/terminal, I like to have it render markdown files for me, or slide shows (Marp format), or matplotlib images, all of which I then view again in Obsidian. You can imagine many other visual output formats depending on the query. Often, I end up "filing" the outputs back into the wiki to enhance it for further queries. So my own explorations and queries always "add up" in the knowledge base. Linting: I've run some LLM "health checks" over the wiki to e.g. find inconsistent data, impute missing data (with web searchers), find interesting connections for new article candidates, etc., to incrementally clean up the wiki and enhance its overall data integrity. The LLMs are quite good at suggesting further questions to ask and look into. Extra tools: I find myself developing additional tools to process the data, e.g. I vibe coded a small and naive search engine over the wiki, which I both use directly (in a web ui), but more often I want to hand it off to an LLM via CLI as a tool for larger queries. Further explorations: As the repo grows, the natural desire is to also think about synthetic data generation + finetuning to have your LLM "know" the data in its weights instead of just context windows. TLDR: raw data from a given number of sources is collected, then compiled by an LLM into a .md wiki, then operated on by various CLIs by the LLM to do Q&A and to incrementally enhance the wiki, and all of it viewable in Obsidian. You rarely ever write or edit the wiki manually, it's the domain of the LLM. I think there is room here for an incredible new product instead of a hacky collection of scripts.

@koylanai you're not missing anything, some people just haven't caught up with the full significance of what this technology actually is. you have to understand what a neural network is before you can really grasp it. learning to code is like learning mental math after calculators

Listening to lectures from Stanford, MIT, etc., on a treadmill motivates me more than playing techno but I still don't understand why people say “spend time on learning these human-written coding languages,” when there is a new, alien technology that replicates the human brain, improves every hour, and can be trained by you on literally anything. The best thing would be to learn the depth of that technology because it could be used to learn and build anything on your behalf, no? The more you understand, the better you can guide; but the more you understand AI, the more and better context you provide to your agents; like you don't need to learn all the details of a language by reading books, you can turn those books into a context system or training data because most human-written languages will be extinct soon anyway. What am I missing?

@yacineMTB 🫡

ONLY HOT DADS IN TECH CAN REPLY TO THIS POST. YOU ARE NOT ALLOWED TO REPLY IF YOU AREN"T A HOT DAD IN TECH.

@krispuckett natural selection will naturally favor leaner organizations with less friction from the control center to the edges where execution happens. there are too many misaligned / rent-seeking incentives along a long chain of intermediaries for it to survive against shorter chains

It’s a bit funny. This essay misses the relational element of creating with others. 2000 words and zero thoughts about trust, relationships, humanity, and soul. I know I’m in constant danger as a middle manager. If all I’m doing is passing info back and forth, I don’t belong at that place. It’s the idea de jour that middle management should die. But great middle managers can create and shape culture, trust, speed, and safety. Maybe my role should die, maybe it will. AI can route information better than managers. It can. The question is whether the things that make teams actually work, trust, safety, someone willing to fight for your growth, can be modeled well and with a human touch. Jack wrote 2,000 words about how organizations work and never once mentioned why people do their best work. It’s never an information problem, it’s a human one.

@BrianHatano @jack if a company was always unpredictable it would be impossible to do business with them. the opposite is closer to the truth: companies succeed according to the degree to which they _are_ predictable, i.e. they fulfill their promises and make good on their word

@dremnik @jack A company cannot be autonomous and still be a company. Autonomous machines are predictable. Companies succeed by being unpredictable in their dealings with unpredictable society.

@thenanyu the codifier’s curse, the prisoner’s dilemma of collective obsolescence

When you’re remote, then all of your actions are mediated by computers, which means that the AI can access & act on them This advantage only compounds further as models improve x.com/jack/status/20…

@beffjezos the funny thing is the afterglow of these effects are usually temporary. BJ will be going back on all this in a few months more than likely

VC's adding an anti-one-shotting clause to term sheets is insanely based. Investors are betting on your will to power, you shouldn't be risking ego death. No psychedelics until IPO. Too many late stage founders get absolutely cooked by these drugs and lose their drive.

i don't understand this debate at all. the comparison is entirely wrong. it makes no sense to rebuild something like slack in house, but it also makes no sense to pay 220k / year for it. the obvious evolution here is that slack gets replaced by a smaller company that doesn't have to charge 220k / year you don't want to have to deal with in house maintenance on every tool you use, but the cost of software is plummeting anyways

@ishansheth_ @varunram the question for me is whether their structural economics can allow it? supposing an agent-native team builds the same with far less capex, how will they compete?

@dremnik @varunram isn't that what notion is trying to do tho?

Will Linear become Notion first or will Notion become Linear first?

@rywiggs @mercury absolutely! will do

@dremnik @mercury Let me know if you have any feedback! Our API team has been cooking

there are 2 things that would enhance my life tremendously: - a programmable bank - private AI to sort through the transactions accounting is a machine task.

@hypersoren @juliarturc who are the few? 👀

@juliarturc 🤌👋👋 Agree. My sense is the (or much of) west coast tech is without ideas for abundance, grinding off an antiquated set of business models while telling incoherent stories of AGI There are a few tho who get it

Why so many of us feel career-homeless in tech: >Startups full of fraud, grifters and short-term thinking >FAANG full of politics and slightly behind >Frontier labs in a race with no morals >Academia full of title collectors >Content creation ridden by AI fakes and sensationalism Who is starting the renaissance and how do I get in touch with them?

my perspective on this is that we need schools to reorient like accelerators. theoretical knowledge is becoming a commodity, so the time to experiment has to shrink. it should be entirely project based, with maximum experimentation and room to fail, not the extractive debt-based model we have now. planning on writing a piece on this soon

We're going to need a whole new schooling system. An online learning AI can have a theory of mind for each student and do perfect curriculum fine-tuning. This will be a way for human brains to distill some of the intelligence of big models. Who's building this?

bet on openness bet on increasing entropy bet on a cambrian explosion