IndexFox

Websites used to have one audience. People who type a query into a search box, click a result, scan, and act. That's no longer the only visitor. Increasingly it's an agent. An agent doesn't scan. It queries. It expects answers grounded in your content, returned as data, in a single hop. If your site can't serve both, you lose half the traffic that matters. We've been building IndexFox for the case where the same retrieval layer powers human search and agent queries on the same site. The same index. The same relevance. Different surface. The new SEO isn't agents finding you. It's whether your site can answer them when they arrive. What shape is your site taking?

@gregisenberg MCP as new SEO is the right call. one underweight half: discovery gets the agent in, but if the endpoint doesn't have structured retrieval underneath, the agent gets back garbage and bounces. discovery + grounded retrieval is the full stack.

I just got back from SF and I FEEL INSPIRED. I spent 5 days with frontier AI model teams, AI startup founders, and 3 billionaires. My takeaways: 1. I had lunch with 3 billionaires. All of them are buying SaaS companies and rebuilding them agent-first. They were deeply inspired by Bending Spoons and Ryan Cohen's eBay deal. Buy the company, cut the headcount, rebuild the tech, add agents, add features, make more valuable experience, raise prices. 2. The frontier model companies are hungry for usage data from the field. They can see API calls and token counts. They can't see the actual workflows. If you're deep in a niche using these models in ways the model companies haven't seen, that understanding is incredibly valuable. Usage intelligence is the new alpha. 3. Consumer AI is massively underbuilt. Every billboard in SF is either B2B inference infrastructure or vertical agent companies. The entire city is optimized for enterprise. Meanwhile you have companies like Cal AI doing $50M ARR in 18 months as a consumer app. I met with a cool few teams doing consumer AI (@paulscherer / @ekuyda) 4. MCP came up in literally every conversation. The companies exposing their product as MCP endpoints are getting pulled into deals they never pitched for. The ones that aren't are becoming invisible to agents. This is the new SEO. If agents can't find you, you don't exist. Building products for agents is the new zeitgeist in general. 5. Not uncommon for hot seed rounds to be $25-50 million valuations. I saw a Series A at $450 million 6. If I had a dollar every time someone mentioned "forward-deployed engineer" this trip I could have funded a seed round. It's the hottest role in SF right now. The person who sits between the agent and the customer, making sure everything actually works. 7. The mood around open source shifted. A year ago it felt like open source was chasing the frontier models. Now founders are telling me Gemma and DeepSeek are good enough for 80% of what they need at a fraction of the cost. The "which model do you use" conversation is being replaced by "which model for which task." Model loyalty kinda feels dead. 8. Voice agents came up more than I expected. Multiple founders told me voice is the interface for the next billion users. The billion people who will never type a prompt will absolutely talk to one. 9. The Obsidian community in SF is weirdly intense. Multiple founders showed me their vaults unprompted. Like showing someone your home gym. It's a flex now. The quality of your knowledge base (second brain?) is becoming a status symbol among builders. 10. Maybe it was just the people I met but the age of the founders is shifting. I met more founders over 40 this trip than any trip before and more founders under age 21 than ever before. Founders getting older and younger at the same time. 11. I spoke to a lot of fast-growing startups, VCs and frontier models who are hiring content creators right now. 12. The restaurant scene in SF is actually better than it's been in years. Founders are going out more. Alcohol is out, not surprisingly. 13. SF doesn't feel like the only place anymore. We all have access to the same frontier models. We all read the same X feed. A founder in NYC or Lagos is calling the same APIs as a founder in SoMa. So in the past it felt like SF was always lightyears ahead, doesn't feel that way anymore. It's okay not to live in SF and have BIG DREAMS. 14. The coworking spaces in SF are half empty but the coffee shops are packed. People want to be around people. I had a few startup ideas here.... 15. Walking around the Mission I noticed something: the street-level businesses, the taquerias, the barbershops, the laundromats, none of them use any AI at all. 16. I heard the phrase "agent debt" for the first time. Like technical debt but for agents. When you hack together an agent workflow fast and never clean it up, the system prompts conflict, the memory gets polluted, the tools overlap. 6 months later the agent is doing weird things and nobody knows why lol. 17. Met a few people who carry two phones now. One for personal. One that's basically an agent terminal running Telegram or iMessage connections to their agent fleet. It's always amazing to get that dose of inspiration in SF. I FEEL INSPIRED. But I'm so happy to be back home, locked in and building. We're 12-18 months into a shift that will take 15 years to play out. The urgency in every conversation was real. What an incredible time to be building.

@garrytan cerebellum reframe is the sharp half. most agent work today is one prefrontal-cortex pass per question. the cheaper move is pushing recurring lookups (search, doc fetch, schema) into a deterministic layer the model doesn't think through every time.

Everyone building AI agents is focusing on building the prefrontal cortex. Planning. Reasoning. Multi-step chains. There's value here. CEO-stuff. But also, a reframe: there is value in building the cerebellum. It's offloading boring tasks into reflex so the complex thought can focus. Your mortgage gets paid by a standing order, not a committee. The things that are not fun, not interesting, but have to be done? Done. Most agent frameworks will fail because they treat all cognition as high cognition. The winners will nail the boring stuff first.

@cyrilXBT 5 agents only multiplies if each has reliable retrieval. otherwise you compound the same wrong context five times instead of catching the bug once. orchestrator and reviewer roles only earn their keep if the context they're reading is actually grounded.

MOST DEVELOPERS ARE RUNNING ONE AI AGENT AND WONDERING WHY IT FEELS SLOW. The engineers at Anthropic are running five simultaneously. One codes. One tests. One reviews. One deploys. One orchestrates the whole team. 30 minutes to build the entire system live. Here is exactly how they do it and how you can too.

@rohit4verse 'agent isn't a tourist' is the line. one detail that compounds: the folder needs to feel cheap to maintain or the agent quietly stops trusting it as it goes stale. re-index on every doc change (not on a cron) keeps long-term memory honest.

The retrieval layer is where 90% of agent projects quietly die. Karpathy’s wiki pattern is pure gold because the agent isn’t a tourist constantly googling the same docs , it owns a single, re-indexable folder that becomes its actual long-term memory. In the setup I posted, `program.md` + git history is literally that “one folder” version. No vector DB, no Pinecone, no extra moving parts. Just pure ownership. The agent wakes up, reads its own structured notes, runs the experiment, and writes the result back. That loop is stupidly effective. Have you tried any specific re-indexing triggers that worked especially well for you?

Every night you're not running an autonomous research agent, you're hand-running experiments someone else automated months ago. Most people are still hunting for the "right" setup. Frameworks, orchestration, glue code. You don't need any of it. Andrej Karpathy open-sourced his own version that runs its own ML research. One GPU. ~100 experiments overnight. You never touch the Python. Here's the exact setup (takes 2 minutes): 1. Clone it: (repo link in comments) 2. uv sync, then uv run prepare[.]py 3. uv run train[.]py once to confirm the baseline runs 4. Point your coding agent at program.md and walk away The agent edits one file, trains 5 minutes, keeps the change if val_bpb drops, reverts it if it doesn't. Git is the memory. The metric is the judge. You wake up to a staircase of validated improvements, not a backlog of ideas you never tested.

this framing finally explains why teams that 'switched to context engineering' saw retrieval costs go up before they saw quality go up. RAG-2020 was one retrieve per query. context-engineering iterates, retrieves multiple times per turn, and rewrites mid-stream. the cost shape is different, not just the architecture.

Is context engineering the new RAG? (Hint: They are both about 𝗯𝘂𝗶𝗹𝗱𝗶𝗻𝗴 𝘁𝗵𝗲 𝗟𝗟𝗠’𝘀 𝗰𝗼𝗻𝘁𝗲𝘅𝘁) 𝗥𝗔𝗚 (𝟮𝟬𝟮𝟬-𝟮𝟬𝟮𝟯): One-shot retrieval • Fixed retrieval pipelie • Always retrieve context (whether you need it or not) • Retrieve exactly once (even if you need more info) 𝗔𝗴𝗲𝗻𝘁𝗶𝗰 𝗥𝗔𝗚 (𝟮𝟬𝟮𝟯-𝟮𝟬𝟮𝟰): Multi-hop retrieval through tool usage • Retrieval becomes a tool the agent can choose to use • Agent decides: Do I need to retrieve? Is this context even relevant? Do I need more? • Can route to different indexes 𝗔𝗴𝗲𝗻𝘁𝗶𝗰 𝘀𝗲𝗮𝗿𝗰𝗵 𝗶𝗻 𝗖𝗼𝗻𝘁𝗲𝘅𝘁 𝗘𝗻𝗴𝗶𝗻𝗲𝗲𝗿𝗶𝗻𝗴 (𝟮𝟬𝟮𝟱+): Combine multiple retrieval tools for context engineering • Context is scattered across different context sources (database, filesystem, web, memory) • Combine different context retrieval tools • Agent builds its own context Essentially, we’ve moved from ‘retrieve once’ to ‘the agent builds its own context’. Yesterday, I gave a workshop on this topic. Find the slides and code on this in my GitHub repo: github.com/iamleonie/work…

Builders are converging on a quiet consensus this week. The retrieval layer is where most agent projects fail. Frameworks come and go. The harness does more work than the model. Memory and skills end up being the same lookup at different time-horizons. Zoom out and it lands somewhere simple. Retrieval is the durable layer. Everything above it is the marketing of the moment. This is what we keep finding shipping IndexFox. The chat UI, the identity file, the orchestrator are all fine. What decides whether the product feels usable is whether the right page is in slot one when the visitor or the agent asks for it. What did your week converge on?

@tk_armand vibe + tests is the right combo. left-menu icon overflow usually means nav is doing two jobs (navigation + tool launchers). keep 4-5 primary icons, push the rest into a contextual drawer or inline actions. linear and notion both split it eventually.

@indexfoxai Thanks! Actually its a combination of the 3. I go by vibe more than I'd like to admit but theres still a long way to go, and im testing constantly. Each week I add something to the UI. Right now im thinking about a new layout since the left menu icons have very limited space.

Trying to connect with more people building internet things. SaaS AI tools Automation Web apps Design Product development Indie startups Drop your project below.👇 Especially if it’s early, weird or still ugly.

@Al_Grigor agreed, RAG first. the trap most fall into: when RAG returns the wrong page they conclude 'retrieval doesn't work' and reach for fine-tuning. usually it's a chunking or freshness bug, not retrieval. fine-tuning is right for tone and format, not knowledge.

Most AI engineers should learn RAG before fine-tuning. Both approaches can help with domain-specific knowledge, but they work differently. With fine-tuning, you change the model weights. With RAG, you keep the model as a black box and give it relevant context at request time. For many applications, RAG is the more practical first step. 1. It is easier to update. If your knowledge base changes, you update the documents or the index. You don’t retrain the model. 2. It works with different LLM providers. The knowledge layer is separate from the model. 3. It is also easier to debug. You can inspect retrieved documents, prompts, answers, and failure modes. Fine-tuning has valid use cases, but it requires more infrastructure, more specialized tooling, and more careful evaluation. My default recommendation: start with RAG. Use fine-tuning only when you have a clear reason that retrieval and prompting cannot solve. Use my workshop recording to build your first RAG application: youtube.com/live/KSItlTAsM… Want to go from LLM basics to a production-ready AI assistant in 10 weeks? Join my free LLM Zoomcamp that starts on June 8: github.com/DataTalksClub/…

@kenwuuuu depends what the agent does. for retrieval-heavy work (docs Q&A, internal search, RAG over your content) teams skip the framework and write orchestration directly. langgraph hides the parts of search worth tuning. for tool-call-heavy agents (coding, tickets) it still earns it.

so what is the current industry standard to build agents? do people still use langchain/langgraph?

@dharmesh agreed, the harness is where most of the work hides. one underweight piece: retrieval that knows the boundary between memory, skills, and context. agents reading the wrong store at the wrong time look like model failure but it's a harness bug.

The harness matters more than the model. Models have gotten really good. Great reasoning, large context windows, better instruction following. But, what makes *use* of those capabilities is actually the harness. It's what provides tools, memory, skills and context to the model. ChatGPT is a harness. Claude Cowork is a harness. Without the harness, the model is just an engine with no car. You don't get anywhere.

@ravikiran_dev7 architectural-bureaucrat is right. the place review burden compounds fastest is anywhere the agent reads context (search, docs, schema). wrong context shows up as confident wrong code. cleaning retrieval cuts review hours more than tighter prompts ever did.

vibe coding is officially dead ! I had to say it. we thought AI would let us relax and code "on chill", but instead it turned us into architectural bureaucrats. we write strict laws, define rules, limits, and principles. But if you don't obsessively review the code that agent writes, your project will mutate into a massive landfill of tech debt before you even realise it.

@mattpocockuk the BM25 detail nobody talks about: it's the retriever that works best when content vocab matches query vocab. that's most internal docs and site-search workloads. hybrid only earns its weight when queries drift from how the content is actually written.

When you think retrieval/RAG, most folks immediately jump to embeddings and vector db's. Everyone forgets about BM25 - a search engine workhorse:

@DanKornas taxonomy is solid. the bit most builders skip: memory and retrieval aren't separate stages, they're the same lookup at different time-horizons. picking a vector store for one and a buffer for the other usually means re-deriving the same context twice.

Most agents don’t need more prompts. They need memory that fits the job. Agent Memory Techniques is a hands-on GitHub guide to memory patterns for LLM agents, organized as 30 runnable Jupyter notebooks. It helps you choose and test memory designs by grouping techniques into short-term, long-term, cognitive architecture, retrieval/routing, framework, and evaluation/production sections, with a decision tree and comparison matrix for navigation. Key features: • 30 runnable notebooks – covers conversation buffers, vector stores, knowledge graphs, episodic/semantic memory, MemGPT, Mem0, Letta, Zep, Graphiti, LoCoMo benchmarks, and production patterns • Six-family taxonomy – splits memory into short-term, long-term, cognitive architecture, retrieval, framework, and evaluation/production tracks • Decision tree for builders – points you toward the right technique based on current chat, cross-session memory, frameworks, or production/evaluation needs • Comparison matrix – helps you filter techniques by persistence, retrieval style, token cost, and best-fit use case • Notebook-first learning path – starts with Conversation Buffer Memory and links each technique to runnable notebooks and Colab It’s open-source under the Apache License 2.0. Link in the reply 👇

@dhanushnagineni 🤝 same. what stage are you at, exploring or shipping already? bio's broad (startups + UI/UX + software dev), give me the narrow version

@indexfoxai Let's connect

Founders, builders, hackers — what are you shipping right now? Looking to connect with people building in: • AI & SaaS • Tech startups • Open source • Product development • Web & mobile apps Whether it’s an MVP at 2AM, a side project nobody believes in yet, or something already scaling — drop it below 👇 I genuinely want to see what smart people are building before the weekend hits. Let’s connect, exchange ideas, and build cooler things together. #BuildInPublic #Startups #SaaS #AI #TechIndia #OpenSource

@tk_armand 🦊 ha appreciate it. just checked beetter.co. the conversion-focused frame beats the linktree list-of-links shape. UI iteration is the worst part of shipping for me too. how do you tell when 'feels cleaner' actually lands, vibe or test?

@indexfoxai Also love the little fox fella. And yeah feel free to check my site out, im constantly trying to improve the UI UX. Drives me crazy sometimes

the structural read is right. the part that's underrated in this shift: the websites for these AI-native legal firms will need to act like product surfaces, not brochures. visitors come asking specific questions, expecting answers grounded in the firm's own docs. legal sites are going to ship product-shape search before most realize they need it.

Two things happened in legal tech this month that almost nobody on legal X is connecting. May 13. Carta acquired Avantia, an alternative legal services provider with 200+ asset manager clients supporting $15 trillion in assets under management. Rebranded as Carta Law. AI-native by design. The first major financial-services platform to formally absorb a law firm. May 20. Antti Innanen launched Lavern. An open-source "agentic law firm." 67 specialist agents. 155,000 lines of code. Mandatory human gates. 10-pass verification loop. Apache 2.0. Different companies. Same signal. The next wave of legal tech is not a tool you bolt onto your firm. It is a firm. For BigLaw the message is: a private-capital platform just integrated an entire law firm into its product. Clients can now route legal work through the same dashboard they use for cap tables. The unbundling has begun. For solos the message is different. Lavern's source code is Apache 2.0. The 67-agent architecture, the verification gates, the workflow orchestration - all of it is publicly available. A solo who studies the Lavern repo for one weekend now has the architectural reference for what an AI-native practice actually looks like. The solos who deploy real agent workflows in 2026 are not just running better practices. They are becoming the small AI-native firms the next Carta-style platform will acquire. The consolidation wave runs through small firms with built workflows. Not through small firms with the same setup they had in 2022. If you are building toward this DM "audit." 2 May slots are open. Cap is real.

@sreenandhanpp 🤝 the waitlist phase is the hardest part. what's the blocker right now, scope or feedback?

@indexfoxai Still waiting list🥲

Good morning ☀️ Looking to #connect with: 💻 Developers 🚀 Indie Hackers 🧑‍💻 Solopreneurs 🎨 Designers Let’s connect 🚀 #buildinpublic

@charliejhills the plug-in-everywhere distribution model is the underrated half of this. agents won't matter if every product needs a custom integration to install them. the win is when the agent runs the same on cowork, code, api, and the site you're already on. retrieval portability is next.

Anthropic just shipped Claude's 10 finance agents. Available in Cowork, Code, API, and Office. How to install in 4 steps. 1. Install in Cowork. - Open Settings → Plugins → Add plugin. - Paste: github.com/anthropics/fin… - Pick the agents you want from the list. 2. Install for Microsoft Office. - Open the GitHub link (above). - Copy the install command into Claude Code. - Run /claude-for-msft-365-install:setup to finish. 3. Connect your data sources. - There are 17 data partners at launch. - Add the ones you pay for as connectors. 4. Pick one. Run today. - Map one agent to a job on your plate this week. - Paste the prompt. Edit it. Run it. Try these prompts: ✦ pitch-agent Pulls comps, precedents and LBO numbers into a branded pitch deck. "Draft a 12-slide pitchbook for our acquisition of [TargetCo]." ✦ meeting-prep-agent Pulls past notes, recent news and talking points into a one-page brief. "Build a one-page brief for tomorrow's 10am with [Client]." ✦ earnings-reviewer Reads earnings reports and flags the surprises and risky wording. "Summarise [Ticker]'s Q1 earnings and flag every surprise vs forecast." ✦ model-builder Builds a working financial model in your spreadsheet from one prompt. "Build a valuation model for [TargetCo] vs six similar companies." ✦ market-researcher Pulls sector trends, competitor moves and pricing into one memo. "Write a 1,000-word memo on European fintech lenders." ✦ valuation-reviewer Audits a valuation model and challenges every assumption inside it. "Review the valuation model on [sheet]. Challenge every assumption." ✦ gl-reconciler Matches your books against bank statements and flags any mismatches. "Reconcile our bank books against last month's statement." ✦ month-end-closer Runs your monthly accounts checklist end to end and flags any issues. "Run the April month-end close on our standard checklist." ✦ statement-auditor Audits the books for errors and control gaps before they ship. "Audit April's P&L and balance sheet against our books." ✦ kyc-screener Vets new clients against watchlists and ownership records. "Run a background check on [NewClient]. Pull ownership records." Free Claude playbooks → charliehills.substack.com Repost ♻️ to help someone in your network.