Anto
516 posts

Anto
@anto_edd
Full-time marketer at a billion-dollar company. Weekends spent building @Hire_AIHuman @theofficial_app @snoopdoc_app



Panama City, Panama 🇵🇦

been building in this space for years now, and have followed nishkarsh for years as well - congrats on the launch! since this is in the same space we're building in, i dived deep into it and have thoughts. the launch itself is very hype-y, and is meant to trigger rage bait 1. it's positioned as a database, but is almost a @supermemory-like system 2. their example of "vector dbs" not being able to do this, is really a question of "embedding models". and embedding models have superpositions, they are cheap and are easily able to infer differences between them. it's not hard to ask claude to do a mini experiment to prove this (attached below). What does matter is: is it able to track how knowledge evolves? time passes? this made me curious so i read their paper 3. their research paper is hardcoding and gaming the benchmark by different prompt for every category!!! (see image below). If their benchmarking is fixed, supermemory will remain the SOTA. 4. they reinvented contextual retrieval paper by Anthropic from 2024 and called it "the orphaned pronoun paradox" 5. they mention they use a custom "in-memory vector store" = at about 500GB, you will have to pay more than $10k for just the RAM. 6. inference is run too many times in the pipeline - which means for every LLM token you ingest, you will end up paying 5x more than token cost for the graph + contextualization + storage. 7. latency and cost numbers were never reported. My hunch is because of the architecture, the latency will struggle at scale. but i can't tell - their product is behind demo gate. 8. the benchmarking code is not OSS (from what i can tell). not replicable + who knows how much context they are injecting into the model? what's the K? 9. inorganic, undisclosed ads (just read the quote tweets). influencer accounts with 400k+ followers all saying the same thing. people keep getting away with this @nikitabier lol i'm all in for healthy competition and progress in this fields, enjoy seeing good work being done by others. but its easy to just say things. "no one will check." playing the game the right way is hard, and everyone's just saying whatever they can to impress people. TLDR is: you should use this if you want to spend 2-5x more for no real marginal improvement and enjoy unhealthy research and business practices. attached: 1. experiment to disprove hypothesis of vector dbs not understanding grey vs grey 2. one of their prompts, which just says "say i dont know". they scored 100% :)






















