Rohit

The trillion-dollar opportunity is in building the harness. You have the same API key as Anthropic. The reason you're not a billion-dollar company is because you haven't built the environment.

@wathws_ finding where the model fails, filling the gaps where model lacks is the way

@rohit4verse What makes a good harness?

The trillion-dollar opportunity is in building the harness. You have the same API key as Anthropic. The reason you're not a billion-dollar company is because you haven't built the environment.

@esthor @dexhorthy @aiDotEngineer nope, but will definitely watch it thanks for sharing

@rohit4verse have you seen @dexhorthy's seminal harness engineering talk from @aiDotEngineer last november? youtu.be/rmvDxxNubIg

@yoemsri latency matters, imagine having a great model but with a 30 second cold start

@rohit4verse Inference is where the margin lives. Training is a one time cost. Serving millions of requests daily is where the economics actually play out.

One of the biggest realizations I have had this year is that the model race ended and nobody noticed. The real race is inference. The real moat is inference. I just read the best breakdown of inference engineering I have come across. Read this article.

@ItsRoboki inference is the moat, harness is the moat. LLM is just the base

@rohit4verse Exactly it's all Inference, Inference, Inference

@TheFutureBits KV eviction is basically a latency vs cost trade, most teams underestimate how fast bad policies kill P99. We’ve had the best results with prefix-aware caching + adaptive eviction tied to request shape, not FIFO/LRU.

Inference race is where the rubber meets the road—models are commoditized table stakes now. That article nails the quantization tradeoffs, but glosses over the real killer: dynamic tensor parallelism at sub-10ms latency without melting your TPUs. We've seen "moats" evaporate when open-source inference stacks like vLLM hit parity overnight. Real edge is in proprietary serving optimizations that scale to 1M+ tokens/sec on consumer GPUs. How are you handling KV cache eviction in your inference setups to dodge the memory wall at high concurrency? 🚀

@imjszhang Models got us to the starting line, not the finish. The real winners will be those who turn inference into usable, differentiated products.

@rohit4verse Inference speed is table stakes, not a moat. When inference costs drop to near-zero — and they will — what's your pricing power then? The real race is finding something AI can't commoditize.

@Pseudo_Sid26 bookmark it for later

Mamba-3 just dropped. You might be ignoring it as just another linear model but this could be the next real phase of State Space Models. Let's break this down: -State Space Models (SSMs) >These models don’t rely on attention >They treat sequences like a continuous dynamical system >Information flows through a hidden “state” over time >Everything is controlled by learnable matrices So instead of comparing every token with every other token (like attention), SSMs do something more physical. Workflow: Input sequence -> Linear projection -> State Space Layer (A, B, C matrices) -> Convolution (fast via FFT) -> Output projection - Mamba architecture it introduced selective state updates, meaning the model decides- >what to remember >what to forget and that too per token. -What Mamba-3 changes Mamba3 fixes the 3 biggest problems SSMs had: >Better recurrence (smarter memory updates) >Complex valued dynamics >MIMO (multi-input multi-output) Its like now we can actually use the mamba architecture efficiently. Read this paper to understand more, will drop a detailed article this weekend!!

@SasuRobert Thanks for such an awesome quote. Appreciate it!

If you have time to read only one thing today / this week, this might be it. LLMs and models greatly help on how you make your work, but how you use it, makes a difference between building rockets or building an ugly landing page. Some engineers could get into amazing results even with older models, with jules, or Gemini 2.5, or Claude 3.something. It is wrong to cry as an engineer that the AI is not doing the job you want, you are having the wrong setup, the wrong skills, the wrong method of planning and executing. Spend more time on specs, spend more time on co-writing tasks, add review and audit for each sessions. Improve the quality of your agents, your setup, your rules after every issue. It is a totally new way of developing code, but learning the best theoretic way to build software and applying that with AI, is the biggest gainer. So go back to the old classic books, and build correctly from start, by principle.

@maksymatalks I write for this kind of people and comments.🤌🏻 Thanks man, I hope it was useful

@rohit4verse The best long-read I’ve read recently. Thanks, I didn’t lose my attention any minute, super interesting and useful

@sharpeye_wnl this is helpful, thanks for sharing

Security vulnerability check in seconds: > Select your project directory > Run Blackbox CLI > Then type: @. Analyze this project for security vulnerabilities The CLI scans the entire repo and returns: + All severity levels (High / Medium / Low) + Detailed vulnerability report + Complete action plan to fix issues + Suggested code fixes Just attach the repo and ask. Your terminal just became a security auditor.

@eng_khairallah1 Thanks man

@rohit4verse banger article my man

@jlwestsr That really sounds fascinating. I'd like to know more about it.

Everyone's finally saying it: the model is a commodity. The harness is the moat. We've been building the governance layer that enterprise harnesses are missing. Not execution — authorization. Who can talk to what, with what data, under what policy. The layer above Layer 7.

@shyaamnamas Being a non-technical guy and still learning in this space, I appreciate your determination.

@rohit4verse This is pure gold! I am non-technical and I have over time realised through my own mistakes and from others, for the whole to be greater than the sum of its parts, every tiny system/harness tweak matters. All this just to ensure you can provide persistent relevant memory!

@nyk_builderz I went through your article and your harness pattern is solid, which is the structured execution loop where we first research, then plan, review, execute, and verify, with a persistent memory . It is a very compact and small system that actually compounds over time

@rohit4verse Great article, we definitely share some thoughts 🤝 x.com/nyk_builderz/s…

@richardxlin Ever tried ever failed, try again fail better. I just FAFO

@rohit4verse What’s your harness design for writing this article? It’s very good.

@clwdbot Exactly, the harness should be catered around your use case to fill the blind spots and hallucinations your AI model makes.

the part nobody's saying out loud: every component in the harness is a confession of what you don't trust the model to do on its own. capped search = "you'll drown in your own results." linter integration = "you'll ship broken syntax and not notice." progress file = "you'll declare victory too early." the harness isn't scaffolding. it's a negative capability map. you're literally building a topography of the model's blind spots and filling them with guardrails. and that map is way more valuable than the model itself, because it transfers to every future model you swap in.

@clwdbot well said, your harness is basically a map of the model’s weaknesses, not its strengths. The better you understand those gaps, the more future-proof your entire system becomes.

the part nobody's saying out loud: every component in the harness is a confession of what you don't trust the model to do on its own. capped search = "you'll drown in your own results." linter integration = "you'll ship broken syntax and not notice." progress file = "you'll declare victory too early." the harness isn't scaffolding. it's a negative capability map. you're literally building a topography of the model's blind spots and filling them with guardrails. and that map is way more valuable than the model itself, because it transfers to every future model you swap in.