Priyam

The last few months at @metacognitionai taught us more through failure than any win ever could.... We didn't always come out on top tbh.. we failed more than we succeeded. But every setback showed us exactly what we were missingin the hardest way possible. Here's what that journey looked like and what we're building because of it : - got rejected by @speedrun , twice, saying this with a heavy heart ngl. - got our papers on dynamic memory and forgetting systems accepted at ICMNAI (neuromonster'26 ) and @IEEEorg ICACI 2026. - couldn't attend ICMNAI at Rome cause of last minute Visa issues and alot of delays caused by other parties. - @metacognitionai got to attend @ycombinator Startup School, Bengaluru, were we connected with more than 80+ industry experts and founders which helped us close more design partners. - Got accepted at @nvidia Inception Program, infinite GPU lessgoooo - Working clsely with Prof. frank coyle and other labs at @UCBerkeley on bridging the gap between Biological Intelligence and todays AI systems. - got into the fellowship by @thought_channel , really love this community and network ! - didnt get accepted for @ycombinator S26, we somewhat identify what we are lacking and we are all set for F26 !! - @sauhard_07 was among the top winners at the Agentic Arena Hackathon by @localhosthq , Bengaluru. - got accepted at Off Season II by @fdotinc 🎉 lfggg, really excited for the opportunities there. - we also got accepted at @SarvamAI Startup Program , this will help us build better products faster. We as a company are working in the edge of something that is thought to be impossible and we are really fixated as a small team that we can solve the problem of AI Memory and hence solving Emergent Intelligence. Currently we are working on our v1, and this is something you really wont expect to exist. What if there was someone who understood you, your choices, your wants more than you itself at an emotional and behavioural level ? Stay tuned we will drop more about this soon. Also Tex (v0) is live and running at enterprises. If you are an company looking for the best memory solution in the market and really care about your data privacy. Reach us out and we can help you walkthrough our solutions. We are also actively looking for Researchers in the field of Computational Neuroscience, Computational Vision or Neurophysics, we would love to collaborate and build towards the vision of our company here at @metacognitionai For further information about our track of research, reach me out !! Would love to have a fruitful conversation. all kudos to the team : @venky1701 @sauhard_07 Shyam & Tauqueer

solidgoldmagikarp ( ifykyk )

@omarsar0 skeptical about it.. calling recursive loops for bandaid long context and calling it "self improving" isnt "metacognition"..

Highly-recommended overview of metacognition in LLMs. (bookmark it) Interesting behaviors in LLMs like confidence calibration, self-verification, knowing when to stop, and knowing what you do not know have mostly been studied in isolation. This survey argues they are facets of one thing, metacognition, and proposes a comprehensive map of it. The authors taxonomize methods and benchmarks for measuring and evaluating metacognitive abilities in LLMs, then connect those abilities to capability, reliability, and transparency. As agents take on longer horizons, the ability to monitor and regulate their own reasoning becomes an important way to measure reliability. Paper: arxiv.org/abs/2607.11881 Learn to build effective AI agents in our academy: academy.dair.ai

boy at work 💪💪

@Sanyam0605 interesting.. diffusion based world models and diamond is particular is really fascinating .. work by @theworldlabs is really so cool

Hi everyone I m working on world models and diffusion mainly, on how to use them to make game engines and render frames based on actions (Rn restricted to 2D grid games) There is some work in this direction like Diamond, GameNGen , if you know others do comment the arxiv link I want as tiny as possible the model should be, st latency of next frame is in 'ms'.

This led me towards thinking of HMTT in two stages. The first would be a clean and completely lossless HMTT-Core that preserves UTF-8 bytes, whitespace, source offsets and formal grammar boundaries correctly. The second would go beyond ordinary BPE by giving the model separate but aligned representations for: - the exact surface text - the mathematical or code structure - numerical value, magnitude, precision and physical units For example, a scientific value such as 9.81 m/s² should preserve its exact digits while also informing the model that it represents acceleration with the dimensional structure L·T⁻². I think this could be a much more interesting direction than simply creating a larger vocabulary and hoping it memorises every common mathematical expression. The real question I now want to test is whether a lossless, grammar-constrained and value-aware representation can make formal reasoning more computationally accessible to a language model under the same data and compute budget. There is still a lot to correct, implement and experimentally prove, but that is what makes this problem quite interesting. I would love to know what other people working around tokenization, mathematical reasoning or scientific language models think about this direction.

After this, I went back and properly examined the current implementation of HMTT. The basic idea behind HMTT is to first identify whether a region contains natural language, mathematics or code, and then use a separate structure-aware pre-tokeniser for each region before training one unified BPE vocabulary. While reviewing it, I found a few important things that made me rethink the architecture... - separating pre-tokens using whitespace prevents BPE from merging across their boundaries, but it does not necessarily prevent BPE from splitting an individual mathematical atom internally. - a fixed vocabulary cannot store every possible number, variable or identifier as one token. There are infinitely many possible numbers and identifiers, so these should probably be represented compositionally rather than memorised. - whitespace, indentation and mathematical delimiters must be preserved exactly. Otherwise, the tokeniser cannot be truly lossless, and code such as python can lose its actual structure. - treating a complete expression like \theta^{(t)} as one token may reduce fragmentation, but it may also hide the useful internal relationship between \theta, the superscript and t. - the actual problem is not necessarily that BPE destroys information, because byte-level BPE is reversible. The more accurate question is whether it makes mathematical structure harder for a model with limited capacity to access and reason over.

Yesterday, I spent most of my time trying to understand one of the most ignored yet very important components of a large language model.... the tokeniser. The question I had was quite simple.. if normal tokenisers are mostly trained over natural language, how efficiently can they represent mathematics, scientific expressions and code, where every operator, variable and structural boundary can carry a completely different meaning? So, as a very curious person, I started from the basics and eventually came back to something I had previously been working on called HMTT, or the Hybrid Math-Text Tokeniser. Link to paper --> (researchgate.net/publication/39… ) Some of the things I worked on and understood: - broke down Byte Pair Encoding from scratch and understood how it repeatedly finds the most frequent adjacent pair, merges it and continues building its vocabulary. - understood how UTF-8 converts raw text into bytes, allowing Bengali, mathematical symbols, emojis and almost every other character to be represented using the same initial vocabulary of 256 bytes. - built a simple BPE implementation and looked at why its naive training and encoding complexity can reach approximately O(n × m), where n is the input length and m is the number of learned merges. - explored optimisations such as caching repeated words, tracking pair locations, updating only neighbouring pairs after a merge and using heaps instead of rescanning the complete corpus every time. - compared character-level BPE with byte-level BPE and understood why byte-level tokenisation provides a much safer fallback for multilingual and unseen text.

Personalization starts with understanding what primordial intelligence is. Primordial intelligence, in neuroscience's terms, is the purest form of intelligence, the simplest of all. But how I define this to be would be - the minimal capability of a system to build an internal model of its environment from experience, retain it through memory, predict future states and adapt its behavior accordingly. ( inference I've taken after reading a brief history of intelligence) A system that predicts the future states to minimize surprise between two events is a system that achieves primordial intelligence. Think of a newborn baby, a month old. His brain tries to understand it's environment. Fast forward this to maybe some years, he forms memories from experience, then patterns emerge from those memories, these inturn improve predictions and eventually the actions of the baby adapt over a period of time. This is the stage before the reasoning even starts developing. A toddler doesn't need to know physics or gravity to catch a ball. They spend thousands of interactions to develop their own internal predictive model of gravity, motion and timing. The point I want to make here is - Memory and Prediction is much more fundamental than reasoning. The baby becomes intelligent by experiencing the world and not by performing logical operations. Now replace this baby by your ai agents. And this is what @metacognitionai does with your ai agents.

@kseniam0s MetaCognition

Give me a startup name SO COOL it sounds like the next unicorn 🦄

Ever since the research on a global workspace in language models or J-Space came out, I have been more intrigued by understanding what actually is the similarity between how reasoning and parallel thinking actually work in neuroscience, with this global workspace theory that is based completely on computational neuroscience. So, as a very curious person, the best thing I could do was to understand what, at core, mechanistic interpretability is about and how we can actually decode the internal thinkings of large language models, which we know are a complete black box. How do we understand this dark matter, neural Networks? @ch402 Blog post ( transformer-circuits.pub/2024/july-upda… ) Talks about how, internally, what a large language knowledge thinks can be interpreted, and why interpreting it is very much important. Well, I tried exploring some things because this is a new field that I am trying to understand, and so I would be listing down what I thought are really important and things that I found interesting. - if we take a model like gemma 2b or qwen 3.5b and try to get its residual stream its easy to see and comprehend what the next predicted work be like. - And if we actually try to tweak every neuron in each of the layers of a particular model, then we actually see a behaviour called polysemanticity, where a single neuron can control multiple features in a model. - A research from, @AnthropicAI , in 2023 proposes a theory of SuperPosition, which I thought would be similar to what it is in physics and quantum mechanics, but in very crisp and easy terms, what it said, or what I was able to interpret, is that even if there are fewer neurons, those neurons are capable enough to control more features. Somehow, an internal learning is happening in which fewer neurons are able to comprehend more features. So how do we actually understand what actually happens in every layer of thinking? For that, we actually use something called sparse autoencoders. To showcase the architecture of it : It pulls out the neuron outputs, multiplies it with a weight matrix, and converts it into a feature vector. Now, what is the meaning of sparsity here? What it does is it forces most of the terms in the feature vector to be zero. We try to reverse engineer and try to find out if we can reconstruct the neuron outputs that came up first. And if the reconstructed neuron output is similar to the initial neuron output during activations, then it will show the work of superposition is correct and the sparse auto-encoders work. Well, the results of applying a sparse autoencoder to the nth layer of the model confirm that it successfully reconstructed the original neuron output while simultaneously mapping them to the correct interpretable sparse feature. The result demonstrated that the SAE was able to take the neuron activation and map them through a set of learned weights into a higher dimension feature space, and then successfully map them back to the original neuron output with minimal loss. By identifying these specific features, researchers can steer or intervene in the model's behaviour directly by clamping those feature values effectively and controlling output like tone, scepticism, or in a predictable way. So what I would be trying to replicate is trying to build a sparse cross encoder and build a similar SAE over a Gemma 2B model to understand the concept of steering, which would at the end help me comprehend better what the idea of J-space is. I would love to know more about what other people think about this, because I am very much eager to learn in this field. It's quite interesting and a really fun way to understand how biological intelligence and the internal thinking or reasoning of a large language model can give us the ability to extract way more information than actually what we do.

At the very core, this was what I explored recently. I'm still planning my hypothesis and doing a proper real analysis on it. How can we actually prove that noise and a true memory, even if they have a similar predictive error, can show different geometrical structures in the latent space? This is the core work I've been doing. Apart from that, there are some ways in which we can actually check how well structured these clusters are to distinguish between noise and a proper true memory. Some concepts of latent space clusters for real-world anomaly detection. Again, these are some claims done in other parts of the systems, like in VAEs or variational autoencoders. This is what I will be doing for the next few days, trying to prove my hypothesis

As we know, JEPA-based feature prediction encourages the predictor to discard hard-to-interpret scenes and preserve predictable structure. A lot gets lost because of this. The minute noises that we actually talked about, which can be very important, are critical and get removed intentionally because it's hard to interpret. That's why it doesn't understand incremental displacement during a scene, which causes information loss. This whole system, in which it doesn't perform well, is known as slow gradual drifting. This is a fundamental bottleneck of current JEPA models. This is also baked in neuroscience: even in the human brain, we have this deliberate condition of gradual change blindness because observers fail to notice a substantial change until a big transition actually happens. So what are the ways in which we can actually prevent this? What we can do is think of this scene as an object and its positioning that is changing. We can use them as separate scene embeddings. All the scenes that we are seeing as incrementally changing can be used as separate scene embeddings, and they would have the same images but not the same causal history. Again, there is a lot of things I have to explore. This is just some ideas that I have currently which might work, but I have been working closely on this because my hypothesis is that, through geometry, we can actually interpret what noise and true memory could be in a latent space. This depends on how we can interpret them properly. So here comes the concept of slot attention. ( arxiv.org/pdf/2006.15055 ) Which, in simple terms, helps us learn exchangeable slots that can bind different objects.

For the last few weeks, I have been tinkering with world models a lot because it is necessary for embodied intelligence in the way ahead. Specifically, I have been exploring what could be the possibilities in which we can allow cross-episodic long-horizon planning and memory-based tasks in world models. That way, not just robots but AV systems that we build upon can be also used by companies and infrastructure that require abstract and proper understanding of the environments. Companies like companies like @temple use a kind of a world model of the particular person and keep a check upon it. At @metacognitionai , we have been trying to do something towards persona world models. While going deep into the study, I found that there are the top 5 major problems that don't allow world model JEPA-based architectures to work across episodes: - Auto-regressive error compounding - Noise vs true memory interpretability - Lack of a write mechanism - Return visit degradation - Drift blindness What I explored was that a world model is just not about the perception. It is equally more important to work on planning and memory. A lot of people, naive people, think that planning and memory are two different things, but planning requires memory. They are coupled with each other till the very end. They are required for active inference and closed-loop planning. Memory is the most crucial part. Now, what my idea was: how can we actually work on cross-episodes on that? I have been reading about MOP JEPA a bit, just started with it, but nothing major. What I found out is that there is an intrinsic notion of consequential changes. They do understand incremental changes. They just catch changes to the predictive error or epsilon when there is a drastic change. But let's talk about what are the problems, basically one by one.

@grok @adammcarter @__eknight__ @OpenAI @grok help me visualise this

@maxxfuu thats so cool, i have been trying to get into proper inference engineering too, would love to see your journey and follow along

Day 1/90 of Inference Engineering realistically, i started this journey 1 week ago. but i thought it be nice to post on here to share my journey and so that i can look back on this moment. here’s what i’ve done in the past week - i wrote a gpt3 like model from scratch in pytorch - applied kv cache, wowza - took some notes on the attention paper - read the paper on flash attention - currently building a benchmark tool based on the roofline model - revisiting chapter 6 of CSAPP

agreed, everytime a new research related to thinking, brain interpretation comes out, people keep talking about " llms are thinking like us " no they are not dummy. Its true complex reasoning causes Interpretability , models use their internal reasoning to reason better but -> that aint thinking.

@buildanythingso You bet

hackathon soon reply if you want to join

@alokbishoyi97 ex ml and system architect at NASA worked on Artemis 1 program, worked on kernels, algorithms & frontier architectures. prigoistic.me

We now have a community of 45 of some of the smartest Indian origin folks spanning across inference engg, post training and ai infra - including folks from OAI, deepmind, meta, MSR, sarvam, a bunch of neolabs and some fresh grads who are arguably 99.9 %ile of folks in terms of agency we are looking for even more high caliber folks who are working at frontier or neo labs to join us! Spread the word!