CausalFlopping

Happy New Year Everyone , let this year be successful for y’all , let’s welcome 2026. Failure Reflection for 2025: I didn’t post my learning’s and projects because I was scared of judgement.I was stuck in tutorial hell but I might be stuck again currently tutorial hell and more.

this will take you 5 mins to go through and is a solid starting point before touching the GPU

@VazeKshitij Congratulations 🙌🥳

Surreal and humbling. Thank you so, so much guys.

@neural_avb Congratulations 🙌🥳

woohoo 9k follows we finally broke out of 8k prison hell

@VazeKshitij How’s ur day boss🙌

I got 0 motivation to tweet ANYTHING today btw

LLM Knowledge Bases Something I'm finding very useful recently: using LLMs to build personal knowledge bases for various topics of research interest. In this way, a large fraction of my recent token throughput is going less into manipulating code, and more into manipulating knowledge (stored as markdown and images). The latest LLMs are quite good at it. So: Data ingest: I index source documents (articles, papers, repos, datasets, images, etc.) into a raw/ directory, then I use an LLM to incrementally "compile" a wiki, which is just a collection of .md files in a directory structure. The wiki includes summaries of all the data in raw/, backlinks, and then it categorizes data into concepts, writes articles for them, and links them all. To convert web articles into .md files I like to use the Obsidian Web Clipper extension, and then I also use a hotkey to download all the related images to local so that my LLM can easily reference them. IDE: I use Obsidian as the IDE "frontend" where I can view the raw data, the the compiled wiki, and the derived visualizations. Important to note that the LLM writes and maintains all of the data of the wiki, I rarely touch it directly. I've played with a few Obsidian plugins to render and view data in other ways (e.g. Marp for slides). Q&A: Where things get interesting is that once your wiki is big enough (e.g. mine on some recent research is ~100 articles and ~400K words), you can ask your LLM agent all kinds of complex questions against the wiki, and it will go off, research the answers, etc. I thought I had to reach for fancy RAG, but the LLM has been pretty good about auto-maintaining index files and brief summaries of all the documents and it reads all the important related data fairly easily at this ~small scale. Output: Instead of getting answers in text/terminal, I like to have it render markdown files for me, or slide shows (Marp format), or matplotlib images, all of which I then view again in Obsidian. You can imagine many other visual output formats depending on the query. Often, I end up "filing" the outputs back into the wiki to enhance it for further queries. So my own explorations and queries always "add up" in the knowledge base. Linting: I've run some LLM "health checks" over the wiki to e.g. find inconsistent data, impute missing data (with web searchers), find interesting connections for new article candidates, etc., to incrementally clean up the wiki and enhance its overall data integrity. The LLMs are quite good at suggesting further questions to ask and look into. Extra tools: I find myself developing additional tools to process the data, e.g. I vibe coded a small and naive search engine over the wiki, which I both use directly (in a web ui), but more often I want to hand it off to an LLM via CLI as a tool for larger queries. Further explorations: As the repo grows, the natural desire is to also think about synthetic data generation + finetuning to have your LLM "know" the data in its weights instead of just context windows. TLDR: raw data from a given number of sources is collected, then compiled by an LLM into a .md wiki, then operated on by various CLIs by the LLM to do Q&A and to incrementally enhance the wiki, and all of it viewable in Obsidian. You rarely ever write or edit the wiki manually, it's the domain of the LLM. I think there is room here for an incredible new product instead of a hacky collection of scripts.

Gonna start this and probably be consistent posting progress on what I do daily basis. 😭

@freshlimesofa That’s some good Japanese food 🙌🥹

Had some Japanese food today.

@srijatwt @MATSprogram Congratulations 🥳🙌🎉

got a mail I never thought I'd receive :) would love to connect with fellow @MATSprogram scholars, looking forward to this summer!

@wordsofteekay Congratulations 🙌🎉

After 12 years working as a software engineer, last month I decided to take a new adventure in my career and accepted a Machine Learning Engineer role. It's been a month since I've been working on data science and ML problems. The struggle of my 'knowledge gap' fires my curiosity and keeps me alive. It's been quite some time since I've felt this excited. It's been a fun and exciting month. I should write about that.

I made a Claude Code skill that turns any arxiv paper into working code. Every line traces back to the paper section it came from & any implementation detail the paper skips will be flagged, and not assumed. open sourcing it - github.com/PrathamLearnsT…

@neural_avb Ye it’s still better than LinkedIn no doubt🙌

@CausalFlops28 I had no firsthand idea what X/twitter was like coz I never used it until last year. I like the content here more than linkedin… esp in the tech side.

Spent 50 minutes arguing with likely-bots trying to clear their doubts about quantized local slms

@vinx_codes hey 🙌

Looking to connect with people in tech Whether you're into: Frontend • Backend • Full Stack DevOps • AI/ML • Data Science UI/UX • Freelancing • Startups IF YOU'RE INTO TECH... LET'S CONNECT

Probably i need to start writing here. 😀

@neural_avb I think it's time to get a video on edge models🙌🙌

Lessons from this post: > people don’t understand what edge models are for > people don’t understand LLM usecases outside “coding” > people don’t understand there is AI outside LLMs > people don’t understand shipping local narrow ai features > most people here are bots

@YearsProgress 1/4 of the year is done. Have you achieved 1/4 of the goals you set for this year?

2026 is 25% complete.

@freshlimesofa I will always remember you in ma heart ❤️

Would y'all still remember me if i stopped tweeting for 2 weeks ?

@SwapAgarwal Yep building an llm from scratch using the exported WhatsApp chat of mine and aim is to try to mimic the style of how I chat

Here's my April's goal: Build a Large Language Model (From Scratch) If you are a mid-level/senior dev and want to join me on this journey, comment below or DM me!

@VazeKshitij Yea fr I always focus on theoretical which also important but without implementation idt anyone can truly understand without implementation

@CausalFlops28 Facts

Here's the biggest difference between a college student and an experienced engineer - the ability to think and then implement things at scale. A college student only cares about making a project work. If it runs as expected, a pastry and a bottle of soft drink (or hard ones for some) are ordered, marks are secured, and you move on to the next semester. An experienced engineer does not have that privilege. They have to think about replicability, scale, the differentiators for products that are produced at scale, the harware that comes with it, the edge-vases that don't show up at a decive level, but show up at the mass produced scale, and a thousand more things. They are not only able to implement, but design and think about systems at scale, and to have a mind that can do that is a MASSIVE advantage when it comes to working at a company. And truth be told, this is something that can only be learnt via failures, bug-fixes, mistakes, and fuck-ups. You CANNOT learn to do this by sitting in a class. There's a saying in Marathi that loosely translates as - "I've survived a couple more rains than you" And by God, that applies so aptly to engineering.