nullptr

@Madbonze16 @prajdabre @SarvamAI penalizing long reasoning traces must be common tho irrespective of compute constraints?

@resetptr @prajdabre @SarvamAI This might be down to maximum reasoning tokens they specified during training, which might be because they didn't have a lot of compute to allow it a lot of reasoning tokens

ran some quick weekend experiments on @SarvamAI's 105B model on a subset of the IndicMMLU-Pro dataset Sarvam's model is really good at reasoning efficiency. uses ~2.5x less tokens to reach ~same accuracy

yeah that was my first guess after reading their blog post but the reasoning is less verbose which efficient tokenization doesn't really explain for eg, Sarvam's vs GLM's reasoning excerpt (which continues on beyond this screenshot) for a question on cramer's rule (correction - i should've said chars not tokens)

@resetptr @SarvamAI Fewer* Its because of the tokenizer.

@garybasin @zack_overflow technically you could tho, right? gumbel softmax etc, although maybe not as well or efficiently

@zack_overflow Just can’t train it with gradients

Feels like the coolest part of this research is not "we put computer in LLM" but making attention scale logarithmically?

@zack_overflow i wonder what the intermediate steps of the hard coded transformers look like. will certain paths "look wrong" in the gradients that you could then learn and then pause in between?

@ChristosTzamos super cool! any plans to share the weights?

1/4 LLMs solve research grade math problems but struggle with basic calculations. We bridge this gap by turning them to computers. We built a computer INSIDE a transformer that can run programs for millions of steps in seconds solving even the hardest Sudokus with 100% accuracy

@silicognition include the you're reading papers in a subfolder. use md files to review and discuss ideas, track references, etc

@silicognition claude code has been really helpful for this keep a folder with your venv, claude.md etc set up and ask cc to create new jupyter notebooks based on ideas you want to explore

people who are doing research, how do you go from reading papers & ideation to getting down to something concrete which can be actually done? i have ideas, read a lot of papers but from a fuzzy cloud of insights & inspirations, i would like to get to the finish line help pls!

@tenobrus corollary is even if you already have set up strict mode / structured outputs it wouldn't drop performance / diversity that much i find tool definitions allow me to separate / organize descriptions, examples, etc better

fyi that neither codex nor claude code enable strict mode / structured outputs in their agent loops. u can check the source or intercept requests. they just rely on the models to make valid tool calls without grammar enforcement. so if they're not doing it, why are you?

sidenote: sarvam's APIs are kinda flaky, repeated 504 gateway errors which required multiple retries. i'm sure this'll get better with time tho. great job!

all 4 are within ~2% accuracy of each other reasoning is in English tho even when prompted in Indic languages which was interesting. will spend some more time exploring why sarvam’s so much more token-efficient (it's prolly, ((most definitely)) data)

@yule_gan @yacinelearning thanks for sharing! Shao et al immediately sprung up in my mind when i read your post

@resetptr @yacinelearning Actually our findings help explain the results in the paper you mentioned.

I’m sooooo going to cover this guys this is borderline blasphemous

@Mihonarium @NeelNanda5 @OwainEvans_UK it is, but having it formalized and properly defined helps develop a lexicon to discuss/research it

@NeelNanda5 @OwainEvans_UK I’m so confused about what is the development here. Isn’t that just obviously what LLMs do? Like, the default way they think when without chain of thought?

Out of context reasoning is one of the most fascinating developments in the science of how LLMs work. This primer by @OwainEvans_UK, one of the main discoverers of the phenomena, is a great introduction

@infoxiao @kchonyc @karpathy i once had o3 produce consistently bad results until i realized it was because i was using the plural of a word instead of singular

@kchonyc @karpathy i once measured if using 'i' or 'you' made a difference for gemini. it did not. i was disappointed.

thanks to @karpathy , now i have cracked the mystery why my agent doesn't follow my instruction closely enough.

@adocomplete is this also supported on claude code?

28 Days of Claude API - Day 24 - PDF Support Send a PDF to the API. Get answers about the content. Every page is processed as both text & image so nothing gets missed. Three ways to use: URL, base64, or Files API file_id. No preprocessing. No custom parsers. Just send & ask.

@yacinelearning had to dig deep for this one x.com/StellaLisy/sta…

@yacinelearning reminds me of arxiv.org/pdf/2506.10947

#multi-context-window-workflows" target="_blank" rel="nofollow noopener">platform.claude.com/docs/en/build-…

TIL anthropic has specific suggestions for long-running tasks. starting new instead of compacting convos is a suggestion! need to try it out some more

welp

whatever happened to moltbook?