Maxwill Lin

@lateinteraction @cohenrap Yeah, symbolic recursion is such elegant way to put the whole idea! My point is the key diff "comapred to prior methods" (e.g. Claude Code subagents) is the "symbolic" part, as recursion is already quite commonly used.

@tensorfi @cohenrap Yeah almost! You forgot the importance of symbolic recursion. In simple terms, the model must write code that calls LLMs. Calling LLMs as tools doesn’t cut it. (This can be folded under post-training/instruction but it’s scaffoldy in the simple sense that models need a self-API).

RLMs are not sub-agents or the ability to iteratively retrieve context. I know because I trained multi-hop models for reasoning & retrieval in 2020, including compaction.* RLMs are the simplest/purest scaffold that understands its own prompts via recursion, not via attention. They support an extremely simple but unusual claim: Models need to be able to access their own conversations with the user and their own horizon symbolically and recursively. The model should be only allowed to understand this long context by *writing code* that launches LLMs, and composing these into the final response. Note that the number of LLM launches can be linear or even bigger in the context size, not a small constant number of sub-tasks. This sounds big until you remember that attention is already quadratic. I'll have to confess that I always found (and still find) the conventional pattern of "sub-agents" rather boring. This is the superficially related structure where the model is given a special tool it can invoke by writing out prompts for and receiving the output. Verbalizing specific individual sub-calls as tool calls token-by-token hides the internal reasoning from the main context, which is an OK outcome for sub-task delegation. But it's a completely unrelated pattern to teaching models to understand their own context/horizon recursively. Sorry I'm a bit of a pedant for understanding concepts precisely, but this seemed needed. *The title is quite literally "Robust Multi-Hop Reasoning at Scale via Condensed Retrieval", arXived on Jan 2nd 2021. It could work for many steps, retrieve text from a massive corpus, compact/condense its own context, and iterate further.

@lateinteraction @cohenrap My take is 1.2. treat everything with unbound size as a variable instead of expanding it by default. 3. models need to be instructed/post-trained for this paradigm. And I believe these are the ONLY foundamental diffs from Claude Code subagents x.com/tensorfi/statu…

OK then let me offer more useful thoughts real quick :D An agent that has an LLM tool is one of the baselines in the paper. It's better than some other baselines, but not that good actually. It has three major problems that together mean that the root LLM cannot scalably use it to actually understand the context. 1) First, the context it's given is not symbolic. It's given an actual verbalized prompt as tokens. So you've got to externalize the user prompts. 2) Second, the root LLM calls it by verbalizing sub-calls. The root LLM has to "copy" stuff token by token, which means it can't launch Ω(N) sub-calls because it has to produce them one at a time. So you've got to write code that can contain loops or recursive helper functions or whatever else, and those loops should make the sub-calls. 3) Third, the difference in the "policy"/behavior. When the root model is given an LLM as a tool, it's simply trained/instructed to use it for task delegation. Task delegation is great but that only delegates nuggets of work, not the actual core processing of the input. Think the difference between "go think about this 1 small problem" versus "go understand every chunk of the input". BTW it might actually bring more conceptual clarity to play with dspy.RLM because the way it fits with Signatures makes it much more obvious how it's special.

@a1zhang And nothing stops Claude Code to do the same if instructed to use file system to store I/O, but definitely better to be post-trained to adapt to this paradigm. text + tools -> files + terminal scripts ~= var + REPL

RLM (Recursive Language Models) by @a1zhang et al. is impressive. My take is, compared to subagents, the ONLY core diff is simple: default to conditional disclosure w/ constant-sized metadata for length-unbound I/O Or in 3 words: text → var

Decided to join in on the hackathon fun at ETH NYC! Creating an ML-empowered AMM on the Vanna Blockchain @0xVannaLabs to quote dynamic spreads to reduce LVR in AMMs. Protect. Liquidity. Providers. @billionxdev @tensorfi

@cryptodavidw From but not in🤣

any ZK peeps in taiwan BTW?

@wythern @cryptodavidw Lasso and Jolt

@cryptodavidw because of folding??

I feel like r1cs is cool again

For more details about zkAlpha, check out the presentation, and feel free to DM! docs.google.com/presentation/d…

I would also like to express my sincere appreciation to @Antalpha_Labs for hosting me and allowing me to meet so many interesting individuals during my time in Paris. The experience was nothing short of amazing!

I am excited to share that my team zkAlpha at ETHGlobal Paris won 7 prizes and ~9k USD in total (including @Filecoin grand prize and @1inch best use of smart contracts) out of 1400+ attendees and 321 projects.

Meet OLA in Paris! @ola_zkzkvm

#ePrint A Vulnerability in Implementations of SHA-3, SHAKE, EdDSA, and Other NIST-Approved Algorithm: N Mouha, C Celi ia.cr/2023/331

Teaching GPT-3 about KZG commitments. I crashed it at the end (as probably most human interlocutors would do long before), but the ability to take in new instructions/information on the spot is absolutely astonishing! The age of AI starts now. No more winters I would predict.

@hyperbola_cc Why

又創了最晚下班的記錄🥲🥲

Must watch video lecture on the foundations of BFT and Blockchains on a simple variant of Tendermint! @Tim_Roughgarden takes one of the most complicated topics in distributed computing and breaks it down to simple components with both clear intuition and strong rigor

I have the same discovery during my research on the protocol. An additional fact is that since @StargateFinance uses the "defaultConfig" that can be set by the owner address (a @gnosisSafe multisig proxy).

@ryan_levick congrats

Super excited to be joining the Rust Foundation board! 🎉 I look forward to working hard to help members of the Rust project do what they do best! 🦀

New Narwhal-based project now available! Bullshark, a partially-sync consensus embedded into Narwhal. Similar commit rule and resilience to (crash-)faults to Tusk, but 33% lower latency and no DKG/coin needed. sonnino.com/papers/bullsha… with @LefKok, A. Spiegleman, N. Giridharan