Nan Jiang retweetledi
Nan Jiang
31 posts
Nan Jiang
@nanjiangwill
building fun stuff @modal
San Francisco, CA Katılım Ocak 2018
375 Takip Edilen98 Takipçiler
Nan Jiang retweetledi
This paper by Ivan Lee (@ivn1e) & @BergKirkpatrick was great! Best thing I’ve seen at #COLM2025 so far!
Readability ≠ Learnability: Rethinking the Role of Simplicity in Training Small Language Models
openreview.net/forum?id=AFMGb…
English
Nan Jiang retweetledi
📢 (1/16) Introducing PaTH 🛣️ — a RoPE-free contextualized position encoding scheme, built for stronger state tracking, better extrapolation, and hardware-efficient training. PaTH outperforms RoPE across short and long language modeling benchmarks
arxiv.org/abs/2505.16381
English
amazing Jason, amazing Nexad, please check this out!
Jason Hu@onjas_6
Let’s be real—ads have annoyed me for years. Pop-ups, spam, etc… while the world is moving towards AGI, the ad world felt stuck in the past. So I decided to flip the script. Today, I’m proud to share: Nexad has raised a $6M seed round, led by @a16z SR04, @Prosus_Ventures , @p72vc , Carya, and more. 🧵
English
Nan Jiang retweetledi
Coding agents can debug their own outputs, but what if none of the fixes are correct? We overcome sparse rewards by making them continuous📈 Instead of having binary execution rewards, we introduce a learned verifier to measure how close the current solution is to a correct one📏
GIF
English
Nan Jiang retweetledi
I teach a class where students code up an ML library from scratch in Python. Wenting showed me that a Claude Agent (with interactive unit test feedback and the spec) could solve it 100%. We thought it would be fun to scale this idea to every Python library in the world.
Wenting Zhao@wzhao_nlp
Introducing the commit0 interactive environment for coding agents. Challenge: generate Python libraries from scratch. Commit0 is designed with interactivity, dependencies, and specifications as first-class considerations. We include a benchmark with 50+ challenging libraries.
English
So... can agents now build a package from scratch? Test them on Commit0!
This is an amazing and fun project this summer! Huge thanks to Wenting and to everyone in the lab for their support and guidance! 🚀👏
Wenting Zhao@wzhao_nlp
Introducing the commit0 interactive environment for coding agents. Challenge: generate Python libraries from scratch. Commit0 is designed with interactivity, dependencies, and specifications as first-class considerations. We include a benchmark with 50+ challenging libraries.
English
Nan Jiang retweetledi
🚀 Introducing RouterBench, the first comprehensive benchmark for evaluating LLM routers! 🎉
A collaboration between @withmartian and Prof. @KurtKeutzer at @UCBerkeley, we've created the first holistic framework to assess LLM routing systems. 🧵1/8
To read more: blog.withmartian.com/post/router-be…
English
@DimitrisPapail @Krafton_inc @SeoulNatlUni @UMich @UWMadison interesting work! we also look into this problem across other architectures!
x.com/nanjiangwill/s…
interesting to explore other alternative architectures
Nan Jiang@nanjiangwill
❓Are attention-based models needed for In-Context Learning(ICL)? 🤔Can emerging architectures perform ICL? 🎉Check out our #ICLR2024 paper "Exploring the Relationship Between Model Architecture and In-Context Learning Ability" 🎉 #LLM Paper: arxiv.org/abs/2310.08049 🧵[1/9]
English
arxiv drop tonite
"Can Mamba Learn How to Learn?: A Comparative Study on In-Context Learning Tasks"
with all-star set of collaborations from @Krafton_inc @SeoulNatlUni @UMich and @UWMadison
English
@akyurekekin Yes!! also interesting to see what might happen when adding syntax information
English
@nanjiangwill you are asking if I train a model with ICL with a class of languages (we did w/ regular languages in the paper), how well they generalize other languages? This is a great follow-up Q. that we mentioned in the intro.
English
A really interesting corollary that I realized:
copying ("x|x") = ICLL with singleton languages
Ekin Akyürek@akyurekekin
@EranMalach @brandfonbrener Ah, I found the connection between the copying task ("x|x") and in-context language learning (ICLL): copying is a subset of ICLL w/ regular languages such that the languages consist of a single element and each instance have 2 examples. copying = ICLL with singleton languages
English
@akyurekekin ah sorry for not making it clear, mainly want to discuss OOD cases. interesting to think about with ICLL.
what if we try to get rot-2("f|?") with rot-2("x|z"), rot-2("a|c") in context, but the models are trained on rot-1 examples?
English
@nanjiangwill could you open "complicated"? when I say "x|x" I meant x as a string/sequence token not a single character, the task from the repeat after me paper
English
Nan Jiang retweetledi
1/ Today in Science, we train a neural net from scratch through the eyes and ears of one child. The model learns to map words to visual referents, showing how grounded language learning from just one child's perspective is possible with today's AI tools. science.org/doi/10.1126/sc…
English
A sick study on how model architecture (i.e. attention alternatives) influence ICL ability
Check it out! (p.s. @nanjiangwill is on the PhD market!! 🥳)
Nan Jiang@nanjiangwill
❓Are attention-based models needed for In-Context Learning(ICL)? 🤔Can emerging architectures perform ICL? 🎉Check out our #ICLR2024 paper "Exploring the Relationship Between Model Architecture and In-Context Learning Ability" 🎉 #LLM Paper: arxiv.org/abs/2310.08049 🧵[1/9]
English
We're excited to contribute to the exploration of alternative architectures and emergent capabilities!!
🎉🎉🎉 Huge congrats and many thanks to Ivan Lee and Prof. Taylor Berg-Kirkpatrick @BergKirkpatrick
🧵[9/9]
English
Section 3.1: A Simple Few-Shot Natural Language Task
1) Stronger models tend to have worse performance when not relying on semantics
2) Most architectures fail in the flipped setting while Hyena is the best model compared to other models that is not pre-trained.
🧵[8/9]
English
❓Are attention-based models needed for In-Context Learning(ICL)?
🤔Can emerging architectures perform ICL?
🎉Check out our #ICLR2024 paper "Exploring the Relationship Between Model Architecture and In-Context Learning Ability" 🎉 #LLM
Paper: arxiv.org/abs/2310.08049
🧵[1/9]
English