Jane Pan retweetledi
Jane Pan
27 posts
Jane Pan
@JanePan_
CS PhD at @nyuniversity, @NSF GRFP, @Deepmind Fellowship, @SiebelScholars | @Princeton @Princeton_nlp '23 | @Columbia '21.
Katılım Şubat 2023
145 Takip Edilen440 Takipçiler
Jane Pan retweetledi
Can LLMs control their chains of thought (CoT)? If so, they could evade CoT monitors 🚨
We introduce the CoT Controllability eval suite to find out.
Our results leave us cautiously optimistic that today’s models struggle to obfuscate their CoT in ways that undermine monitorability.
In this thread, I explain additional findings that I find interesting
Joint work w @OpenAI
Tomek Korbak@tomekkorbak
We have a new eval to help keep chains of thought (CoT) monitorable: CoT Controllability. This tests whether LLMs can control their CoT, helping to evade CoT monitors. So far, the results leave us cautiously optimistic: today’s models struggle to obfuscate their reasoning in ways that undermine monitorability.
English
Excited to see NeuNeu out — learned scaling laws from open-source LM trajectories!
Michael Hu@michahu8
if you truly believe in the bitter lesson, then why hand design scaling laws? introducing: neural neural scaling laws (NeuNeu), a neural network - trained on open-source LM trajectories - that predicts LMs' future downstream task performance 🧵👇
English
Jane Pan retweetledi
Our work on LLM novelty as the frontier of original and high-quality output was accepted to #ICLR26! Come talk to us about how model scale, SFT, and RL affect this trade-off! See you in Brazil!🇧🇷h/t to my awesome collaborators @hhexiy @valeriechen_ @JanePan_ @jcyhc_ai
Vishakh Padmakumar@vishakh_pk
What does it mean for #LLM output to be novel? In work w/ @jcyhc_ai, @JanePan_, @valeriechen_, @hhexiy we argue it needs to be both original and high quality. While prompting tricks trade one for the other, better models (scaling/post-training) can shift the novelty frontier 🧵
English
Jane Pan retweetledi
🚨Prompt Curriculum Learning (PCL)
- Efficient LLM RL training algo!
- We investigate factors that affect convergence: bsz, # prompt, # gen, prompt selection
- We propose PCL: lightweight algo that *dynamically selects intermediate-difficulty prompts* using a learned value model
English
Bored of seeing pristine, perfect posters? Come see me at Hall X5, Board 105 at 6pm to witness my masterpiece, featuring bonus Sharpie scribbles and a QR code that betrayed me at the last moment 😤
Jane Pan@JanePan_
I'll be at ACL Vienna 🇦🇹 next week presenting this work! If you're around, come say hi on Monday (7/28) from 18:00–19:30 in Hall 4/5. Would love to chat about code model benchmarks 🧠, simulating user interactions 🤝, and human-centered NLP in general!
English
I'll be at ACL Vienna 🇦🇹 next week presenting this work! If you're around, come say hi on Monday (7/28) from 18:00–19:30 in Hall 4/5. Would love to chat about code model benchmarks 🧠, simulating user interactions 🤝, and human-centered NLP in general!
Jane Pan@JanePan_
When benchmarks talk, do LLMs listen? Our new paper shows that evaluating that code LLMs with interactive feedback significantly affects model performance compared to standard static benchmarks! Work w/ @RyanShar01, @jacob_pfau, @atalwalkar, @hhexiy, and @valeriechen_! [1/6]
English
Jane Pan retweetledi
What does it mean for #LLM output to be novel?
In work w/ @jcyhc_ai, @JanePan_, @valeriechen_, @hhexiy we argue it needs to be both original and high quality. While prompting tricks trade one for the other, better models (scaling/post-training) can shift the novelty frontier 🧵
English
Jane Pan retweetledi
We're excited to receive wide attention from the community—thank you for your support!
We release code, trained probes, and the generated CoT data👇
github.com/AngelaZZZ-611/…
We have labeled answer data on its way. Stay tuned!
Yulin Chen@YulinChen99
Reasoning models overthink, generating multiple answers during reasoning. Is it because they can’t tell which ones are right? No! We find while reasoning models encode strong correctness signals during chain-of-thought, they may not use them optimally. 🧵 below
English
Do reasoning models know when their answers are right?🤔
Really excited about this work led by Anqi and @YulinChen99. Check out this thread below!
Yulin Chen@YulinChen99
Reasoning models overthink, generating multiple answers during reasoning. Is it because they can’t tell which ones are right? No! We find while reasoning models encode strong correctness signals during chain-of-thought, they may not use them optimally. 🧵 below
English
Our work bridges the gap between existing static benchmarks and real-world usage, and we hope to inspire future work on scalable methods for evaluating models in a collaborative setting.
Read our preprint at arxiv.org/abs/2502.18413!
[6/6]
English
We also investigate how much a code model adjusts its solution in response to feedback. Weaker models tend to make many surface-level changes that do not greatly change code behavior; stronger models may make relatively small edits that highly affect code behavior.
[5/6]
English
When benchmarks talk, do LLMs listen?
Our new paper shows that evaluating that code LLMs with interactive feedback significantly affects model performance compared to standard static benchmarks!
Work w/ @RyanShar01, @jacob_pfau, @atalwalkar, @hhexiy, and @valeriechen_!
[1/6]
English
Jane Pan retweetledi
@tallinzen @akyurekekin @yoavartzi @NeelNanda5 I really like the paper from Jane Pan (w @danqi_chen) abt this: arxiv.org/abs/2305.09731. ICL in big models is clearly a mix of task recognition and "real learning" (you're not learning to translate from 3 examples, but you're not getting an arbitrary label mapping from the prior)
English
Our pre-print is available on arXiv here:
arxiv.org/abs/2407.04549
[7/7]
English
We follow the canonical definition of reward hacking, observing a divergence between the ground-truth reward (human expert judgment) and its proxy (an LLM judge following the same scoring criteria as the humans).
Our results complement recent work on output degradation via iterative refinement when measured with secondary objectives (arxiv.org/abs/2402.06627) or with reference-based metrics (arxiv.org/abs/2402.11436).
[6/7]
English
Do LLMs exploit imperfect proxies of human preference in context? Yes!
In fact, they do it so severely that iterative refinement can make outputs worse when judged by actual humans. In other words, reward hacking can occur even without gradient updates!
w/ @hhexiy, @sleepinyourhat, @ihsgnef
[1/7]
English