Sabitlenmiş Tweet
Piyush Tiwary
353 posts
Piyush Tiwary
@backpropogator
Ph.D. at @iiscbangalore, SR at @GoogleDeepMind | Previously @AdobeResearch @IITPAT Tweets may seem random individually but show patterns at scale.
Bengaluru, India Katılım Mayıs 2015
1.1K Takip Edilen316 Takipçiler
New VS Code update is so damn buggy.
Been trying to connect to Remote SSH servers since morning and it just keeps throwing random errors and breaking constantly.
@code
English
Piyush Tiwary retweetledi
Not every inspiring sporting story ends with a trophy. Over the last few days, Lakshya Sen has shown India what courage, resilience and belief truly look like. His run to another All England final, through extraordinary wins and immense physical pain, has been about far more than a result. He has reminded young India that greatness lies not only in winning, but in the honesty of effort, the dignity of the fight and the strength to keep believing. I am Proud of you, @lakshya_sen . Very, very proud.
English
@imrahulmaddy Many people read @thinkymachines blog on On-policy distillation. Self distillation is a very natural and low-hanging idea after that?
English
If these are just the self-distillation papers that hit twitter, i can imagine there are more undiscovered ones. How did so many researchers discover the same idea at the same time? 🤔
Axel Darmouni@ADarmouni
Another work that confirms Self-Distillation’s effectiveness, this time from @AIatMeta ‘s MSL, by Zhao et al In « Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models », the authors perform Self-Distill similarly to the other SD setups: —> Student gets to output —> Teacher is same model, but outputs with ground truth + CoT —> Model trained on KL divergence between Student and Teacher Worth noting: —> Setup applied to maths here —> Teacher = initial student, not using EMA but thus not trained —> Trained on full vocabulary or sampled tokens only, with just like the other studies better results on full vocabulary Findings: —> Performs similarly to GRPO on similar setup of params —> Much, much more token efficient than GRPO, to an actually absurd degree The fact there are so many works colliding on the strength of SD means we’re only at the beginning =)
English
Piyush Tiwary retweetledi
To be a successful AI researcher it helps to be good at math but not too good.
English
Piyush Tiwary retweetledi
A quick post about something that keeps happening to me very often (and I know this is ubiquitous, and happens to essentially everyone pushing the state of the art):
1. Paper A solves problem X.
2. Despite numerous submissions, paper A is not published because the reviewers do not understand its significance.
3. Years later, paper B claims to have solved problem X. It gets published. It does not cite paper A (probably does not even know about its existence).
English
@aishwarya_2x21 Looks like notes from @hkproj's tutorial?
English
@prajdabre This joke is too googley to be shared outside 😆
English
Piyush Tiwary retweetledi
Almost tearing up seeing this!
Government of India is working towards standardizing the UX across its services and has even a Figma design system out.
English
Piyush Tiwary retweetledi
An excuse makes today easy but tomorrow hard.
Discipline makes today hard but tomorrow easy.
English
Poster Session details:
Time: 4:30PM PDT, Wednesday 15/07 (tomorrow!)
Poster Location: West Exhibition Hall B2-B3 #W-319
Come say hi!
English
