Crick Wu

Our team at Google DeepMind has a full-time Research Scientist position available at our Mountain View site. Minimum qualification: PhD in ML/NLP. Please email me with: your CV and Google Scholar link; a brief description of the impactful work you have done; and what you aim to achieve in the next years.

🤔️Why bother designing prompts to enable LLMs to self-improve? The preference data contains information on self-improvement. Thus, models can learn to self-improve implicitly! 📖Paper: arxiv.org/abs/2310.00898

MoE Meets Instruction Tuning: A Winning Combination for Large Language Models [1/3] arxiv.org/pdf/2305.14705 @shengs1123 @zhouyanqi30 @barret_zoph @LiamFedus @_jasonwei @hwchung27 @ShayneRedford @xinyun_chen_ @tuvuumass @CrickWu @albertwebson @zyzzhaoyuzhe @denny_zhou

How do you turn a language model into a chatbot without any user interactions? We introduce LIMA: a LLaMa-based model fine-tuned on only 1,000 curated prompts and responses, which produces shockingly good responses. * No user data * No mode distillation * No RLHF

Excited about the simple and efficient technique developed from the team!

This work is a result of wonderful collaboration with @JunwenBai, @sidbrahma, Joshua Ainslie, @kentonctlee, @zhouyanqi30, Nan Du, @zyzzhaoyuzhe, @CrickWu, @boli_xyz, @Yukuwu, @mchang21. We believe CoDA and CoLT5 highlight the potential of conditional computation for efficiency.

Tips for ensuring reliable #blockchain #infrastructures: Testing infrastructures is important and different from testing application software. Here is how we do it at @sentioxyz (full article linked below by @TYSentio and @qiaokan) 🧵1/6

🧮 I finally spent some time learning what exactly Neural Tangent Kernel (NTK) is and went through some mathematical proof. Hopefully after reading this, you will not feel all the math behind NTK is that scaring, but rather, quite intuitive. lilianweng.github.io/posts/2022-09-…