Yao retweetledi
Yao
13 posts
Yao
@yaozhaoai
Researcher/engineer working on LLMs and agents. ex Google Deepmind.
California, USA Katılım Aralık 2019
386 Takip Edilen254 Takipçiler
We're building Compound, an AI Analyst built for finance. It works like a team of analysts reporting to you — researching, delivering insights, and producing work. Start compounding your impact.
Compound AI@getcompoundai
Meet Compound - the world’s first AI Analyst for finance you can trust. AI for spreadsheets and financial analysis is finally here, but most tools are too brittle for real use cases. Compound is different - built for scale, accuracy, and auditability - so you can 10X your output. - Upload unlimited number of files to analyze - Kick off multiple AI Analysts at the same time - Audit and edit the work output in the browser Comment for access to the beta. For more on what Compound can do, see below 🧵
English
Yao retweetledi
One of the top machine learning conferences #ICLR2025 is this week. But there’s 3000+ accepted papers, which is a lot to sift through. Use RadPod to chat with them all and quickly hone in on your interests.
Examples queries:
“find papers with more than one OpenAI-affiliated author”
“find papers that propose alternatives to Transformer architecture in LLM”
“give an overview of all spotlight or oral papers with Yann Lecun as author”
You can even get a link to the OpenReview reviews easily.
English
Yao retweetledi
Recently a huge new batch of files on the JFK Assassination was released by the National Archives as a result of a presidential executive order. A whopping ~80,000 pages of scanned PDFs -- available but not accessible. AI to the rescue!
Except none of the AI apps can handle this type and amount of context ... until now.
We built RadPod AI to enable highly-accurate, deep research on your (possibly huge) data.
English
Yao retweetledi
Transformer decoder with MoE and efficient attention has been available at tensor2tensor library since 2017
A paper that trained Transformer decoder with MoE, efficient attention and up to 11k context length was released in September 2017 (arxiv.org/abs/1801.10198).
English
@Francis_YAO_ @denny_zhou So true, nearly forgot about it, used to spent so much time on it.
English
If letting you name one ml technique that was considered to be critical in building AGI but now you think it is irrelevant or at least not important, what is on top of your mind?
English
@GriffinAdams16 @peterjliu Thanks for sharing your paper, super interesting. Same scrutiny on HF data composition is much needed too!
English
@peterjliu @yaozhaoai Very cool and nice follow up to first SLiC!
There’s also unexplored upside in how to construct these offline candidate sets. We show it has a large impact on performance in a new ACL preprint arxiv.org/abs/2305.07615.
This line of work can be scaled to more diverse methods.
English
Here is our “slick” RLHF-alternative without RL: arxiv.org/abs/2305.10425 (SLiC-HF)
TL;DR: Works as well as RLHF, but a lot simpler.
About as easy and efficient as fine-tuning. Much better than simply fine-tuning on good examples.
From great collaborators: @yaozhaoai, @rishabh_joshi4, Tianqi Liu, @khalman_m, @Mohamma78108419, @peterjliu.
Peter J. Liu@peterjliu
The true star of RLHF is F=feedback. You may not need RL and you may not need humans.
English
Key of learning from feedback is a different signal than supervised fine-tuning: distinguish better/worse seqs vs generate plausible seqs
Evidence: contrastive learning and RL learn from feedback equally well, both much better than fine-tune on only positive feedback.
Peter J. Liu@peterjliu
Here is our “slick” RLHF-alternative without RL: arxiv.org/abs/2305.10425 (SLiC-HF) TL;DR: Works as well as RLHF, but a lot simpler. About as easy and efficient as fine-tuning. Much better than simply fine-tuning on good examples. From great collaborators: @yaozhaoai, @rishabh_joshi4, Tianqi Liu, @khalman_m, @Mohamma78108419, @peterjliu.
English
Amazing how much progress in AI is due to two chain rules: one from calculus, the other from probability.
English
Yao retweetledi
We are hiring for a full-time researcher/engineer in the Brain (Google Research) team who will focus on text generation research and its applications. A wide variety of backgrounds and experiences will be considered. DM if you're interested or have leads.
English
Yao retweetledi
DistilBERT by @SanhEstPasMoi is one of the most popular models on the @huggingface model hub, but there wasn’t a clear equivalent for Seq2Seq models. Now there is! I'm happy to introduce our paper on “Pre-trained Summarization Distillation”: w @srush_nlp arxiv.org/abs/2010.13002
English
Yao retweetledi
Excited to release PEGASUS in @huggingface transformers: 12 new SOTA summarization models: huggingface.co/models?filter=…
from Google Brain (@GoogleAI) intern @JingqingZX , and colleagues @yaozhaoai, ,Mohammad Saleh, and Peter Liu (@peterjliu). 👇
English
Yao retweetledi
New SOTA results for abstractive summarization just posted to arxiv.org/abs/1912.08777! We have a new way to pre-train for summarization, and evaluated our PEGASUS model on 12 diverse downstream summarization tasks, achieving SOTA on all, in some cases by a significant margin.
English