Oscar Davis

126 posts

Oscar Davis banner
Oscar Davis

Oscar Davis

@osclsd

Research Intern @Apple MLR; PhD ML @UniofOxford; generative modelling; previously at @MSFTResearch, @EPFL, @imperialcollege

Paris, France Katılım Mayıs 2024
350 Takip Edilen755 Takipçiler
Sabitlenmiş Tweet
Oscar Davis
Oscar Davis@osclsd·
We were all wondering whether Categorical Flow Maps (CFMs) could scale... 🤔 I couldn't help trying it out... So we scaled CFMs to 1.7B parameters over 2.1T tokens 🚀🔥 Short summary 🧵⬇️
English
4
32
127
15.1K
Dr. Richard
Dr. Richard@richnanophd·
@osclsd 1.7B parameters is no joke. Would love to see how convergence behaves at this scale. Thanks for pushing boundaries and sharing! 🔥👏
English
1
0
1
40
Oscar Davis
Oscar Davis@osclsd·
We were all wondering whether Categorical Flow Maps (CFMs) could scale... 🤔 I couldn't help trying it out... So we scaled CFMs to 1.7B parameters over 2.1T tokens 🚀🔥 Short summary 🧵⬇️
English
4
32
127
15.1K
Oscar Davis retweetledi
Tyler Farghly
Tyler Farghly@tylerfarghly·
[📄preprint] Diffusion models 🤝 MCMC ! Diffusion model samplers are biased due to discretisation 💡The fix: Metropolis-type adjustment on corrector steps ❗️Challenge: no access to the density ratio, only the score 🔑Insight: the score (and some maths) is all you need... [1/3]
Tyler Farghly tweet media
English
5
50
345
18K
Oscar Davis retweetledi
Pedro
Pedro@pmpcurvo·
Guide with examples, not rewards 🐘 Controlling what a pretrained generative model produces is still mostly a choice between three slow options: fine-tune it, attach a reward network, or search at inference. We found flow matching allows a fourth, and it costs almost nothing. In deterministic interpolants, the velocity of the flow is determined by where the trajectory is headed: the endpoint mean. Shift that mean, and the entire flow shifts with it. This turns control into a matter of reference. Change the examples that define the endpoint, and you change the direction the model follows. The examples need not be perfect. They only need to point the flow toward the attribute you want. Color, identity, style, and structure, all controllable through examples. 🧵👇
GIF
English
6
29
168
33.1K
Oscar Davis retweetledi
Floor Eijkelboom
Floor Eijkelboom@FEijkelboom·
Very excited about our work on finding the right drifting direction 🐎 We tackle a core open question in drifting: when does “no drift left” mean the model really matched the data? Kernel-gradient drifting is the answer (with natural extensions to manifolds + discrete data)!
Maria Esteban@Maria__Esteban

🏎️Drift in the right direction🏎️ Introducing kernel-gradient drifting models: a reformulation of drifting models where the kernel itself defines the direction of motion through its gradient. 📜Paper: arxiv.org/pdf/2605.10727 💾Notebook: tinyurl.com/mv2jhuky

English
0
12
81
10K
Sam Acquaviva
Sam Acquaviva@Sam_Acqua·
Great work from Oscar on scaling up flow models!
Oscar Davis@osclsd

🚨 Before concluding: As noted by @Sam_Acqua and many others, we all ought to be very skeptical of Gen PPL as a metric, especially in isolation. ❌ It is actually a bit crazy that we have been using it for so long. Hence, the additional metrics, and the presence of several qualitative samples in the appendix. Please have a look yourselves to get a better understanding of the sample quality! 🔍 There's comparisons across SD/non-SD, number of NFEs, and others.

English
1
0
2
571

Keşfet

@richnanophd @pmpcurvo @eb1aexperts @FEijkelboom @jdeschena @LiDavid2002 @Sam_Acqua @LucaAmb