Dhruvesh Patel ✈️ ICML 2026

117 posts

Dhruvesh Patel ✈️ ICML 2026

@_dhruveshp

Ph.D. final year @umasscs | IBM Research | Meta | Abridge | Building non-autoregressive LLMs with math, engineering, and stubbornness 🧑‍🔬

Amherst, MA Katılım Şubat 2012

561 Takip Edilen133 Takipçiler

Sabitlenmiş Tweet

Dhruvesh Patel ✈️ ICML 2026@_dhruveshp·8 Tem

@brozonoyer and I will be presenting LoFlexMDM today at the #ICML poster session! 🗓️ Wed, Jul 8, 2026 • 5:00–6:45 PM KST 📍 Hall A • Poster #2805 We're cooking up something interesting for insertion based text dLLMs too! Come chat with us to find out more.

Dhruvesh Patel ✈️ ICML 2026@_dhruveshp

Variable-length masked diffusion models (FlexMDM and friends) generate by inserting mask tokens into any gap and unmasking them. But the insertion/unmasking schedule is fixed and data-independent. So the model has to learn to produce every sequence in every possible order. For structured data that's a huge waste of capacity. How do you learn data-dependent insertion and unmasking orders without breaking tractable training? We propose LoFlexMDM, which does exactly that. 🧵👇

English

249

Dhruvesh Patel ✈️ ICML 2026@_dhruveshp·10 Tem

We will be presenting our poster today at 4-5pm at the #SPIGM workshop at #ICML2026. If you are working on diffusion for text, either using discrete or continuous space approaches, you will find our results interested. Come chat with us. With @rozonoyer96703 and Jacopo Minniti.

Tim G. J. Rudner@timrudner

What if diffusion models could think ahead instead of being greedy at every step?🤔 We introduce: Learned Relay Representations for Forward-Thinking Discrete Diffusion Models

English

1.1K

Dhruvesh Patel ✈️ ICML 2026@_dhruveshp·1 Tem

If you'll be at #ICML2026 in Seoul 🇰🇷, come say hi in person. We will be at "Bridging Research and Open Source," a social for the ML open-source community. 📅 Wed July 8, 19:00-21:00 📍 COEX center, Seoul, rooms E1-E4

English

Dhruvesh Patel ✈️ ICML 2026@_dhruveshp·1 Tem

xLM is still alpha, so we'd genuinely like your bug reports and PRs. We also accept model and task contributions. Checkout github.com/dhruvdcoder/xl… Built at IESL, UMass Amherst with @brozonoyer, Durga Prasad Maram, and Sai Sreenivas Chintha.

English

Dhruvesh Patel ✈️ ICML 2026@_dhruveshp·1 Tem

Every non-autoregressive LM we picked up came with its own training script, dataloader, and eval code. so when a metric moved we couldn't tell if it was the new idea or the plumbing. So we built xLM: one CLI and one harness for training, eval, and generation. code: github.com/dhruvdcoder/xl… pypi: pypi.org/project/xlm-co… docs: dhruveshp.com/xlm-core/dev demo paper: arxiv.org/abs/2512.17065 📍 Come say hi! We will be at "Bridging Research and Open Source," social at #ICML2026. COEX center, Seoul, rooms E1-E4 | 📅 Wed July 8, 19:00-21:00 🧵👇

English

862

Dhruvesh Patel ✈️ ICML 2026@_dhruveshp·26 Haz

@HuLillian39250 Great work! Thanks for sharing. Do you also evaluate on conditional generation tasks?

English

Keya Hu@HuLillian39250·26 Haz

On unconditional OpenWebText generation, ELF+PD achieves the lowest generative perplexity among distilled discrete and continuous diffusion LM baselines, enabling strong few-step and even one-step generation.

English

481

Keya Hu@HuLillian39250·26 Haz

🧚Progressive Distillation of ELF🧚 We pushed ELF, a continuous diffusion language model, even further toward few-step generation ⚡️ ELF already generates 1,024-token sequences in just 8–32 steps without distillation. Now, with progressive distillation, ELF+PD scales all the way down to few-step and even one-step generation! Blog: linlu-qiu.github.io/assets/html/el… 🧵

English

117

23.1K

Dhruvesh Patel ✈️ ICML 2026 retweetledi

Haw-Shiuan Chang@Haw_Shiuan·22 Haz

Where does the data flywheel⚙️♻️ of LLM service providers come from? 🚨Our latest paper shows that it could come from your mouse🖱️ and eyes👀! With Jeffrey Gomez, @mehulpatwari_ , Aryan Sajith, @HamedZamani [1/N]🧵

English

2.7K

Dhruvesh Patel ✈️ ICML 2026@_dhruveshp·24 Haz

@MILIJOULE Haha. Thank you!

English

Amit@MILIJOULE·23 Haz

@_dhruveshp This looks amazing. keep Rocking Package

English

Dhruvesh Patel ✈️ ICML 2026@_dhruveshp·23 Haz

English

2.5K

Dhruvesh Patel ✈️ ICML 2026@_dhruveshp·23 Haz

This is a joint work with @brozonoyer, Tahira Naseem, Gaurav Pandey, @RamonAstudill12 , and @andrewmccallum. Happy to chat more about the paper at ICML 2026 in Seoul 🗓️ Wed, Jul 8, 2026 • 5:00 PM – 6:45 PM KST | 📍 HALL A #2805

English

187

Dhruvesh Patel ✈️ ICML 2026@_dhruveshp·23 Haz

📈 On BracketSAFE molecule strings, LoFlexMDM improves the generation quality significantly over FlexMDM for both de novo and fragment-constrained generation. The cost is a small dip in diversity, which is expected since a sharper order means less randomness. Furthermore, the learned order is interpretable: it commits structure first (ring closures, fragment separators), then fills chemistry (atoms, bonds, branches), and decides *where* fragments attach before *which* fragments attach.

English

109

Dhruvesh Patel ✈️ ICML 2026@_dhruveshp·23 Haz

But why Kumaraswamy CDFs? With a shared `a`, the hazard simplifies so both events share the same shape function and only b_ins, b_unmask set the rate. Under the time-change τ = -log(1 - t^a), the whole thing becomes an exponential race between per position b_ins and b_unmask, with the precedence constraint that insertion fires before unmasking. That buys you closed-form per-position likelihoods and parallel inverse-CDF sampling of event times. No numerical integration.

English

210

Dhruvesh Patel ✈️ ICML 2026@_dhruveshp·23 Haz

But how do we keep the training tractable? We parameterize each position's insertion and unmasking CDFs as Kumaraswamy CDFs, F(t) = 1 - (1 - t^a)^b. Fix the shape `a` to a shared constant, and let the auxiliary network predict the per-token rate parameters b_ins(x), b_unmask(x).

English

106

Dhruvesh Patel ✈️ ICML 2026@_dhruveshp·23 Haz

The trick: separate the order from the content and learn the order purely through per-position target *hazard rates* produced by an auxiliary network. The generator is trained to match the target rates and therefore the order without changing the terminal distribution.

English

100

Keşfet

@rozonoyer96703 @brozonoyer @HuLillian39250 @mehulpatwari_ @HamedZamani @MILIJOULE @RamonAstudill12 @andrewmccallum