Artyom Gadetsky

(1/n) The usual assumption in GFlowNet environments is acyclicity. Have you ever wondered if it can be relaxed? Does the existing GFlowNet theory translate to the non-acyclic case? Is efficient training possible? We shed new light on these questions in our latest work! @icmlconf

@jiaxinwen22 @jiaxinwen22 @janleike You may find our ICLR 2025 paper (openreview.net/forum?id=ohJxg…) highly relevant and to be discussed. We develop the framework that employs the joint probability of answers to a set of questions to perform fully unsupervised adaptation of an LLM/VLM.

New Anthropic research: We elicit capabilities from pretrained models using no external supervision, often competitive or better than using human supervision. Using this approach, we are able to train a Claude 3.5-based assistant that beats its human-supervised counterpart.

@AlexGDimakis @wzhao_nlp You may find our recent ICLR paper (openreview.net/forum?id=ohJxg…) interesting as well. We show that one can perform fully unsupervised adaptation of an LLM by seeking for answers that maximise their joint likelihood defined by an LLM itself. Works for reasoning tasks too.

"RL with only one training example" and "Test-Time RL" are two recent papers that I found fascinating. In the "One Training example" paper the authors find one question and ask the model to solve it again and again. Every time, the model tries 8 times (the Group in GRPO), and a gradient step is performed, to increase the reward which is a very simple verification of the correct answers, repeated thousands of times on the same problem. The shocking finding is that the model does not overfit to this one question: RL on one example, makes the model better in MATH500 and other benchmarks. (If instead you did SFT repeating one training question-solution finetuning, the model would quickly memorize this answer and overfit). But with RL, the model has to solve the problem itself, since it only sees the question, not the answer. Every time it produces different answers, and this seems to prevent overfitting. The other papers are relying on the same phenomenon: you can have a small number of training questions and re-solve them thousands of times. You can do this for the test set (as test-time RL does) and still not overfit. We also independently saw this by doing RL training on half the test set and seeing benefits in the other half for BFCL agents. My thought now is that this shows our RL learning algorithm must be extremely inefficient. When a human is learning by solving a math puzzle, they immediately learn what they can learn by solving it once (or twice). No further benefit would come by assigning the same homework problem to students a tenth time. But in RL, we keep asking the model to re-solve the same question thousands of times, and the model slowly gets better. We should be able to have much better RL learning algorithms since the information is there. (1/2)

Large (Vision) Language Models are Unsupervised In-Context Learners Joint inference enables fully unsupervised adaptation for LLMs and VLMs (no labels, no prompts). Instead of per-input zero-shot prediction, it solves all inputs together, uncovering structure across tasks. Two scalable forms: unsupervised fine-tuning and ICL. Achieves +39% on GSM8K, matching supervised methods across NLP, math, vision, and API-only models like GPT-4o.

Tired of manual prompt engineering to solve new task with your LLM? We introduce Joint Inference—a framework for fully unsupervised adaptation of large (vision) language models that often performs on par with supervised approaches 🔥 #ICLR2025 In collaboration with @zamir_ar lab — huge kudos to our amazing students: @artygadetsky, @andrew_atanov, @YulunJiang, @zhitong_gao, Ghazal Hosseini Mighan 🔗 Website: brbiclab.epfl.ch/projects/joint… 📄 Paper: openreview.net/pdf?id=ohJxgRL… 💻 Code: github.com/mlbio-epfl/joi…

@y0b1byte You might find this interesting as well openreview.net/forum?id=ohJxg… Joint probability of responses under a model may also provide a signal to get improved responses without having any ground truth.

this is a very counterintuitive result

@norpadon @EmilMieilica You can also use Plackett-Luce in case the function being optimized is defined only for hard permutations, i.e., imagine you learn a custom order of generating words in the sentence instead of left-to-right. arxiv.org/abs/1911.10036

@EmilMieilica This allows you to define smooth relaxations to the permutation operators similar to how you can use softmax to make a smooth version of an argmax You can also apply the classic Gumbel-softmax trick to Sinkhorn matrices to get a stochastic approximation: arxiv.org/abs/1802.08665

By the way, if you didn't already know, sorting is a (kinda sorta) differentiable operation

1/ GFlowNets are known for training a forward policy to generate complex objects step by step. However, an equally important piece specific to the GFlowNet paradigm is a backward policy, which undoes these steps and plays a crucial role in training.

How can we discover fine-grained classes within a coarsely labeled dataset? Excited to present FALCON at #ICML2024, a method for uncovering fine-grained classes without fine-grained supervision. FALCON is versatile and works across various domains! W/ @GrcicMatej @artygadetsky Website: brbiclab.epfl.ch/projects/falco… Paper: arxiv.org/pdf/2406.11070 Code: github.com/mlbio-epfl/fal…

Scaling up is everywhere (and is effective), but what about scaling down? I am excited to share our new work on solving vision tasks with simple low-resolution vision sensors (1-pixel cameras).

Can unsupervised transfer outperform zero-shot transfer? In our #ICML2024 paper we show it can! 🔥 We present 🐢TURTLE , a method for fully unsupervised transfer with any foundation model! TURTLE also achieves SOTA unsupervised performance 🔥 Kudos to @artygadetsky @YulunJiang Paper: arxiv.org/pdf/2406.07236 Website: brbiclab.epfl.ch/projects/turtl… Code: github.com/mlbio-epfl/tur…

@francoisfleuret pytorch.org/docs/stable/ge… Might be helpful

I want to compute the stuff formalized in the first pic in O(log(T)). The implementation in the second pic is correct but numerically unstable. I have a solution that requires a full-fledged associative scan. Is there a simpler one?

Je vais à Montréal! This June I'm starting a new position as an assistant professor at @UMontreal and as a core academic member of @Mila_Quebec. Drop me a line if you're interested in working together on problems in AI4Science, Optimal Transport, and Generative Modeling.

How to infer human labelling of a given dataset in a model-agnostic way? Check our new method HUME accepted at @NeurIPSConf as #spotlight!🌟 HUME provides a new view to tackle unsupervised learning. Kudos to my fantastic PhD student @artygadetsky! Paper arxiv.org/abs/2311.02940

@SashaVNovikov WOW

#AlphaTensor: adapting AlphaZero to symbolically find better (exact) matrix multiplication algorithms. By putting coefficients of the symbolic expression into a tensor, the algorithm design task becomes an (NP-hard) low-rank tensor decomposition problem, which we attacked with RL

@andrey_oshev Если бы ты не бы женат, то подумал бы, что это таргетированная реклама тебе от Насти

В чем прикол? Аккаунт под замком, я не подписан и никогда не был, но все равно вижу твиты и даже ответы к ним. Имейте ввиду :)

We will be presenting “Leveraging Recursive Gumbel-Max Trick for Approximate Inference in Combinatorial Spaces” during #NeurIPS2021 poster session 8! We iteratively apply the Gumbel-Max trick to obtain structured variables instead of categorical. Poster: neurips.cc/virtual/2021/p…

New 🦙 LaMa inpainting is out! Trained only in 256x256 but generalizes to ~2k! The core of the system is Fourier convs in the spectral domain. It is very good at repetitive patterns! 📰 arxiv.org/abs/2109.07161 💻 github.com/saic-mdal/lama @PyTorch 🧪 bit.ly/3iPw4pv

“Uncertainty-Aware Machine Translation Evaluation” is now on arXiv! A first step towards informative confidence estimates for MT quality predictions. Accepted to #EMNLP2021 findings. arxiv.org/abs/2109.06352 [1/7]

Super happy to finally see our work led by magnificent Irina @irisaparina online. We train a system for turning questions into executable database queries with annotation simpler than full queries. @pytorch code released 🥳 #EMNLP2021 #NLProc @YandexAI arxiv.org/abs/2109.06162