Fabio Fehr

@DamienTeney Thanks @DamienTeney for the encouragement! The higher spearman's correlation shows the length is more adaptive to shorter and longer length summaries. In the Appendix (Figure 5 and Table 7) Curation Corpus and CNN/DM dataset we get improved OOD performance for longer summaries.

@FabioFehr Interesting results. To be totally convincing, I'd like to see the same exp with ID/OOD data swapped. Eg if shorter summaries are better OOD, you want to make sure there's not simply a systematic "making shorter summaries" happening (which wd be detrimental with sets swapped).

I am so excited by our latest work on ArXiv! TL;DR: We apply NVIB to pretrained Transformers which allows for an information theoretic post-training regularisation. With no weight updates we achieve improved performance in out-of-domain generalisation! arxiv.org/abs/2312.00662

An overview of Large Language Models (LLMs) pitched in a very accessible way! youtube.com/watch?v=zjkBMF…

I am really excited our short paper "Learning to Abstract with Nonparametric Variational Information Bottleneck" got accepted to the findings of EMNLP 2023! A big thanks to my college Melika Behjati and supervisor James Henderson

🤖🧠NEW PAPER🧠🤖 Bayesian models can learn rapidly. Neural networks can handle messy, naturalistic data. How can we combine these strengths? Our answer: Use meta-learning to distill Bayesian priors into a neural network! Paper: arxiv.org/abs/2305.14701 1/n

"HyperMixer: An MLP-based Low Cost Alternative to Transformers" at ACL2023, with @_florianmai, @ArnaudPannatier, @FabioFehr, Haolin Chen, François Marelli, and @JamieBHenderson. arxiv.org/abs/2203.03691 @unige_en @Idiap_ch @EPFL_en 2/2

To volunteer at a conference? - What are the pros and cons?

I am excited to announce our @iclr_conf 2023 paper: "A VAE for Transformers with Nonparametric Variational Information Bottleneck" openreview.net/pdf?id=6QkjC_c… We propose to model Transformer embeddings as nonparametric mixture distributions using Dirichlet processes. @FabioFehr

I am very excited to announce my first paper towards my PhD has been accepted at @iclr_conf 2023 in Kigali, Rwanda! "A VAE for Transformers with Nonparametric Variational Information Bottleneck" @JamieBHenderson @Idiap_ch arxiv.org/abs/2207.13529 openreview.net/forum?id=6QkjC…

A Variational AutoEncoder for Transformers with Nonparametric Variational Information Bottleneck deepai.org/publication/a-… by James Henderson and @FabioFehr #Autoencoder #Vector

TL;DR: we propose an MLP-Mixer whose parameters are modulated by an MLP. It avoids the O(T^2), captures long-range dependencies attention-style, and is easier to meta-optimize! With @_florianmai, @ArnaudPannatier, @fabiofehr, H. Chen, F. Marelli, and @JamieBHenderson.