Aaron Daubman

Our NeurIPS paper is published on arXiv. In this paper, we propose a new optimizer ADOPT, which converges better than Adam in both theory and practice. You can use ADOPT by just replacing one line in your code. arxiv.org/abs/2411.02853

@bernhardsson What temperature do you read at?

LLMs are cool but I swear my brain has been doing "next token prediction" every time I turn a page in a book for the last 30 years... still feeling a mini-thrill on every guess

"Innovative and open-source visualization application that transforms various data formats, such as JSON, YAML, XML, CSV and more, into interactive graphs."

"Make sure that your model can overfit on small training set" might be the single best sanity check when building ML models. It helped me solve countless implementation errors and to better understand capacity. I first heart it from @fchollet, thanks!

i'm often saying this!!

Learning ID-free Item Representation with Token Crossing for Multimodal Recommendation Represents items using learnable multimodal tokens instead of IDs, achieving better recommendation performance with only 20% of parameters compared to ID-based methods arxiv.org/abs/2410.19276

@tubstr @Frenck Is this pre-release frigate 15 or something else?

Tell me you have a smart home... 🏠 💡 Without telling me you have a smart home… 🤨 🤖 𝙂𝙊! 🚀 #SmartHome #HomeAssistant #Technology #IoT

@Frenck I updated one device managing my SDN last night and a bunch of morning routines failed this morning until plugs and switches were power cycled 🤦‍♂️

Having Medicare cover in-home care for the elderly is the single most substantive (and, I think, materially beneficial) policy proposal either candidate has offered in this race, and as far as I can tell, it's gotten almost no attention from the press, or on social media.

We have solved a long-standing issue with local (per node) IDF/significance computations. Introducing a global IDF model, plug-and-play with Vespa. Now, it is also much more relevant for Vespa streaming mode, which doesn't build an index. Read more at blog.vespa.ai/global-signifi…

Model2Vec distills a fast model from a Sentence Transformer by passing its vocabulary through the model, reducing embedding dims via PCA and applying Zipf weighting. Inference with the resulting static embeddings are lightning-fast, e.g. 10k texts/sec: huggingface.co/blog/Pringled/… 🧵

DyVo: Dynamic Vocabularies for Learned Sparse Retrieval with Entities Enhances Learned Sparse Retrieval by incorporating Wikipedia entities, addressing limitations in handling complex vocabulary and improving retrieval. 📝arxiv.org/abs/2410.07722 👨🏽‍💻github.com/thongnt99/DyVo

And the dirty secret is how much $$$ folks are spending on annotations of dubious quality.

A joint embedding model that leverages a fine-tuned LLM to handle music descriptions from various datasets has been released. In addition to semantic queries (genre, instruments, mood, theme), it also supports metadata similarity queries for music search."

Man sits by me on train. MAN: Loads of psychopaths around here ME: Really? MAN: Loads mate ME: How'd you know? MAN: There's signs aren't there? ME: I guess? MAN: I love them (47 minutes of awkward silence.) Man leaves train, he has a bike. I realise he was saying 'cycle paths'.

Attention in Large Language Models Yields Efficient Zero-Shot Re-Rankers Uses LLMs' attention patterns instead of generation, outperforming existing approaches on complex retrieval tasks while significantly reducing latency. 📝arxiv.org/abs/2410.02642 👨🏽‍💻github.com/OSU-NLP-Group/…

solving problems using LLMs that can be solved by fine-tuning BERT is a skill issue

Seriously impressive: patent troll blackmails Cloudflare claiming they are infringing on 4 BS patents in 100 use cases. Cloudflare goes to court, and proves they don't infrige on anything from the BS patent, and has ALL the patent troll's patents donated to the public!

Me at standup: "Tim, I'm focused on the future!"