Iacopo Vagliano

Needed update to tripod and tripod AI or reporting LLM studies!

Don't do RAG Proposes cache-augmented generation (CAG) to eliminate retrieval latency and minimize retrieval errors. What is CAG? CAG aims to leverage the capabilities of long-context LLMs by preloading the LLM with all relevant docs in advance and precomputing the key-value (KV) cache. The preloaded context helps the model to provide contextually accurate answers without the need for additional retrieval during runtime. When to apply CAG? It's a useful alternative to RAG for cases where the documents/knowledge for retrieval are of limited, manageable size. My thoughts: As LLMs advance in capabilities, I suspect that what we know as RAG today could change significantly either architecturally or how it's optimized. CAG is one in a growing list of developments and new ideas that have emerged recently to address limitations like poor retrieval relevancy and latency. There could also be hybrid methods that combine preloading with selective retrieval. Don't sleep on long-context LLMs. They are here to stay.

The data science revolution is getting closer. TabPFN v2 is published in Nature: nature.com/articles/s4158… On tabular classification with up to 10k data points & 500 features, in 2.8s TabPFN on average outperforms all other methods, even when tuning them for up to 4 hours🧵1/19

Spunti interessanti per chi vuole sapere di più sull'IA e le sue possibili applicazioni.

Nice opportunity in a great group!

Thank you for tuning in! Slides for my talk "Integrating Graph Neural Networks and Large Language Models" dropbox.com/scl/fi/pzgk789…

An example to keep in mind when deciding about preprints. Thanks Richard for sharing it.

@michaelcochez opening the learning on graph conference Amsterdam #LoG AMS @LogConference

Nice hints!

🚨 Are PostDocs Alright? 🚨 Join us on 27.11.2024, for a joint event by @LeibnizPostDocs & German Postdoc Network as we discuss the latest findings from the Leibniz PostDoc survey! 🕛12:00-13:00 CET Register now: bit.ly/40SQ6GA #postdocs #IchBinHanna

Don't miss it!

Meet Tülu 3 -- a set of state-of-the-art instruct models with fully open data, eval code, and training algorithms. We invented new methods for fine-tuning language models with RL and built upon best practices in the community to scale synthetic instruction and preference data. Demo, GitHub, technical report, and models below 👇

📢 25 november komen we in actie tegen de bezuinigingen op onderwijs en onderzoek. Laat ook je stem horen voor toegankelijk onderwijs, tegen ontslagen en tegen de langstudeerboete. Teken de petitie een kom in actie! campagnes.degoedezaak.org/campaigns/stop…

Incredible LLM Creation Visualization in this Site. Click on each section, like Embedding, LayerNorm, Self Attention, and it will show you the mechanics of that section . (link in comment)

@MihaelaVDS @uni_copenhagen @JEKlopotowska

Excited to speak at @uni_copenhagen's NorPEN meeting, exploring how AI can transform drug safety & present effectiveness studies & real-world evidence for Nordic countries in my talk: "Pharmacoepidemiology Reimagined: Could AI be a Game-Changer?" All info: eventsignup.ku.dk/norpen/confere…

.@MistralAI launched a ton of new AI features/models today! The best part? It's all absolutely free. Here's everything you need to know: 👇

If you look for new research challenges in NLP, here you are with an inspiration

📢 Call for Papers: Exploring the History of the Web, from Inception to Present @TheWebConf 2025 @TheOfficialACM 📃 2-4 pages paper submission 12/12/2024 👉 www2025.thewebconf.org/history-of-the… #TheWebConf25 #WebHistory #InternetHistory #www cc @w3c @CERN @Inria @timberners_lee @oshaniws

G-Retriever (@xxhe/graph-retrieval-augmented-generation-rag-beb19dc30424" target="_blank" rel="nofollow noopener">medium.com/@xxhe/graph-re…) that leverages the strengths of GraphRAG, LLMs, and GNNs, has now been integrated into the PyG library. Thanks to @jure and the @PyG_Team team for making this possible!

Great read - "Understanding LLMs: A Comprehensive Overview from Training to Inference" The journey from self-attention mechanism to the final LLMs. This paper reviews the evolution of large language model training techniques and inference deployment technologies. -------- → The evolution of LLMs and current training paradigm Training approaches have evolved from supervised learning to pre-training and fine-tuning, now focusing on cost-efficient deployment. Current focus is on achieving high performance through minimal computational resources. → Core architectural components enabling LLMs' success The Transformer architecture with its self-attention mechanism forms the backbone. Key elements include encoder-decoder or decoder-only designs, enabling parallel processing and handling long-range dependencies. → Key challenges in training and deployment Main challenges include massive computational requirements, extensive data preparation needs, and hardware limitations. Solutions involve parallel training strategies and memory optimization techniques. → The role of data and preprocessing in LLM development High-quality data curation and preprocessing are crucial. Steps include filtering low-quality content, deduplication, privacy protection, and bias mitigation. 🔍 Critical Analysis & Key Points: → Data preparation strategies drive model quality Processing raw data through sophisticated filtering, deduplication and cleaning pipelines directly impacts model performance. → Parallel training techniques enable massive scale Using data parallelism, model parallelism and pipeline parallelism allows training billion-parameter models efficiently. → Memory optimization is crucial for inference Techniques like quantization, pruning and knowledge distillation help deploy large models with limited resources.