Joemon Jose

Big congrats to Ángela López Cardona & Sebastian Idesis on our latest IJCNN publication! We review how cognitive signals, esp. eye tracking, enhance language & image understanding in LLMs & VQA. Preprint: lnkd.in/erHm9NUx #AI #NLP #eyetracking #LLMs #IJCNN #TID

Computer Languages and their founders

"An Instruction-Response Perspective on Large Language Models in Information Retrieval Tasks" by @hideo_joho and @joemonj (Glasgow) has been accepted at #sigir2025 as a perspective paper.

This is the first outcome of my sabbatical at Glasgow. Hopefully, more to follow. Thanks again to @joemonj and @ir_glasgow for their collaboration.

New short course: Attention in Transformers: Concepts and Code in PyTorch. Last week we released a course on how LLM transformers work. This week, go deeper and learn about the technical ideas behind the attention mechanism, and see how to code it in PyTorch. This course is built with @joshuastarmer, Founder and CEO of StatQuest. The attention mechanism was a breakthrough that led to transformers, the architecture powering large language models like ChatGPT. Transformers, introduced in the 2017 paper: "Attention is All You Need" by Viswani and others, took off because of its highly scalable design. In this course, you’ll learn how the attention mechanism, a key element of transformer-based LLMs, works and implement it in PyTorch. You'll develop deep intuition about building reliable, functional, and scalable AI applications. What you will do: - Understand the evolution of the attention mechanism, a key breakthrough that led to transformers. - Learn the relationships between word embeddings, positional embeddings, and attention. - Learn about the Query, Key, and Value matrices, and how to produce and use them in attention. - Walk through the math required to calculate self-attention and masked self-attention to learn why and how they work. - Understand the difference between self-attention and masked self-attention and how one is used in the encoder to build context-aware embeddings and the other is used in the decoder for generative outputs. - Learn the details of the encoder-decoder architecture, cross-attention, and multi-head attention and how they are all incorporated into a transformer. - Use PyTorch to code a class that implements self-attention, masked self-attention, and multi-head attention. There're lots of exciting technical details in this course. Please sign up here: deeplearning.ai/short-courses/…

📣 📣 The #sigir2025 LiveRAG Challenge is open at: sigir2025.dei.unipd.it/live-rag-chall… Prizes: - First Prize: $5000 - Second Prize: $3000 - Third Prize: $2000 Deadlines: - Application submission deadline: February 24, 2025 More info? Ask sigir2025-liverag-gen@tii.ae.

I am excited to kick off this year with our first perspectives article, exploring bio-inspired AI and its groundbreaking applications in Telecommunications and Cybersecurity! lnkd.in/dKsuiM-i #telefonicascientificresearch #discovery #tid #ai #neuroscience

@joemonj @alexk_z @iarapakis @XinXin_SDU @YueFeng__ @twitoogle @ron_junchen_fu Important Dates Submission Deadlines: January 1, 2025 Acceptance Notification: 13 January, 2025 Camera-ready: 2 February, 2025 EReL@MIR Workshop: 28-29 April 2025 Deadlines refer to 23:59 (11:59 pm) in the AoE (Anywhere on Earth) time zone.

Glad to organize the 1st EReL@MIR Workshop (Efficient Representation Learning for Multimodal IR) erel-mir.github.io at The Webconf25 (Sydney, Australia) with @joemonj @alexk_z @iarapakis @XinXin_SDU @YueFeng__ @twitoogle @ron_junchen_fu . Welcome everyone to contribute.

We (@ZihanWa54274484, @joemonj, HaitaoYu, Weizhi Ma,@zhaochun_ren, @xinxin) are holding the R^3AG workshop(r3ag-sigir-ap.github.io) on SIGIR-AP2024 (December 9-12, 2024, Tokyo, Japan); The topics mainly include Reliable RAG-related research. Submission DDL: October 3, 2024

@ZihanWa54274484, @Solis_xurige, @XinXin_SDU, @joemonj, Haitao, Weizhi, and I will be organizing the workshop about Refined and Reliable Retrieval Augmented Generation at SIGIR-AP2024 @ACMSIGIR_AP

📷 Thrilled to announce our workshop "Information Retrieval's Role in RAG Systems (IR-RAG)" has been accepted at #SIGIR2024! 📷 Can't wait to share ideas, innovations, and connect with fellow enthusiasts. #IRRAG2024 #InformationRetrieval #AIResearch @rstless-group/ir-rag-sigir24" target="_blank" rel="nofollow noopener">coda.io/@rstless-group…

Beyond a successful submission, it has also been an adrenaline-fueled research sprint positioned on the SoA of Reinforcement Learning-based Recommender Systems with Large Language Models, with @sasxbd, @joemonj and @alexk_z Our paper is available here: arxiv.org/abs/2403.16948

Congratulations to Donna, Fernando, Mark, Mounia, and Yiqun! #sigir

"Reinforcement Learning-based Recommender Systems with LLM for State Reward and Action Modeling" has been just accepted at #SIGIR2024 LLM (LE) simulates the user interactions generating state actions and rewards that are used to train the model with RL. arxiv.org/abs/2403.16948

I really love the idea of MonsterCLEF at #clef2024! monsterclef.dei.unipd.it Makes so much sense but also hard to pull off! A meta-challenge organized so that task-specific LLMs run against diverse tasks without adapatation. New test data for each task, so no data leakage!

I am really honoured to join this amazing group of IR scholars. SIGIR has been a big part of my growth as a researcher, so a big thank you. It feels extra special to be welcomed into the SIGIR Academy with @IR_oldie; we both started working on IR around the same time in Glasgow.

Our paper, "IISAN: Efficiently Adapting Multimodal Representation for Sequential Recommendation with Decoupled PEFT," has been accepted at #SIGIR2024 as a full paper!🎉🎉🎉 Thrilled to share this collaboration with @Solis_xurige , @joemonj , @alexk_z, & @iarapakis. 👏👏👏

Glad to share that our paper "CFIR: Fast and Effective Long-Text to Image Retrieval for Large Corpora" has been accepted at #SIGIR2024 as a full paper, a joint work with @Solis_xurige, @richardm_, and @joemonj.

Delighted to share that I'll be spending part of my sabbatical at the University of Glasgow until March 2025. Grateful to @joemonj for hosting me. Looking forward to meeting former colleagues and new friends.