P.S. This is not an investment suggestion—don’t go buying NVDA stock based on this!!!
English
Xia “Ben” Hu
103 posts
Xia “Ben” Hu
@huxia
Associate Professor of CS@Rice working on AutoML, XAI and Network Analytics. Author of AutoKeras and NCF.
Houston, TX Katılım Eylül 2009
326 Takip Edilen663 Takipçiler
If DeepSeek can push AI forward in areas like logical reasoning and mathematical derivation, the demand for compute power won’t decrease—it’ll skyrocket due to the long-tail effect. As AI applications become more widespread, compute needs will expand to everyday tasks.
English
A few months ago, while working on a project that required an open-source model for mathematical reasoning, we came across DeepSeek's project. At the time, we even considered building something on top of it, though unfortunately, my proposal didn’t get funded 😂.
English
Xia “Ben” Hu retweetledi
AWS just released a new Multi-Agent AI framework
It lets you manage multiple AI agents, dynamically route LLM queries, maintain context across AI Agents and can be deployed locally on your computer.
100% opensource.
English
Xia “Ben” Hu retweetledi
After "Attention Is All You Need", AI paper titles be like:
English
Xia “Ben” Hu retweetledi
📢Postdoc Position📢
Dr. Xia Hu @huxia and I are looking for a Chairman's Postdoctoral Fellow in Efficient and Trustworthy LLMs @RiceCompSci
If you are interested, please do not hesitate to apply: docs.google.com/document/d/1Ks…
English
Thanks @LangChain for mentioning our work Self-extend. Here are my (biased) thoughts: 1) Long-context and RAG complement each other for now, at least till the long-context capability of models could become similar to short contexts.
LangChain@LangChain
RAG for long context LLMs: Video Will long context LLMs really kill RAG? This is a talk @RLanceMartin gave at a few recent meetups that pulls together threads from a few different projects related to this question. Multi-needle in a haystack shows limitations in long-context LLM reasoning & retrieval over multiple facts. But, RAG may evolve in a few ways. While query analysis likely remains critical, we may see a shift towards full document indexing (e.g., RAPTOR, multi-representation indexing) & long context embeddings. We also may see a shift away from a naive prompt : response paradigm to a "flow" paradigm where RAG answer are built iteratively (Self-RAG, C-RAG) with post-retrieval reasoning and feedback. 📽️ Video: youtu.be/SsHUNfhF32s 📓 Slides: docs.google.com/presentation/d… ⛓️ Links: 1/ Multi-needle analysis w/ @GregKamradt blog.langchain.dev/multi-needle-i… 2/ RAPTOR (@parthsarthi03 et al) github.com/parthsarthi03/… youtube.com/watch?v=jbGchd… 3/ Dense-X / multi-representation indexing (@tomchen0 et al) arxiv.org/pdf/2312.06648… blog.langchain.dev/semi-structure… 4/ Long context embeddings (@JonSaadFalcon, @realDanFu, @simran_s_arora) hazyresearch.stanford.edu/blog/2024-01-1… together.ai/blog/rag-tutor… 5/ Self-RAG (@AkariAsai et al), C-RAG (Shi-Qi Yan et al) arxiv.org/abs/2310.11511 arxiv.org/abs/2401.15884 blog.langchain.dev/agentic-rag-wi…
English
2) How to handle long contexts is challenging - The way how commercial models are handling this is unclear and they chose not to open it, also the community conjectures that they didn't deal with the problem in an elegant way that may cause other problems (no experiments).
English
Without fine-tuning, Self-Extend has significantly improved performance of the Gemma-2b-it performance in the needle in the haystack task, increasing its capability from less than 8k (pretraining window) to over 90k!!
lnkd.in/gD_DqJvi
HongyeJ@NeurIPS@serendip410
Despite the mixed feelings about Google's latest Gemma model, we're big fans! @GoogleAI Why? Coz we found it pairs incredibly well with our SelfExtend 🤣🤣🤣 - like, perfectly! With Self-Extend, no fine-tuning needed, we effortlessly expanded Gemma's window from 8k to 90k+! On the 'Needle in the haystack' task, Gemma-2b-it even struggled at 8k, but with SelfExtend, Gemma-2b-it easily tackles it within 90k range! #AI #Gemma #SelfExtend #LLMs 🚀 Paper: arxiv.org/abs/2401.01325 Github: github.com/datamllab/Long…
English
Xia “Ben” Hu retweetledi
Excited to introduce KIVI🥝, the first 2bit KV cache quantization breakthrough! 🚀 KIVI can be directly integrated into existing LLMs without any tuning.
📄 Paper: arxiv.org/abs/2402.02750
💻 Code: github.com/jy-yuan/KIVI
#KIVI #LLM #AI #MachineLearning
English
Xia “Ben” Hu retweetledi
Welcome more people to trying Self-Extend/LongLM on more diverse tasks and models to see when it works and identify its limitations. Here is our implementation github.com/datamllab/Long… . Credits to @serendip410 !
Alex Graveley@alexgraveley
Worked for us on llama2 as well!
English