Patrick Hunt

More news on the hiring front @cloudera : we are hiring over 175 people across Europe, Asia and the Americas to build an amazing data platform! linkedin.com/posts/phunt1_c…

My AI/ML team at Cloudera is expanding - we have open Manager as well as IC positions (more coming, keep an eye) in the USA and in India. Find out more here, great opportunity to work with some awesome folks on latest technologies to enable customers: cloudera.wd5.myworkdayjobs.com/External_Caree…

What character is this from Jules? Fail Octopus? Fail Cuttlefish? That thing from Alien? (and why can't I use images in X polls....?)

Got a chance to try Aider with gemini-2.5-pro-exp-03-25 this weekend - very impressed! This is the first time I've been able to use Gemini models successfully to vibe-code a project from scratch (with some help from grok as PM). Kudos to the Google team. Need. More. Quota. :-)

This has to be one of the best blogs that I have read on model context protocol (MCP) by @swyx @latentspacepod

🎉 Big milestone alert! 📷Our Hughes Hallucination Detection Model (HHEM) just hit 2 MILLION downloads on @huggingface! Launched last Nov, we hit 1M downloads 3 weeks ago, and now we hit 2M downloads. This recent acceleration is a clear leading indicator that the #RAG AI Assistants/Agents market is maturing and moving into more production use cases where accuracy is paramount. Thank you to our amazing community for making this possible! 📷bit.ly/4h1f8sy 📷 #NLP #HallucinationDetection #MachineLearning #RAG

@ofermend @vectara Congratulations @ofermend , @awadallah and the @vectara team! I'm excited to give Mockingbird a try - the details look amazing! vectara.com/blog/mockingbi… Please consider providing model access through @ollama - it's my main consumption route these days for testing out LLMs.

I'm super excited to share the news about @vectara Series A, and the launch of our RAG focused Mockingbird LLM: businesswire.com/news/home/2024…

With this funding, we're launching Mockingbird, a groundbreaking LLM for Retrieval-Augmented Generation (#RAG). Perfect for healthcare, law, and banking with unparalleled accuracy & performance. Join us as we revolutionize #AI for regulated industries!

@karpathy @lmsysorg Our generations version of Low-background Steel? en.wikipedia.org/wiki/Low-backg…

Nice, a serious contender to @lmsysorg in evaluating LLMs has entered the chat. LLM evals are improving, but not so long ago their state was very bleak, with qualitative experience very often disagreeing with quantitative rankings. This is because good evals are very difficult to build - at Tesla I probably spent 1/3 of my time on data, 1/3 on evals, and 1/3 on everything else. They have to be comprehensive, representative, of high quality, and measure gradient signal (i.e. not too easy, not too hard), and there are a lot of details to think through and get right before your qualitative and quantitative assessments line up. My goto pointer for some of the fun subtleties is probably the Open LLM Leaderboard MMLU writeup: github.com/huggingface/bl… The other non-obvious part is that any open (non-private) test dataset inevitably leak into training sets. This is something people strongly intuitively suspect, and also why this GSM1k made rounds recently arxiv.org/html/2405.00332 Even if LLM developers do their best, preventing test sets from seeping into training sets (and answers getting memorized) is difficult. Sure, you can do your best to filter out exact matches. You can also filter out approximate matches with n-gram overlaps or so. But how do you filter out synthetic data re-writes, or related online discussions about the data? Once we start routinely training multi-modal models, how do you filter out images/screenshots of the data? How do you prevent developers from e.g. vector embedding the test sets, and specifically targeting training to data that has high alignment (in the embedding space) with the test sets? And the last component of this is that not all LLM tasks we care about are automatically evaluateable (e.g. think summarization, etc), and at that point you want to involve humans. And when you do, how do you control for all the variables involved, e.g. how much people pay attention to the actual answer, or the length, or the style, or how refusals are treated, etc. Anyway, good evals are unintuitively difficult, highly work-intensive, but quite important, so I'm happy to see more organizations join the effort to do it well.

@simonw @vijayabhaskarj I realize it's not a "solution" - but have you considered/tried prompt compression? eg microsoft.com/en-us/research…

@vijayabhaskarj Even 100,000 tokens (or a million tokens) isn't enough for most of the things I'd want to use RAG for - I tried my blog's full archive with Gemini Pro 1.5 and it didn't fit into their million token window

Just wondering, If Groq serves bigger context models with the same speed, do we need RAGs anymore? Just load the entire dataset in parallel as context, and we can get a perfect answering machine. It would be costly, but it should easily be the most accurate one out there.

@elicollins Great to see it's already available on ollama! : ollama.com/library/gemma

The Gemini era continues with our introduction of Gemma, a new family of lightweight state-of-the-art open models. We’re releasing two models, built from the same research & tech as Gemini - yet they can run on a laptop. Looking forward to seeing what people create!

I'm really impressed with the results, this is stellar "meaningful search" -- congrats!

(plz retweet) Today we launched my new startup Vectara & announced the general availability of our easy-to-use API-first #NeuralSearch as a service. It comes with 15,000 queries/month for free. I invite you to learn more and signup for the free offering at vectara.com/meet-vectara-p…

I've got a secret to tell you... Your boss is tired of being your manager. They want you to manage them. Managing up: How to get what you want & give your boss what they need 🧵

I recently changed role at Slack, to focus on the HHVM runtime that executes so much of our product code. This is the first time I’ve worked primarily on a compiler and VM, so had to get myself up to speed. Here are some fantastic resources that have helped me do so *quickly*.

Well deserved!

Excited to announce Stemma's launch today. It's been 3 years since Lyft's data catalog - Amundsen was born. It's enabled Lyft and many others like Instacart, Square & Brex to be more effective at data-driven decision making. We are excited to bring the same to the broader market.

We're live! Introducing Continual – the missing AI layer for the modern data stack. Get continually improving predictions directly in your data warehouse without complex engineering. Request access starting today. continual.ai/post/introduci…

Please fill in the dots: "It's 2021. I am starting a new web application in Java. I'll use ...." Please retweet and help me get a wide range of answers. Thank you