Eoin Hurrell

@iamreddave @JFPuget 👋When working on RAG I tend to think of the system as three different sub-systems, and work on evaluating each part in turn: 1) the pipeline into your RAG's knowledge base, 2) the 'retrieval' component (which is the searches) 3) the generation. I wrote: blog.tryreva.com/posts/building…

@JFPuget Backtesting on the last few months worth of searches is a common one. My friend @eoinhurrell is working on testing RAGs

What would be a good benchmark for RAG with an enterprise use case in mind?

@eoghan 👀

Yours truly going deep on what we think is next for AI agents. And telling our competitors all of our plans in the process :) Our experience of life of the internet is about to look very, very different. This is the season finale of Off Script.

At @TryRevaAI we just published a deep evaluation of OpenAI and Anthropic for @intercom's customer support use case, to see the impact for their customers of their move to Claude blog.tryreva.com/posts/benchmar…

Fascinating to see how tech leaders are thinking about generative AI, without factoring in model quality. Surely how well the model can do on your task should be a primary concern? Still, this is a good read on judging vendors for common issues leaddev.com/tech/how-choos…

The feature most missing in the LLM space is clarity

@NoChorus Favs being hidden means I can just say “fav this if you mean Chapters” I guess

@vboykis Science fiction is a fun hobby, I guess

lesswrong out here like , “what if the model escapes the data center” and I’m here simply like “how do I make sure it even makes it correctly into VRAM in the data center?”

Sorry I can’t take the AGI risk seriously, like do you know how many stars need to align to even deploy one of these things. if you breathe the wrong way or misconfigure the cluster or the prompt template or the vLLM version or don’t pin the transformers version —

The Evolving Landscape of LLM Evaluation Navigating Evaluation Pitfalls Thoughts on memorization and overfitting in LLM evals, a shift to 'vibe-based' evals, and how the future of LLM evaluation may look like. newsletter.ruder.io/p/the-evolving…

@NoChorus Some real emotional scenes that’ll cut right through you in Brooklyn, so I hear.

Ireland is seeking legislation to allow the use of facial recognition technology (FRT) for policing, and there currently exist little critical discussion/awareness on its dangers. I'm hoping my piece, fresh out of press, contributes. Pls share widely irishtimes.com/opinion/2024/0…

@EamonLeonard @brandalisms nohello.net/en/ 👀

@brandalisms Hi Andreeeeeeeeaaaaaa!

If someone else just pings me with “Hi Andreea” and then silence, I will jump out a window and run into the mountains and live as a hermit for the rest of my life. Just tell me what you want.

If you're an early stage founder working on your first round of funding and are struggling with how to tell your story to people who could invest, then I wrote this for you. broadstone.vc/blog/what-ever… #startups #fundraising

@NoChorus The Destruction of The Temple by Barry N Malzberg, which just happens to be the book I randomly picked to start reading two days ago from the massive pile I bought in Hay-on-Wye (it’s good so far)

@NoChorus I’ll put “organise some sort of dead drop situation” on the to do list

@jmason Congrats, great interview. That last paragraph, just as an aside!

@NoChorus It’s been a rough week

@DetectiveKen sickos.jpeg

@DetectiveKen Absolutely, they tend to be easy reads

@DetectiveKen Horus heresy is very much a “pick which ones you like based on the armies you like” series, and great fun. I’ll never read all 50. Recommend any ones covering Chaos marine armies, especially the thousand Sons