Ben Roesch

New in Hinsley: multiple choice forecasting questions (beyond Yes/No) and crowdsourcing campaigns - a survey-like experience to collect predictions from external participants via shareable links. hinsley.ai/release_notes/…

AI forecasting isn't just about accuracy - it's about agility. Now, the questions you're asking can evolve as fast as the world they're trying to anticipate. hinsley.ai/blog/throwing-…

For several months, we've had Hinsley (hinsley.ai) submit regular forecasts on one of the forecasting platforms we run in conjunction with Bertlesmann Foundation (rangeforecasting.org). As of today, HinsleyBot is officially at the top of the accuracy leaderboard!

v1.4.0 of @raif_ai is available now for your #rails apps. Google adapter, plus a long list of agent, web admin, & general quality-of-life improvements. Full list: github.com/CultivateLabs/…

My latest blog post from the @cultivatelabs blog, spelling out why the obsession with forecasting accuracy risks ignoring the bigger challenges - cultivatelabs.com/posts/will-ai-…

@aymorgan @c_sonnier Raif v1.3.0 just shipped with built-in eval support. Might be of interest! docs.raif.ai/key_raif_conce…

@c_sonnier @paolino @rails @obie @kieranklaassen @aviflombaum Thanks very much! Will check it out. 👍🏻

Hey RoR AI gang - long time listener, first time caller here 👋🏼 Currently using RubyLLM (🫡 @paolino ), do you have any advice around implementing EVALS within a @rails AI app? Seen @obie talks about #roast and #lexa by @kieranklaassen but I'm just getting started, any advice?

@raif_ai @mattsolt @cooperx86 @lucianghinda might be of interest!

Just shipped v1.3.0 of @raif_ai with built-in support for LLM evals! Evaluate the quality of LLM responses for different models in your Ruby on Rails apps. Built-in support for using LLM-as-judge. docs.raif.ai/key_raif_conce…

@windlejacob12 We don't use langchain (we extracted @raif_ai from our app), but we've do most of our orchestration via a Workflow model/class. Largely just a pre-defined series of LLM calls. We tried some more agentic stuff but found workflows to work better.

Starting to integrate Langchain Ruby into our Rails backend for agent workflows and entity processing. Has anyone used langchainrb in production? Would love to hear your experiences with prompt chaining and LLM orchestration in Ruby apps 🤔 #Rails #Langchain #Ruby #AI

Raif has a new docs site! docs.raif.ai

We released a new version of Raif, a Rails engine for working with LLMs. It now supports streaming responses in chat & working with provider managed tools like web search/code execution. github.com/CultivateLabs/… It powers all of the LLM features in ARC: arcanalysis.ai

@_samadahmed @neejkapoor Raif is a Rails engine that includes a Chat UI -- github.com/CultivateLabs/…

@neejkapoor @paolino i need something like that so bad

Rewrote the RubyLLM docs entirely. Old structure started to feel scattered & repetitive. New approach: Rails Guides style – clearly explain once, link everywhere, TOCs in each page. Less reading, more building. rubyllm.com

🎊Our 2024 wrap-up just hit inboxes! Dive into our CEO @amsiegel's year-end review + get early access to ARC, our new tool combining human expertise with #AI capabilities to help analysts work smarter. Take a look: mailchi.mp/cultivatelabs/…

Our March #newsletter shares several recent highlights, including: 🤝a partnership with @RANDCorporation & @INFERpub 📊a workshop during a @BertelsmannFdn transatlantic relations event 🏛️2 experts' take on forecasting for national security Read more >> bit.ly/3Vlvfcd

How does #crowdforecasting work, you ask? Our latest blog dives into: ✔️a brief history of forecasting ✔️our typical outputs for analysts and decision makers ✔️how you can harness crowd forecasting for your organization See more >> cultivatelabs.com/posts/how-crow…

We are pleased to share that #INFER is now led by @RANDCorporation. We look forward to this new chapter that will enable INFER to leverage #RAND expertise in our mission to advance the #forecasting capability of the US govt. See our announcement>>infer-pub.com/the-pub/rand-p…

Having trust issues with the news these days? Our latest blog shares how crowdsourced #forecasting is cutting through #disinformation to reveal truth. Read more >> bit.ly/485cInI

An effective #crowdforecasting program hinges on asking the right questions. How does Cultivate develop forecast questions that are useful and actionable for decision-makers? Learn more about our process from a Q&A w/ our teammates in our latest blog >> bit.ly/3r9nRUl

Does forecasting accuracy really matter? Yes, but improving accuracy isn’t necessarily the most valuable aspect of a crowdsourced forecasting effort – and it shouldn’t be its main focus. Learn more in our latest blog >> cultivatelabs.com/posts/does-for…

@shashj @KofmanMichael Someone needs to get them in to a forecasting tournament so we can Brier score their predictions