Inspired Cognition

Check out this new example of using Inspired Cognition's Critique text evaluation API to evaluate chatbots!

Finally, if you want to try it out yourself, try out our power analysis widget (featured in the video at the top of the thread): #determining-testdevelopment-set-data-size" target="_blank" rel="nofollow noopener">docs.inspiredco.ai/aiguide/ai_dat… You can just plug in the system accuracies and confidence levels, then get on to data creation!

Then you run a power analysis, and it tells you how much data you need! If you want more details on how the statistics actually work, you can take a look at this nice resource here: arxiv.org/abs/2010.06595

To compare two AI systems on a task that you care about, you'll often need to create test data. This takes time and money, and often you find yourself asking "how much is enough?" Let us introduce "power analysis" a method that helps answer this question!

We're joining the Japanese NLP society's meeting this week! Find our CTO @odashi_en to discuss our work on building reliable NLP systems. 言語処理学会 #NLP2023 に参加しています！CTOの@odashi_en にお声がけください。弊社の信頼できるNLPシステム構築への取り組みについてお話します。

A rather large community driven release today! 🦜🔗v0.0.106 📃Lots (5!) of document loader additions/improvements! 🍰PromptLayer <> ChatOpenAI integration 🧑‍⚖️Critique (evaluation) integration 🔱Fake embedding class (for dry-runs) 🌲Chroma server support 👇

Check out this example of how the Critique text evaluation toolkit can be incorporated into @langchain to evaluate document-augmented question answering! github.com/hwchase17/lang…

Excited for the integration of Critique into @try_zeno! You can easily and visually compare different systems for text generation, seeing how they perform along different axes and examining individual high- and low-performing examples. Check the online demo!

Building systems with prompts is often ad-hoc, trying a few prompts, looking at the results, and guessing which one is good in practice. To make this more systematic, I made a "prompt gym" notebook to compare different prompts, models, and metrics: github.com/inspired-cogni…

If there’s anything missing, any questions, or if you’d like to discuss with us about your next AI project please get in touch via: 🐦 Twitter messages or DMs 🤖 Discord discord.com/invite/vJHdpCB… 💬 The chat or email buttons on the AI Guide docs.inspiredco.ai/aiguide/index.…

Whew, that’s a lot of content! But we’ve tried to curate the information down to the most important things to know in AI-driven app development. We hope it’s helpful to you, and…

There’s so much information about AI online that if you want to build something with AI, it can be hard to know where to start or which of many approaches to take… Introducing the 💥Inspired AI Guide💥, a curated guide that walks you through step-by-step docs.inspiredco.ai/aiguide/index.…