tom dyson

@Iliane_5 @simonw prettyprompt includes some (currently playful) attempts at this: #sniffing-bad-intentions" target="_blank" rel="nofollow noopener">github.com/tomdyson/prett…

@simonw You can use a smaller model (ex: Ada, Cabbage, etc) to screen messages and then either block/send them to the bigger model (gpt3.5/gpt-4) for processing. Similar setup as OpenAI’s moderation endpoints for ChatGPT.

Anyone know what the current state of the art in prompt injection mitigation methods is? Given the huge amount of new code being deployed against LLMs like GPT3/3.5/4 in the past few months I would expect prompt injection attacks to be widespread and escalating in severity

prettyprompt is the beginning of a collection of tools for cleaning, sanitising and converting inputs to prompts: github.com/tomdyson/prett… These may be useful alongside @langchain apps like Microllama. Issue #1 is prompt injection detection.

Microllama now supports streaming responses. Working this out took me as long as the original @langchain / @FastAPI proof of concept! I've moved sources first to 1. highlight authority of response (vs standard chatGPT) 2. give you something to look at while the answer loads.

Here's Microllama using data from NHS Conditions.

I'm hosting my test Microllamas on Google Cloud Run, which feels like the perfect platform for Langchain-style apps (OpenAI wrappers + FAISS indexes). Hosting is basically free until you're famous, and then OpenAI costs are your problem.

Here's Microllama, which is intended to be the easiest possible way of deploying a talk-to-your-content API: github.com/tomdyson/micro… It leans heavily on @langchain. IMO the most interesting bit is to do with baking the vector index into the container at build time.

this claim of election fraud is disputed