Finally got around to reading what @GoDaddy said about #prompt engineering.
godaddy.com/resources/news…
1. Sometimes one prompt isn’t enough — makes sense to break down prompts instead of one giant prompt to rule them all. Also, would it make sense to present the user a old fashioned list of options get a read on which prompt to route him/her to?
2. Be careful with structured outputs. The old editor in me might ask “where should you not be careful?” That aside, like the idea of lowering the prompt temperature for structured outputs and especially identifying (and testing for) common failure modes.
3. Prompts aren’t portable across models — I’m sure this is right. I would love for someone (maybe us) to analyze how outputs vary across models for different types of prompts.
4. AI guardrails are essential. Given the probabilistic nature of #GenAI output, interrogate the results to determine when to escalate to a number. Makes sense — but how do you determine the full set of guardrails you need? What bad results do you want to look for?
5. Models can be slow and unreliable. 1% of chat interactions fail at the GenAI provider, and they time out after 30 seconds. Makes sense to move to more asynchronous responses.
6. Memory management is hard. They seem to be considering using stacks to implement memory — provide working memory to delegate prompts and reap the results when the discussion moves back to the controller.
7. Adaptive model selection is the future (even though they haven’t implemented it you). How do you reconcile adaptive model selection with the observation that prompts don’t work across models? (Which I suppose means you select the model at “design” time rather than “run” time?)
8. Use #RAG effectively. Suggested a couple of patterns for RAG, which I am not sure I agree with. I would argue that GenAI in the enterprise will involve more RAG than non-RAG, so there will be many, many RAG patters. I wonder if GoDaddy has a bit of a narrow view as they mostly seemed to talking about a customer support use case.
9. Tune your data for RAG. Argues that you can remove extraneous language to improve query performance. Do you really need to do this. Couldn’t you do RAG at design time, shove the results into a database and query that at run time?
10. Test! Test! Test! Points out that test will be more time consuming that build in #LLM integration. I am sure this is true — again data about how much you give in increased testing in return for reduce build effort would be interesting.
English
