Jeelex retweetledi
I made this last weekend to experiment w/ building an app end to end on LLMs: vibecheck.market
It's like Wirecutter, but uses an LLM to recommend product choices based on reddit conversations and reviews, so you don't have to spend 20-30min reading reddit
My experience: I'm late to the game, but this is the first time I've tried building an app end to end around an LLM.
1. It's very fast to build something that's 90% of a solution. The problem is that the last 10% of building something is usually the hard part which really matters, and with a black box at the center of the product, it feels much more difficult to me to nail that remaining 10%. With vibecheck, most of the time the results to my queries are great; some percentage of the time they aren't. Closing that gap with gen AI feels much more fickle to me than a normal engineering problem. It could be that I'm unfamiliar with it, but I also wonder if some classes of generative AI based products are just doomed to mediocrity as a result.
2. On the other hand, I think the discomfort I feel with having an unpredictable black box at the center of a product which can fail in very creative ways some percentage of the time might actually be a competitive advantage for startups. I think Wirecutter, especially branded as NY Times, would have a lot harder time tolerating that unpredictability and unreliability than an app like this (or an actual startup) which can set that product expectation with users to begin with. It might be that the current problems with generative AI are actually the things that create an innovator's dilemma and give startups an advantage to slip under the incumbents. It seems like the ideal spot right now would an app where 90% "done" is a mostly great experience, but the failings are still somehow not stomach-able by incumbents.
3. I do not understand how the economics of LLMs pencil out. When I look at the per concurrent user costs associated with inference, they seem orders of magnitude higher than per concurrent user costs of previous internet technologies. It seems to me that if previous apps like webmail, messengers, etc had costs as high, they would not have been viable products. This is something I want to learn more about.
English
