Cole Hoffer

Synthetic Users are bringing A/B testing upstream into product discovery. Before a single line of code is written. We launched Synthetic Users on Tuesday. Hundreds of agent sessions are already running. And a clear pattern is emerging: builders are running competing ideas and flows through agents simultaneously to see what sticks. This changes the cycle: 💠 Old: Discover → Build → Launch → A/B test → Learn 💠 New: Discover → Experiment with agents → Build the winner → Ship with conviction With experimentation moving upstream, teams can: - Testing 3 onboarding approaches against each other during ideation, not after production build - Pitting value propositions against simulated personas before committing to a direction - Killing losing variants in discovery, not after weeks of engineering With prototypes, you can now answer "Which version performs better?" before you've spent a single sprint building any of them. Try it out today: build.reforge.com

Isn't this the pommel horse guy

Out of all the things we've shipped recently, this one is my favorite. Instant feedback from any type of user... → Create a persona that matches your real users and buyers. → An AI agent takes on that persona. → Point it at any url or prototype. → Get feedback in minutes. This is what the @reforge team built and launched today with Synthetic Users. It's way more than bug reports, but actual strategic product feedback. → A First Timer notices your onboarding language is confusing → A Skeptical Buyer questions whether your pricing page builds enough trust → A Power User finds friction in your core workflow that your team stopped noticing months ago Every finding links to a session replay so you can see exactly where things broke down. This isn't a replacement for talking to real users. It's an acceleration layer. Run synthetic studies to catch issues fast, then reserve human research for the decisions that matter most. This is critical as AI accelerates the amount of product teams are shipping. If you don't also develop ways to validate faster, you'll just ship tons of product debt. Lots of features that few customers use but create massive costs.

Synthetic Users changes how you talk to your customers. We built it so any team can run user research in minutes, not weeks. No recruiting. No scheduling. No transcribing. Try it out today: reforge.com/build/syntheti…

@chunonline @GeminiApp @suno Wow the skyline of Dublin, CA has really grown this year!

one shot (almost?) dumped our cow cat to our backyard. can use WASD to walk and space to jump. so cool. - world model: genie 3 (@GeminiApp) - music: @suno

@hadasie @chunonline @bbalfour @bhalligan do not slander Anna's Taqueria

@chunonline @bbalfour @bhalligan no good tacos

I've spent 50% of my tech career in Boston and 50% in SV. My personal take on the problem (cc @bhalligan) Boston has a classic positioning/differentiation problem. It's good at multiple things, it's great at nothing. It doesn't "spike" on anything for ambitious entrepreneurs and talent. Yes, Boston has good talent coming out of the local universities, interesting surrounding areas (cape, etc), and more. But SV, NYC, and Austin/Texas all have great elements to their value props for ambitious entrepreneurs. That's what attracts and retains. As a result, most (not all) of the people who stay in Boston are those who choose it for family. This creates an adverse selection problem. Most (not all) ambitious talent will trade off family for career and move. My suggestion is that Boston needs to: 1. Choose an ICP 2. Find something specific/narrow to be great at 3. Go all in. Improving the nightlife from terrible to good is not going to cut it. It will remain average at everything, great for nothing. Doing this at a company level is hard. Doing this at a city level...huge undertaking. Miami was on the right track, but they chose the wrong horse (Crypto). We had these exact convos at almost every Boston tech event 20 years ago when I started my career there. Nothing has substantially changed. 80% of my friends who were entrepreneurs when I was there have since moved to either NYC or SV. I say that with deep love. Boston, HubSpot and the many people I met there had a huge impact on my career. I hope something gets catalyzed!

Nothing triggers me more when eval tools promote generic metrics (i.e. Affirmation, Brevity, Levenshtein) as way to make "evals easy" In reality, this is extremely poor data literacy sold as "best practices", in the same way that sugary cereal is marketed as healthy. The only thing that generic metrics do is waste your time and burn tremendous engineering cycles by having you chase vanity metrics. What works is looking at your data and define metrics tailored to failure modes you actually observe. When an eval tool promotes these front and center, I run in the other direction. BTW I have no idea WTF affirmation score even means. This is something I saw in an IRL advertisement. Stuffing a dashboard with a bunch of random metrics is a guaranteed way to waste everyone's time. Don't do it. I wrote more about this here: #q-should-i-use-ready-to-use-evaluation-metrics" target="_blank" rel="nofollow noopener">hamel.dev/blog/posts/eva… Also yes, this is a subtweet 🤣

@benmkramer @chunonline four one

@chunonline six seven

I’ve been mentally preparing myself to back to the ultra speed / future for this China trip especially since the last time I was back was 6/7 years ago. But damn, some things still hit hard. In two minutes, they scanned my fingerprints, checked my temperature, and collected everything. And there are even people dedicated to flipping luggage handles in the right direction on belt for faster pickup. 😵‍💫🏃‍♀️

@JohnPhamous Breakdown on input/output tokens??

tabular numbers

@JohnPhamous Iconic

use the serifed "I" for "AI"

@chunonline @benmkramer when ship?

Scalable product, scalable partnership. 🤝@benmkramer

📈‼️✊🏾

Sapiens for guys who love charts

@JohnPhamous What if your labels are 4x as long???

chart for showing changes between 2 different time periods - uses css subgrid

Pay attention to this: @bbalfour (founder of @reforge, former VP of Growth at @HubSpot) has studied every major platform shift—from Facebook to Apple to Google—and he’s spotted a pattern that’s about to repeat with ChatGPT. Brian predicts that within the next 6 months, ChatGPT will become the next major distribution platform. In our conversation, Brian shares: 🔸 The 4-step cycle every platform follows (and why ChatGPT just entered step 2) 🔸 Why ChatGPT’s platform launch could be bigger than Facebook’s early platform 🔸 The exact signals that ChatGPT will launch a third-party platform within six months 🔸 Why you have six months (not years) to make your platform bet 🔸 Why companies that don’t integrate with ChatGPT will lose to competitors that do 🔸 Why so few companies are actually doing what they need to be doing right now Listen now 👇 • YouTube: youtu.be/cX4cL6B-_aU • Spotify: open.spotify.com/episode/0aSnaj… • Apple: podcasts.apple.com/us/podcast/why… Thank you to our wonderful sponsors for supporting the podcast: 🏆 @DeveloperXM — The developer intelligence platform designed by leading researchers: getdx.com/lenny 🏆 @Basecamp — The famously straightforward project management system from 37signals: basecamp.com/lenny 🏆 @MiroHQ — collaborative visual platform where your best work comes to life: miro.com/lenny

@jarrensj Stallion

3k club github.com/jarrensj

@JohnPhamous BOLD

@choffer1120 Filter them out so they never make it here

bar list - hovering over a row de-emphasizes other rows - no dead gaps between rows to prevent flickering

@chunonline @iamharaldur @Altimor Explain in Labubu terms

@iamharaldur @Altimor Suppose someone works from age 20 to 50 … theoretically, they could only produce 20 Edit buttons. 😲