Matt Dupree

Been thinking a lot about UIs lately while doing customer research for @join_atlas_ai and I've concluded that UIs almost always get worse over time. Companies may hire better designers or PMs to avoid this, but their UIs will inevitably become harder to use anyway. Here's why.🧵

We built a bug finder. We're finding serious, "let's fix that right now" issues in every codebase we run it on. Introducing Detail!

@sarahcat21 is that an inversion? seems like vanilla conway's law

Org design is probably the most important lever in impacting model architectures and research trajectories; we are seeing this weird inversion of Conway’s law wherein models reflect the structure of teams

@Microsoft, can you please demote the person who decided to replace the missing npm dependency quickfix powered by static analysis in @code with a copilot-powered one? It's slower, less accurate, and frankly, an absurd use of LLMs. Thank you.

Today's your lucky day, folks. @join_atlas_ai is winding down, which means @JoshShroyer -- an awesome engineer, person, and friend -- is on the market. I met Josh 8 years ago while working at another startup. It was immediately clear to me that he was a brilliant engineer. Software was a craft that he cared deeply about and he studied with a fervor that I've haven't seen in anyone else in my decade long career. Josh's humility was equally apparent when we first met. Although he knew more than his peers, he was cautious in his thinking and patient with how he communicated. As I got to know Josh better, I learned that in addition to being humble and brilliant, he is also a ridiculously kind person. He really cares about people. He wants to do right by the users and customers he builds for. After we parted ways, I knew I wanted to work with Josh again, so I sought him out after raising capital for my new venture. He has continued to impress me with his ability to quickly pick up new technical skills. He went from iOS dev to fully productive Fullstack web and AI dev in less than a month. He pushes himself hard, and it shows. Although he works extremely hard, he has remained kind, patient, and calm through all the ups, downs, and twists of startup life. I'm grateful that I had the opportunity to work with him twice, and I'm especially grateful that someone so talented and kind put up with my first crack at being a CEO of a startup. This isn't my last attempt at being a CEO, so if you hire Josh, treat him like royalty because I'm definitely going to try convince him to work with me again!

15 minutes left on this poll! I want your vote!

Trying to get a sense of developer attitudes about data privacy. Please pick the option that best fills in the blanks in the sentence below: I am building a (B2B | B2C) product and am (happy | unhappy) with our privacy practices. e.g., pick choice 1 if you're working on a B2B product and are unhappy with your product's data privacy practices.

We've reached the point in platform maturity where businesses need to concern themselves with building for a niche, or delivering features @Apple won't go after, else risk feature absorption @1Password is a good example, growing out of Apple's ecosystem. techcrunch.com/2024/06/18/ios…

@david_laprade and I worked that AI-powered UI/UX startups are doomed at an offsite last quarter. Very glad we didn't try to play in the AI-powered UI/UX space.

PSA: gpt4o's behavior is chaotic and unpredictable when temp is set just above the default (of 1.0). [0] Past temp of 1.1 we're seeing nonsense responses in multiple languages and character sets [0] #chat-create-temperature" target="_blank" rel="nofollow noopener">platform.openai.com/docs/api-refer…

There's consensus emerging among VCs that #LLMs will make software companies look more like services businesses because software will own a task end-to-end instead of merely enabling a human to work on a task faster. I've seen this from partners at @NFX , @FoundationCap, and @OpenViewVenture. The shorthand they like for this "LLMs enable service as a software." I'm betting the company (@join_atlas_ai) that they're wrong. I'm trying to work with GTM/PLG teams instead of replacing them, not because I'm particularly precious about displacing jobs but because LLMs just aren't good enough to do their jobs and that won't change anytime soon. LLM optimists point to LLM's effectiveness in domains like coding, prospecting, and SEO content gen, but the wins here are limited and will be short-lived. Coding is unlike any other domain in that the content (code) generated by LLMs can often be unambiguously verified as correct. The code either runs or it doesn't. It's a mistake to generalize from the coding domain. The effectiveness of LLM-powered SEO content gen and prospecting, moreover, is primarily a function of the competitiveness of the space in which these solutions operate. Once the competition shows up, no one is going to care about 80th percentile performance on cold outreach or SEO content. Humans will be back in the game. E.g., Dave Rigotti banned GPT for content gen at @Inflection_Io this week. I already use "better than ChatGPT" as a filter for content I want to write. LLM optimists often think that there's a clear path to making these models much better. There isn't. Pay less attention to people who have a financial interest in having you believe otherwise. Don't take my word for it either. I'm just incentivized in the opposite direction. @GaryMarcus is too, but he's worth paying attention to.

You're right that LLMs aren't that smart but wrong about the relevance to safety work. @GaryMarcus has already pointed out that LLMs don't need to be particularly smart to cause harm. Instagram's algorithm that feeds pictures of children to pedophiles (@nytimes just ran an article on this) runs on gold old fashioned machine learning and it's plenty harmful. The key question we should ask is whether the invisible hand will allocate sufficient resources to AI safety and to the safe development of new technologies in general. I'm doubtful it will. AI-powered harm will often be a negative externality --- just like the Instagram pedophile example above --- and even the most pro-market economists agree that markets aren't good at handling externalities. @janleike's resignation from OpenAI actually shows how the market is broken here. The invisible hand didn't give Leike enough resources to pursue safety seriously so he left. @sama is unhappy with his departure too. LLMs will continue to develop into increasingly powerful tech that will hurt folks. Everyone loses. Capitalism is great, except when it's not.

It seems to me that before "urgently figuring out how to control AI systems much smarter than us" we need to have the beginning of a hint of a design for a system smarter than a house cat. Such a sense of urgency reveals an extremely distorted view of reality. No wonder the more based members of the organization seeked to marginalize the superalignment group. It's as if someone had said in 1925 "we urgently need to figure out how to control aircrafts that can transport hundreds of passengers at near the speed of the sound over the oceans." It would have been difficult to make long-haul passenger jets safe before the turbojet was invented and before any aircraft had crossed the atlantic non-stop. Yet, we can now fly halfway around the world on twin-engine jets in complete safety. It didn't require some sort of magical recipe for safety. It took decades of careful engineering and iterative refinements. The process will be similar for intelligent systems. It will take years for them to get as smart as cats, and more years to get as smart as humans, let alone smarter (don't confuse the superhuman knowledge accumulation and retrieval abilities of current LLMs with actual intelligence). It will take years for them to be deployed and fine-tuned for efficiency and safety as they are made smarter and smarter.

Anyone else notice that #LLaMA3 performs differently on @GroqInc's API than it does locally? I’m running llama3-8b-8192 (instruct) via @ollama on an M1 mac and seeing consistently better accuracy than the same model on groq for the same tasks (mostly 0-shot text classification)

Theory for B2B SaaS: the smarter your customers seem, the more likely you're nearing product-market-fit

@stripe cc @jeff_weinstein low hanging UI improvement that would help new dashboard users: semantic search over your UI would match queries like "add user to account" to the page the user intends to go to. You shouldn't need to know the magic word "member" to get there

@saksenarjun @forumventures @amruthagujjar @ycombinator lu.ma/snhm17gb

Met @saksenarjun -- a fellow @forumventures founder and former growth PM at Adobe -- and @amruthagujjar -- @ycombinator and x-Meta engineer -- this past week at #PLGTM. We're all working #LLM-enabled tech for GTM and #PLG teams and thought it would be fun to invite folks around SOMA to have breakfast this Thursday and hear some lightning talks about what we're building and why. Registration link is in the comments. We likely have space for one or two more presenters, so if you know someone else working on LLM-tech for GTM teams, tag them in the comments!

I spent the last few weeks reading papers on LLM text classification Some practical takeaways below 🧵(1/9)

@emollick The paper says the exact opposite of this?

@theallinpod @DavidSacks @chamath @friedberg @Jason