Robert Lukoszko

Concept of LLM Extended OS I DON'T believe we will be able to jump to LLM first Operating Systems straight away. Self-operating systems are just like ChatGPT. The need reinforcement learning with human feedback (RLHF). They need it to LEARN how to operate effectively. How can LLMs collect this feedback? You SHOW the user an interface of LLM operating. When LLM makes a mistake -> give negative feedback. What is the best way to do it? Definitely not some server running on the cloud which you can't see except for logs. For the LLM to learn MOST from humans, it needs to operate on systems where humans can provide PROPER feedback. What are the examples of this? the systems we using RIGHT NOW. I picked MacOS. Lets deep dive into how I see it. I called it LLM extended OS First, it has A LOT in common with what @karpathy has shown It has access to terminal, camera, audio, and browsing. The main difference is that it has APPS people use right now. Like: - VS Code - Finder - TODO lists Just think about it. If LLM has imperfect attention. Just like humans do, why not give it a FULL interface WE use including not only - Calendar – allow LLM to schedule tasks - Reminders – set tasks it NEEDS to do - Music – no, just kidding - Discord – allow it to EFFICIENTLY communicate with OTHER LLMS, and send them tasks while being ignored if they are not interested 😂 Another major difference is Graphical user interface (GUI). Yes, VISION is too powerful to NOT make use of it. @josh_bickett just made a great case with self self-operating computer. Which proves the point. One thing I think EVERYONE has missed is something that has been lying here for decades. Accessibility/VoiceOver Some people have trouble SEEING stuff. Thanks to apple, those people can STILL use the computer just like YOU DO. Which is mind-blowing right? And think about it, visually impaired people can navigate the computer and almost FULLY operate just like you can. I tried it, its pure magic. The question is, if those people use only, RAW PRONOUNCED TEXT to navigate, will LLM not be able to do it? You know the answer And there is more to write about it... I'll keep the progress update here @Karmedge Let me know what you think I would appreciate any feedback on this concept and potentially wrong assumptions I made, feel free to correct me. I hold your opinions in high regard on this topic: all @openai team especially: @sama @gdb Open interpreter: @hellokillian Otherside AI: @mattshumer_, @josh AdeptLabs @AdeptAILabs Would appreciate your thoughts: @yushilmi , @raphaelmosaic , @johannes_hage @xathis @willdepue The winner in this race must be humanity, not capitalism.

we're making @blocks smaller today. here's my note to the company. #### today we're making one of the hardest decisions in the history of our company: we're reducing our organization by nearly half, from over 10,000 people to just under 6,000. that means over 4,000 of you are being asked to leave or entering into consultation. i'll be straight about what's happening, why, and what it means for everyone. first off, if you're one of the people affected, you'll receive your salary for 20 weeks + 1 week per year of tenure, equity vested through the end of may, 6 months of health care, your corporate devices, and $5,000 to put toward whatever you need to help you in this transition (if you’re outside the U.S. you’ll receive similar support but exact details are going to vary based on local requirements). i want you to know that before anything else. everyone will be notified today, whether you're being asked to leave, entering consultation, or asked to stay. we're not making this decision because we're in trouble. our business is strong. gross profit continues to grow, we continue to serve more and more customers, and profitability is improving. but something has changed. we're already seeing that the intelligence tools we’re creating and using, paired with smaller and flatter teams, are enabling a new way of working which fundamentally changes what it means to build and run a company. and that's accelerating rapidly. i had two options: cut gradually over months or years as this shift plays out, or be honest about where we are and act on it now. i chose the latter. repeated rounds of cuts are destructive to morale, to focus, and to the trust that customers and shareholders place in our ability to lead. i'd rather take a hard, clear action now and build from a position we believe in than manage a slow reduction of people toward the same outcome. a smaller company also gives us the space to grow our business the right way, on our own terms, instead of constantly reacting to market pressures. a decision at this scale carries risk. but so does standing still. we've done a full review to determine the roles and people we require to reliably grow the business from here, and we've pressure-tested those decisions from multiple angles. i accept that we may have gotten some of them wrong, and we've built in flexibility to account for that, and do the right thing for our customers. we're not going to just disappear people from slack and email and pretend they were never here. communication channels will stay open through thursday evening (pacific) so everyone can say goodbye properly, and share whatever you wish. i'll also be hosting a live video session to thank everyone at 3:35pm pacific. i know doing it this way might feel awkward. i'd rather it feel awkward and human than efficient and cold. to those of you leaving…i’m grateful for you, and i’m sorry to put you through this. you built what this company is today. that's a fact that i'll honor forever. this decision is not a reflection of what you contributed. you will be a great contributor to any organization going forward. to those staying…i made this decision, and i'll own it. what i'm asking of you is to build with me. we're going to build this company with intelligence at the core of everything we do. how we work, how we create, how we serve our customers. our customers will feel this shift too, and we're going to help them navigate it: towards a future where they can build their own features directly, composed of our capabilities and served through our interfaces. that's what i'm focused on now. expect a note from me tomorrow. jack

Great men of history had little to no introspection. The personality that builds empires is not the same personality that sits around quietly questioning itself. @pmarca and I discuss what we both noticed but no one talks about: David: You don't have any levels of introspection? Marc: Yes, zero. As little as possible. David: Why? Marc: Move forward. Go! I found people who dwell in the past get stuck in the past. It's a real problem and it's a problem at work and it's a problem at home. David: So I've read 400 biographies of history’s greatest entrepreneurs and someone asked me what the most surprising thing I’ve learned from this was [and I answered] they have little or zero introspection. Sam Walton didn't wake up thinking about his internal self. He just woke up and was like: I like building Walmart. I'm going to keep building Walmart. I'm going to make more Walmarts. And he just kept doing it over and over again. Marc: If you go back 400 years ago it never would've occurred to anybody to be introspective. All of the modern conceptions around introspection and therapy, and all the things that kind of result from that are, a kind of a manufacture of the 1910s, 1920s. Great men of history didn't sit around doing this stuff. The individual runs and does all these things and builds things and builds empires and builds companies and builds technology. And then this kind of this kind of guilt based whammy kind of showed up from Europe. A lot of it from Vienna in 1910, 1920s, Freud and all that entire movement. And kind of turned all that inward and basically said, okay, now we need to basically second guess the individual. We need to criticize the individual. The individual needs to self criticize. The individual needs to feel guilt, needs to look backwards, needs to dwell in the past. It never resonated with me.

Atoms. atoms.co/vision