Jake Maymar

Everything you love about generative models — now powered by real physics! Announcing the Genesis project — after a 24-month large-scale research collaboration involving over 20 research labs — a generative physics engine able to generate 4D dynamical worlds powered by a physics simulation platform designed for general-purpose robotics and physical AI applications. Genesis's physics engine is developed in pure Python, while being 10-80x faster than existing GPU-accelerated stacks like Isaac Gym and MJX. It delivers a simulation speed ~430,000 faster than in real-time, and takes only 26 seconds to train a robotic locomotion policy transferrable to the real world on a single RTX4090 (see tutorial: genesis-world.readthedocs.io/en/latest/user…). The Genesis physics engine and simulation platform is fully open source at github.com/Genesis-Embodi…. We'll gradually roll out access to our generative framework in the near future. Genesis implements a unified simulation framework all from scratch, integrating a wide spectrum of state-of-the-art physics solvers, allowing simulation of the whole physical world in a virtual realm with the highest realism. We aim to build a universal data engine that leverages an upper-level generative framework to autonomously create physical worlds, together with various modes of data, including environments, camera motions, robotic task proposals, reward functions, robot policies, character motions, fully interactive 3D scenes, open-world articulated assets, and more, aiming towards fully automated data generation for robotics, physical AI and other applications. Open Source Code: github.com/Genesis-Embodi… Project webpage: genesis-embodied-ai.github.io Documentation: genesis-world.readthedocs.io 1/n

@joaomdmoura Glad to hear you are almost back - focus on getting better first - sometimes a brief pause leads to an incredible spark of innovation

COVID update: feeling way better! not 100% but solid 85%! Getting more back into stuff now 😎👉👉 Thanks for bearing with me and my even slower answer rate this week so far

@JukkaUrsin @Apple Wow! Amazing work and amazing insights! Congratulations to you and your team--this is so good I saved it to reference later.

@nathangitter Congrats!

It's been an honor to work with some of the best designers in the world on the Apple Design Team to create something totally new: Apple Vision Pro #WWDC23

@merittthomas Congratulations! 👋👋👏 love that!

This has been a dream of mine since I was a kid—and it finally happened!! I am SO unbelievably proud to have contributed to this. Don’t ever give up on your dreams. ☁️🌈✨ #AppleVision #visionOS

@malonso Brilliant work--Congrats to you and your team!

Vision Pro is out. Super proud of the team and the hard work behind it!

@malonso This makes a great system even easier to use--nice work! Congrats!

Universal Control is out. Just move your cursor to any Mac or iPad nearby. You can even drag and drop files between them. Learn more here: support.apple.com/en-us/HT212757

@jonathan_ravasz Congratulations--excited to see the this new evolution to intuitive interactions!

I am thrilled to share what we've been working on: Apple's new personal computing platform, Vision Pro, is here. It's been a long journey, but I feel fortunate to share it with some of the most talented and dedicated people I've ever worked with. #WWDC23 apple.com/apple-vision-p…

@ddiakopoulos Congratulations to you and your teams--brilliant work! Can't wait to see all of the amazing experiences to come

Quite proud of the announce today and all the people behind the Vision Pro. Remember it’s a team effort and for every Apple contributor on Twitter, there are countless more uncredited individuals working on some of the hardest science & engineering problems on the block. 👏👏👏

@sterlingcrispin Congratulations--looking forward to experiencing all of these clever systems!

I spent 10% of my life contributing to the development of the #VisionPro while I worked at Apple as a Neurotechnology Prototyping Researcher in the Technology Development Group. It’s the longest I’ve ever worked on a single effort. I’m proud and relieved that it’s finally announced. I’ve been working on AR and VR for ten years, and in many ways, this is a culmination of the whole industry into a single product. I’m thankful I helped make it real, and I’m open to consulting and taking calls if you’re looking to enter the space or refine your strategy. The work I did supported the foundational development of Vision Pro, the mindfulness experiences, ▇▇▇▇▇▇ products, and also more ambitious moonshot research with neurotechnology. Like, predicting you’ll click on something before you do, basically mind reading. I was there for 3.5 years and left at the end of 2021, so I’m excited to experience how the last two years brought everything together. I’m really curious what made the cut and what will be released later on. Specifically, I’m proud of contributing to the initial vision, strategy and direction of the ▇▇▇▇▇▇ program for Vision Pro. The work I did on a small team helped green light that product category, and I think it could have significant global impact one day. The large majority of work I did at Apple is under NDA, and was spread across a wide range of topics and approaches. But a few things have become public through patents which I can cite and paraphrase below. Generally as a whole, a lot of the work I did involved detecting the mental state of users based on data from their body and brain when they were in immersive experiences. So, a user is in a mixed reality or virtual reality experience, and AI models are trying to predict if you are feeling curious, mind wandering, scared, paying attention, remembering a past experience, or some other cognitive state. And these may be inferred through measurements like eye tracking, electrical activity in the brain, heart beats and rhythms, muscle activity, blood density in the brain, blood pressure, skin conductance etc. There were a lot of tricks involved to make specific predictions possible, which the handful of patents I’m named on go into detail about. One of the coolest results involved predicting a user was going to click on something before they actually did. That was a ton of work and something I’m proud of. Your pupil reacts before you click in part because you expect something will happen after you click. So you can create biofeedback with a user's brain by monitoring their eye behavior, and redesigning the UI in real time to create more of this anticipatory pupil response. It’s a crude brain computer interface via the eyes, but very cool. And I’d take that over invasive brain surgery any day. Other tricks to infer cognitive state involved quickly flashing visuals or sounds to a user in ways they may not perceive, and then measuring their reaction to it. Another patent goes into details about using machine learning and signals from the body and brain to predict how focused, or relaxed you are, or how well you are learning. And then updating virtual environments to enhance those states. So, imagine an adaptive immersive environment that helps you learn, or work, or relax by changing what you’re seeing and hearing in the background. All of these details are publicly available in patents, and were carefully written to not leak anything. There was a ton of other stuff I was involved with, and hopefully more of it will see the light of day eventually. A lot of people have waited a long time for this product. But it’s still one step forward on the road to VR. And it’s going to take until the end of this decade for the industry to fully catch up to the grand vision for this tech. Again, I’m open to consulting work and taking calls if your business is looking to enter the space or refine your strategy. Mostly, I’m proud and relieved this has finally been announced. It’s been over five years since I started working on this, and I spent a significant portion of my life on it, as did an army of other designers and engineers. I hope the whole is greater than the sum of the parts and Vision Pro blows your mind.

@benz145 Hi everyone--been creating AR/VR/XR since 2018--work/ed with fortune 50 clients--now focused on applying AI to XR.

NETWORKING THREAD 🚨 Apple in XR is going to mean lots of new people joining the conversation. If you’re in the industry and want to connect with new people coming in (or vice versa), drop a line here to introduce yourself so peole can follow you. RT to spread this list around!

@Scobleizer @IrenaCronin Looking forward to it!

This post will save you $75 and prepare you for Apple's "Big Bang" that arrives on Monday. Join us Sunday for a Twitter Space where @IrenaCronin and I will tell us what we know about the Apple Headset and the AI Mesh infrastructure it has built. The Twitter Space will be Sunday at 5 p.m. Pacific Time. You can sign up here: x.com/i/spaces/1lygb… Thank you AI Artist @RobertpHaslam who has been sending me all these videos to get me hyped up for Monday. Here's another one. Some things we will cover: 1. Apple's advantages: -Stores -Supply chain -Ecosystem -Spatial Audio -Trust -AI Infrastructure -TV service -Movie studio and service -Music studio and service -Fitness studio and service -AI Mesh. 2. Why it cost $40 billion. 3. What we expect from the new Siri. 4. The magic of immersion. Why the headset naysayers are wrong. 5. What we expect venture capitalists to invest in from here. Irena and I will talk for a while, maybe even an hour, then we will open it up for the community to tell us what their dreams for the future of computing are and what they are hoping to see on Monday. Then, on Monday, we'll do it again, starting another Twitter Space right after the Apple keynote ends. Regarding the $75, on Sunday we'll send our long paper about the new Siri and its impact to anyone for free. To get a copy, just leave a note here and we'll send it on Sunday.

Combatting climate change means reducing global waste. In this article, I discuss how AI can help discover solutions for sustainability by repurposing old/broken items. #AIforSustainability #ReduceWaste #ClimateChange lifewire.com/ai-contributes… via @lifewiretech

Our VP of Innovation @jmaymar recently spoke with @NTDNews about AI-generated images that look incredibly real. He suggested that we will need AI tools to, "fact-check" images, and determine if they're generated or not. ntd.com/disruption-con… #AI #GenerativeAI #AIArt $VRAR

Great conversation with @natrubio__ about an interesting Microsoft patent that is going to make AI almost frictionless: open.substack.com/pub/patentdrop… #innovation #AI #AR #VR

"Like everything, if the art is viewed as valuable, people will try to steal it." -@jmaymar, in an interview with @lifewiretech, spoke about how @nvidia is launching software that helps AI models comply with copyright laws. lifewire.com/how-new-tech-c… #AI #AIArt #innovation $VRAR

@NVIDIAGeForce no question #BeyondFast

To celebrate the recent launch, we’re giving away a NVIDIA GeForce RTX 4090 with a custom Atomic Heart backplate 👀 Entering is easy: 🟢 Like this post 🟢 Comment #BeyondFast Terms apply: nvidia.com/en-us/geforce/…

@SpatialCanvas @WalkaboutMG @djcarson This is brilliant and creates really great results--thanks for posting this!

@Scobleizer @AppleHolodeck sooner than that :)

The AI we are seeing Microsoft integrate into Bing and other things (OpenAI's ChatGPT) is just the start of exponential change. At Christmas we will have virtual beings we talk to. In a 3D environment we also talk to. The @AppleHolodeck running on AI Mesh. Disruption ahead!

We're excited to reveal that we've integrated conversational AI into our VR platform. Enabling users to communicate with virtual characters in a conversational manner is a significant development. #AI #VR #metaverse #GPT3 #ChatGPT $VRAR