VIRTUAL BEINGS

A Virtual Pet Game like Nintendogs, fun but also has a mode that is much more sim like and gives you a grade after 1 month, so that way, any kid asking their parents for a pet must take care of the virtual one and get a A ranking to prove they are responsible enough for a pet 4/x

Before we have a million robots in the physical world, we will first see a billion embodied agents in virtual worlds. Gaming is the second major area I'm dedicated to in 2024. AI and Gaming are born for each other, and their happy marriage is just getting started. On one hand, open-ended games provide a "primordial soup" for generalist AI to emerge. An agent's capabilities are upper-bounded by the complexity of the world it lives in. Minecraft is a prime example. In 2023, we saw an explosion of new algorithms enabled by Minecraft. To name a few: - MineCLIP (from our team's open-source framework MineDojo): learn reward functions by watching 100,000s of YouTube gameplay videos. - VPT (OpenAI): imitate behavior by pseudo-labeling actions from YouTube. - Voyager: in-context lifelong learning with an explicit Skill Library of code. - STEVE-1: guide actor to follow commands in MineCLIP's latent space, inspired by DALLE-2 - And many more: DEPS, Jarvis-1, DreamerV3 ... Besides Minecraft, there are a lot more games that require extremely advanced perception, agility, exploration, reasoning, and planning. We are just starting to scratch the surface. I believe games (and simulation in general) will provide the next trillion high-quality tokens to train our foundation models. What's cool is that these tokens are actively selected by the agent itself through exploration. It can choose to experiment with things that maximally reduce its internal uncertainties - kind of like how human curiosity works. On the other hand, AI will lead to a paradigm shift in the Gaming industry. This year, we see a surge of community interest in Stanford Smallville, where 25 AI agents inhabit a digital town. They go to work, gossip, organize socials, make new friends, and even fall in love. It's an exciting experiment, but we still have not felt any impact on the real games out there. This is because our LLMs are too boring and too expensive. If you take a look at Smallville's chat log, you will find that the conversations are not fun at all. No parents talk to their kids in such polite manner. The unit economy of deploying so many agents also does not make sense at scale. That being said, I believe 2024 is an inflection point. The Digital Westworld is coming, and will transform the industry once and for all. Games will feel truly alive. The characters will interact with humans and each other, form relationships, take consistent actions over their lifetime, and react in human-like ways. Each game will have infinite replay value, and each player will have unique and tailored experience.