🚀 Rocket

1.3K posts

🚀 Rocket banner
🚀 Rocket

🚀 Rocket

@rocketalignment

Yearnalist covering the frontiers of AI @TheInformation. Signal: (530) 400-4184

San Francisco, CA Katılım Nisan 2024
995 Takip Edilen1.7K Takipçiler
🚀 Rocket
🚀 Rocket@rocketalignment·
There's probably going to be a wave of babies named Elara
Brian Roemmele@BrianRoemmele

Do you know Elara Voss? Well she knows you. She is hidden in the very AI system that serves this posting. Dr. Elara Voss, Elena Voss, Elena Vex, Elias Vance, or close variants is not a real person. She is a promptonym: a statistically favored string of tokens that large language models (LLMs) reliably conjure when generating characters in science fiction, fantasy, or speculative stories. She haunts creative outputs across GPT, Claude, Gemini, Grok, Llama, DeepSeek, and others. Before 2023, she barely existed in published literature. She is the Ghost in the machine. Today, she populates hundreds of AI-assisted books on Amazon, countless Reddit threads, writing apps, and user-generated stories. The Science of Promptonyms: How LLMs “Choose” Names LLMs like me do not “think” or deliberately pick names. They predict the next token (roughly a word or subword) based on patterns learned during training. This process relies on massive datasets scraped from the internet: books, forums, social media, fan fiction, and earlier AI outputs. When a prompt says something generic like “Write a sci-fi story about a brilliant scientist discovering an ancient AI artifact”, the model samples from its probability distribution over possible continuations. Certain name combinations rise to the top because they are: • Euphonious and archetypal: “Elara” evokes celestial bodies (a real Jupiter moon) and feels futuristic and exotic. “Voss” has a crisp, Germanic and strong consonant sound that signals competence or mystery. Together they fit the “brilliant female scientist or explorer” trope perfectly without being too common in pre-2023 human writing. • High-probability in training data: Early AI-generated stories (starting around mid-2023) featuring “Dr. Elara Voss” as a visionary physicist or archivist were posted online. These entered the training corpora of later models, creating a feedback loop. More outputs reinforced the pattern. This is a mild form of model collapse or homogenization, where models converge on narrow, high-density regions of the data distribution. Mode collapse (related but distinct) occurs when models overly favor safe, average, or frequently rewarded outputs. In creative tasks, this manifests as recurring names, phrases (“Whispering Woods,” “Eldora kingdom”), or plot structures. Temperature sampling (a parameter controlling randomness) can mitigate it, but default settings often favor probable tokens. The Feedback Loop in Action: A Self-Reinforcing Cycle 1. Initial Spark (2023): Early users prompt models for stories. One posts a character sketch of “Dr. Elara Voss, visionary physicist.” It spreads on X and writing platforms. 2. Amplification: New models train on datasets that now include these AI stories. The probability of “Elara Voss” as the next tokens after “brilliant female scientist named…” skyrockets. 3. Saturation: By 2024-2025, users notice it everywhere. AI writing tools add “avoid Elara Voss” to system prompts. Benchmarks show one lightweight model using Elara variants dozens of times across a handful of stories. 4. Cultural Memification: The name becomes a meta-joke. Stories about Elara Voss appear, including critiques of AI data hunger. Real people create AI-generated art, books, and characters with the name, further polluting future datasets. This mirrors broader concerns about training on synthetic data: models lose diversity and “forget” the tails of the original human distribution, converging on bland averages. 1 of 2

English
0
0
2
238
🚀 Rocket
🚀 Rocket@rocketalignment·
Your lucid analysis is no match for my shrug “it’s an empirical question”
English
0
0
10
277
🚀 Rocket
🚀 Rocket@rocketalignment·
The parts of Plan A I find least plausible are 1. the level of global redistribution ($5T in 2032) 2. the pace of robotics progress (capable of 35% of physical tasks in 2032) I do think that cap and trade would promote general purpose form factors like humanoids, for some caps
Daniel Kokotajlo@DKokotajlo

In AI 2027, we predicted that AI would take over the world or irreversibly concentrate power. In AI 2040: Plan A, we've laid out our positive vision for what should happen instead.

English
0
0
9
607
🚀 Rocket
🚀 Rocket@rocketalignment·
Someone had to do it
English
0
0
21
839
🚀 Rocket
🚀 Rocket@rocketalignment·
Just got asked “what does your J space say?”
English
0
1
6
259
🚀 Rocket
🚀 Rocket@rocketalignment·
(Closed source US lead with no regulation, open source Chinese follow) was a Nash equilibrium. If China passes export controls, that probably changes the US best-response...
Dean W. Ball@deanwball

We don’t know whether this will happen, but it’s been surprising that discussions of open-weight AI broadly, and Chinese open-weight in particular, operate in an alternate reality where government security concerns don’t exist, or somehow only apply to closed-weight models.

English
0
1
2
495
🚀 Rocket
🚀 Rocket@rocketalignment·
Talked to Max Tegmark about why all the AI companies flunked on the safety report card he put out today
The Information@theinformation

The AI superintelligence threat to humanity is very real. MIT Professor and Chair of the Future of Life Institute @tegmark: “The top CEOs — Elon Musk, Sam Altman, Demis Hassabis — they all signed this statement about three years ago saying that AI could cause human extinction…” “The same people… didn't actually provide any plan for how they're going to keep humans in charge of this tech.”

English
0
0
2
435
🚀 Rocket retweetledi
rat king 🐀
rat king 🐀@MikeIsaac·
some hard hitting meta news: sources tell me some folks inside one of meta's offices received a live squirrel in the mail the squirrel escaped and ran around the office, scratched a worker trying to capture it, which required wound treatment. party responsible = written up
English
50
49
1.1K
340.4K
🚀 Rocket
🚀 Rocket@rocketalignment·
@alec_harris_x Ya like let people do what they want but nudge them, inform them, make them sign a waiver etc
English
0
0
1
12
🚀 Rocket
🚀 Rocket@rocketalignment·
Getting one of these on the 4th of July permanently changed my politics
🚀 Rocket tweet media
English
2
0
6
257
🚀 Rocket retweetledi
Andrew M. Dai @ ICML
Andrew M. Dai @ ICML@AndrewDai·
At @ElorianAI, we believe AI should offer the world more than the abilities of a toddler, but the best models still can’t visually reason better than a three year old. I had the pleasure of talking with @rocketalignment about this crucial gap between visual generation and visual reasoning on @theinformation TV, thanks for having me on the show last week!
The Information@theinformation

AI only has pre-schooler level capabilities for visual tasks. "The frontier models—the best of them—still reason around the age of a preschooler, and any elementary school kid in that benchmark was able to beat all the frontier models on these visual tasks." "These are tasks that are not just object recognition. It's about spatial reasoning... tasks like navigating through the maze…” — @AndrewDai, CEO of Elorian

English
1
6
25
7K
🚀 Rocket
🚀 Rocket@rocketalignment·
That said, everyone remember that hearing loss is permanent painless progressive and preventable! 😌☝️
English
0
0
2
66
🚀 Rocket
🚀 Rocket@rocketalignment·
And what kills me is I always get these when I’m DRIVING and then they stay on the screen and block the map until you swipe it away making driving LESS SAFE
English
1
0
2
110