Badthings ☠️🏴‍☠️🏗️👨‍🚀

Compassion is a skill that can always be sharpened.

@kimmonismus but can it prevent a child element's margin from collapsing into its parent container?

Holy Sh*t: that changes the whole Fable 5 story completely: On June 11, the very same day Amazon reportedly uncovered the jailbreak, “Mythos” allegedly breached almost all classified systems belonging to the NSA and U.S. Cyber Command, not over the course of weeks, but within hours. "On June 11th Mark Warner, the vice-chair of the Senate Intelligence Committee, said that General Joshua Rudd, who leads the National Security Agency and the Pentagon’s Cyber Command, had told him that Mythos “broke into almost all of our classified systems, not in weeks, but in hours”." Via Economist

tis tru! A few hrs ago I ran into a friend (her partner who also works at Meta) trained as an engineer and has been working at Meta in a rather prominent position for several years She said they recently moved him to data labelling after a few prior moves He is a iOS engineer by trade. Now currently doing data labelling So not fake news...you may just not be seeing it

@deedydas This is fake news @deedydas. I really value your opinions on this platforms. Please don’t do this. Source: I work there.

I thought this was a joke. Meta now has made 30-50% of software engineers on core teams become data labelers. Their job is "giving human feedback on AI-generated Github repos" in an org called Agent Data Optimization. Maybe we are all training data generators after all.

if only there was a truly open source world foundational model everyone could use. Sure, most wfm's have some pesky work involved: • Scoping & Architecture • Design • Data Acquisition • Curation • Massive Scale Pretraining • Post-Training and Fine-Tuning • Evaluation and Safety • Release, Continuous Monitoring ...but a guy can dream cant he

you'll own nothing, and like it! 🧑‍🍳🪿

Loops ➰ will be Trees + Pipelines by fall combine all and do now 🫡👨‍💻 Also switch-up your reasoning (when it makes sense to do so). If you only use linear reasoning and consider CoT one-and-done or your goto default, you’re missing out on massive unlocks

@gdb it’s as if the rarest thing a Dentist does is look at teeth 🤯 very strange indeed

software engineering is so different now. hard to remember what it was like even 6 months ago.

@olivercameron @Suhail @jeffrey_hawke @odysseyml Let’s gooo! Odyssey is super rad and needs to be near the top of everyone’s world model radar Congrats, can’t wait to see what’s next

When @jeffrey_hawke and I started @odysseyml in 2023, we believed general world models would become a new class of foundation model. Three years later, it now feels true. And today, we're announcing a $310M Series B. AI can now understand and simulate the world!

@sahil4ai @a16z that’s actually just not accurate most labs actually use large scale RL simulation deploys w zero shot to outdoor terrains hybrid-robotics.berkeley.edu many such examples, this one might not be the most compelling but it’s a quick grab

@paultomkinson @a16z robots implicitly learn the physics model and that's why they can walk, jump and do backflips on 2 legs like humans. they can't do that in the simulator and transfer the learning to real life. Until you can do that, you will 'learn' the wrong things inside the simulator

World Labs CEO Dr. Fei-Fei Li explains says "world model" has become an overloaded term & explains what each kind of world model does: "Right now there are three ways of calling world models when it comes to spatial intelligence." "One is what I call a renderer, when the model puts beautiful pixels on the screen." "Another kind of world model is what we call a planner. That is more for machines, more for robots." "The third kind, which I think is the linchpin of the three, is a simulator." "A simulator could become a renderer. The simulator could become a planner. But this layer is a huge critical path to unlock spatial intelligence. And that's what World Labs is working on." @drfeifei at Bloomberg Tech live with @emilychangtv

@TheAhmadOsman even smaller, more specialized experts in MoE which could be themselves models, is what I speculate Ilya is up to at SSI Hierarchical / nested MoE aka bunch of small specialized models x1000

Frontier intelligence will be beaten by small and specialized models

@antirez @aiComingFast I wonder what the real number is

@aiComingFast It's impossible with 3% the GPUs.

Another important thing: Chinese models are not strong because they distill US models. Distillation of models via API is *impossible*. If somebody tells you the contrary, they don't understand machine learning:

Exactly 👍 and I just cracked my knuckles and distillation is proven extremely useful for specific goals. Mostly efficiency, speed, and cost reduction (to the tune of 25x cheaper). Yes maybe its value is heavily questioned regarding robustness and reasoning capability specifically but its hugely advantageous to do it and many studies and research back and prove this Is the ‘dark knowledge’ in logits hyped up, maybe sure transfer learning or data augmentation are the fall behinds. Agreed, it has perhaps become quite a poorly defined descriptor It’s still heavily done and heavily advantageous, however

This isn't very true. A big part of the problem is that the labs use the term distillation, which is a general post-training technique, in lieu of a specific issue of jailbreaking the API. (1) There is a second debate of *how* impactful distillation is, but it is definitely helpful. (2) This is entirely based on how the Chinese labs are jailbreaking the APIs to get reasoning traces out, which help bootstrap reasoning behaviors in new domains. There's a third point (3) which I take an excerpt from my recent piece, where the labs need to be more transparent why especially point (2) is true. From the third piece: " On the point of distillation, my hypothesis is that API builders don’t have an easy time preventing hacks or jailbreaking because it’s a deeply grounded property of reasoning models to want to output the reasoning traces, and it would make the model far less intelligent to fully patch the behavior. This is based on a few assumptions: a) Chinese labs are not just showing up as customers to Anthropic’s API and paying for tokens in the intended input-output form. If the Chinese labs are paying for intended use behaviors, despite being banned by the terms and conditions, I don’t have a lot of sympathy for the frontier labs manifesting policy actions against this. b) Reasoning traces are disproportionately effective at seeding behavior in downstream models. c) Leading labs work very hard to patch the pipeline of these jailbreaks. So, my logical conclusion is that the model companies would have to weaken their economic position to fully protect their IP. If this is the case, Anthropic would get a lot more sympathy from the AI research community by being transparent. It would also be far easier to have informed policy discussions, and not rely on me proposing Occam’s razor explanations for what the API jailbreaking looks like. " There's no need to misinform people because the labs use a bad term. The labs use this term partially to make the discourse confusing, as you're doing. (1) See interconnects.ai/p/the-distilla… (2) See: interconnects.ai/p/how-much-doe… (3) See: interconnects.ai/p/claude-fable…

ironically, the next thing that will beat this (fastest application ever to 1 billion active users) is the thing Zuck and Meta walked away from *and no, I don’t mean meta’s metaverse

Strawman would it be nice if we plausibly could somehow get there, yes. But it’s likely neither possible, nor necessary ‘100% physics accuracy’ across all scales, all times, and all observables is effectively impossible approximate but consistent and grounded models work extremely well for prediction, planning, and control. ‘Good enough’ suffices for vast majority of things CARLA uses physics sim and Tesla as well to supplement real-world data accurate dynamic and physics modeling is needed, 100% accuracy is not

The most critical thing that everyone is missing is that the "world model" needs a proper physics engine. Unless we can a physics engine that can model the real world to 100% accuracy, we would still have models that are dogshit. this is also why self driving cars can't simply be trained on a simulator and then simply drive off on the streets..

@elder_plinius everybody arguing timeline here the rest is what’s important

sooo what happens when we get some Fable-level open-source shit in the next 3-6 months? GPU bans, moving goalposts, or some secret third thing? 🤔

@VictorTaelin @elder_plinius yes but just a matter of when, so the question is valid

@elder_plinius this is not happening, OSS is currently 1+ years behind

@reactorworld @nvidia Decart might be in trouble very impressive

Real-time video editing is here. SANA-Streaming by @NVIDIA is now live on Reactor. Change the style, the scene, the mood of any video as it plays, with no rendering wait. Try it now: sana-streaming.reactor.inc

@AlexStanif96089 @ns123abc tis but a joke my internet fren

🚨BREAKING: ANTHROPIC SUED FOR FRAUD A customer filed a federal class-action lawsuit Monday claiming he hit 15% of weekly allowance in just one 5-hour sprint. “The actual usage provided by the Max 5x and Max 20x plans is far below the advertised amount.” They are calling it marketing fraud.

@max_paperclips @TheAhmadOsman which actually would work well using a top OS model base. Decoupled DiLoCo orchestrator over vanilla DiLoCo and GaLore and Muon. Hierarchical "Cluster of Clusters" to make it work Spitballin’ here OC

@max_paperclips @TheAhmadOsman so 1% of 5.5 has been achieved and not even remotely yet w data and RL we got a hella-long way to go best to start thinking orthogonal a Mamba Hybrid architecture is the best way, atm to at least, to bridge the gap short term

I really wish Distributed Training was the answer for Opensource AI. Unfortunately, at least for now, it is not the right path forward to make Opensource compete with the frontier.

@GPLv6 @TheAhmadOsman Intellect-2 .33%

@GPLv6 @TheAhmadOsman Bigger fish to fry Orion 100B is 1% of 5.5