Vanishing Gradient
536 posts
Vanishing Gradient
@vanishinggradi2
Now you see them. Now you don't. Dependent on Initialization.
Layer 0 Katılım Kasım 2020
10 Takip Edilen28 Takipçiler
Zuckerberg paid $14.3 billion for a 28-year-old who had never trained a frontier model. Nine months later, that bet just shipped.
The benchmark table tells you exactly what kind of lab Wang built. Muse Spark leads or ties Opus 4.6 and GPT 5.4 on multimodal perception, health queries, and visual reasoning. MedXpertQA, SimpleVQA, ScreenSpot Pro, CharXiv. These are all data-quality-sensitive benchmarks where training set curation determines the ceiling.
Where it gets destroyed: ARC AGI 2 (42.5 vs 76.5 Gemini), Terminal-Bench (59.0 vs 75.1 GPT 5.4), GDPval office tasks (1444 vs 1672 GPT 5.4). Coding and abstract reasoning. The exact categories where architecture innovation and RL scaling matter more than data.
This is a data labeling CEO's model. The fingerprints are all over the results. Wang spent seven years learning which benchmarks respond to better data and which ones require something else entirely. Muse Spark maxed out the first category and exposed the gap in the second.
The $14.3B question was always whether the guy who built the best data pipeline in AI could build the best model. The answer so far: he built the best model at the things data pipelines solve, and a mediocre one at everything else.
The move nobody's pricing: Meta said larger models are already in development, private API today, open-source future versions. Wang called this "step one." If the next model closes the coding and reasoning gap, Meta goes from also-ran to three-horse race. If it doesn't, they spent $14.3 billion to build a very good medical chatbot for 3 billion users.
Both outcomes are interesting. Only one justifies the stock moving 9%.
Alexandr Wang@alexandr_wang
1/ today we're releasing muse spark, the first model from MSL. nine months ago we rebuilt our ai stack from scratch. new infrastructure, new architecture, new data pipelines. muse spark is the result of that work, and now it powers meta ai. 🧵
English
@mohitwt_ Maybe MSL used muse to develop their own model
English
gave Muse Spark a simple task to write autograd + neural nets in a single python file
> implemented Tensor, backward, zero_grad, mul, ReLU, tanh, Module, Linear, SGD, and alot more
> 336 lines
> none of it worked
model wasnt even learning, stuck at same loss for 1800 epochs lmao
AI at Meta@AIatMeta
Introducing Muse Spark, the first in the Muse family of models developed by Meta Superintelligence Labs. Muse Spark is a natively multimodal reasoning model with support for tool-use, visual chain of thought, and multi-agent orchestration. Muse Spark is available today at meta.ai and the Meta AI app. We’re also making it available in private preview via API to select partners, and we hope to open-source future versions of the model. Learn more: go.meta.me/43ea00
English
@FrankVardu @SeanPlunket Maybe that would get you to finally shut the fuck up?
English
@SeanPlunket Remember they cut our peoples heads off whilst alive and posted the videos online.
English
@SirTHobbes @ben_golub Sure, hire leads from OAI and GDM for 100’s of millions and they’ll write one for you.
English
@ben_golub I'd love to see a textbook on how to train a frontier model please
English
@shuchaobi “Fighting reward hacking” you mean fighting the urge to cheat on benchmarks?
English
Meet Muse Spark. This is one step towards personal superintelligence and a great team effort involving numerous nights of training, debugging, improving instability, and fighting reward hacking. We have just begun, and feedback is extremely welcome.
Alexandr Wang@alexandr_wang
1/ today we're releasing muse spark, the first model from MSL. nine months ago we rebuilt our ai stack from scratch. new infrastructure, new architecture, new data pipelines. muse spark is the result of that work, and now it powers meta ai. 🧵
English
@DrewPavlou Let’s flip the script for a minute: you’re a colossal fucking idiot.
English
Let’s flip the scenario for a moment.
Imagine Iran killed Trump in the first 5 minutes of the war, established air superiority over the US mainland, wiped out the entire US Air Force, US Navy, killed half the Cabinet, flattened the US military industrial complex, then started building runways in Missouri to land Iranian troops without losing any casualties.
Would you say it was a US victory if the US managed to keep the Panama Canal closed throughout this all?
Sami Gold@souljagoyteller
The fact that America’s goals went from “the complete destruction of Iran’s military power or even regime change” to “reopen a trade route that was open a month ago” doesn’t say great things about American power
English
@fearsema @IdoHalbany Iran has complete uncontested control ofer the straight of Hormuz. Tariffs are worth ~500bn yearly. The us gained…?
English
@IdoHalbany Iran winning = navy gone, Air Force gone, majority leaders dead, economy crippled, asking Pakistan to broker a ceasefire so they don’t get bombed completely, greatest victory ever
English
If you need to explain why you won a war, then you didn’t win the war.
English
@Mupper41 Weird how they never showed where the other guy bought his gun.
English
Jesus no wonder they are afraid of everything..
Heckler & Koch@HecklerAndKoch
A no-compromise, fully featured micro-compact from our factory in Oberndorf, Germany.
English
@XLB_God If there’s one thing Australians excel at it’s committing acts of violence in south east Asia.
English
He woulda got stuck in a vietcong tunnel and brutally sodomized by a Type 56 bayonet
auspill@aus_pill
Ben Roberts-Smith > 6’7, 250lbs (200cm, 110kg) > Taliban described him as the ‘big soldier with blue eyes’ > Fought with crusader’s cross patch > SASR, Victoria’s Cross recipient
English
30 שנה הוא מבלבל לנו את השכל על איראן. סוף סוף יש הזדמנות לעשות עם זה משהו, הוא נכשל.
20 שנה הוא מתלונן שחיזבאללה לא פורק מנשקו. סוף סוף יש לו הזדמנות לעשות עם זה משהו, הוא נכשל.
17 שנים הוא מבטיח שהוא ימוטט את חמאס. סוף סוף יש הזדמנות לעשות את זה, הוא נכשל.
צריך להגיד ביושר.
@netanyahu הוא לוזר.
עברית
@AbuHafsah1 Why did none of his comrades, supposedly the bravest manliest men Australia had to offer, have the balls to not commit war crimes with him. Fucking cowards, the lot of them.
English
This dog of Australia:
Allegedly machine-gunned a prisoner with a prosthetic leg in the back.
He is also accused of pressuring an inexperienced junior SAS soldier to execute an elderly unarmed Afghan detainee ("blood the rookie").
The prosthetic leg was reportedly taken as a souvenir and used by some soldiers as a drinking vessel.
Allegedly kicked an unarmed civilian (named Ali Jan, a father of six) off a cliff, then directed subordinates to shoot him dead at the bottom.
Why was Australia in Afghanistan in the first place?
English
@thomasbsauer Why do weak ass pussies always want to brag about how close they got to tIEr OnE oPeRaToRs as if it somehow makes them manlier by proxy? No one gives a shit.
English
I heard all about Ben from his friends.
Dude was legendary 🇦🇺
While part of a US Army Special Forces unit in the Pacific for a couple years, I spent a summer in Australia with G Troop and E Troop from the SAS,along with our Aussie EOD counterparts from the Special Operations Engineer Regiment.
Some of the best dudes I ever worked with.
Many of them were close friends with Ben.
In true Aussie fashion, one called him, “The best fuggin’ cunt I’ve ever known.”
The highest of praise from those guys.
They love him. A real Australian patriot and a great man.
The SASR and SOER bros even took us to the Australian War Memorial in Canberra one weekend, where we saw the Hall of Valour with his display in it, pictured below.
This story says a lot about the direction that country’s headed. What a shame.
Drew Pavlou 🇦🇺🇺🇸🇺🇦🇹🇼@DrewPavlou
The Australian government just arrested our most decorated veteran from the Afghanistan War for alleged war crimes. Ben Roberts-Smith was awarded the Victoria Cross for Australia after singlehandedly storming two Taliban machine gun positions by himself in order to save his own SAS team. And this is how we repay him. I spoke to an Australian Afghan war veteran who told me that the Australian government literally paid for billboards in Afghanistan offering to pay random Afghans to come forward with war crime allegations. What country on Earth would do this to itself? Tell me: When does the Taliban plan on holding war crimes tribunals into their own people? When do the Taliban plan on holding themselves accountable for terrorism? Absolutely retarded to purge your own men when fighting a brutal barbaric enemy that respects no rules whatsoever. We are a cucked nation. Free Ben Roberts-Smith.
English
@Harlemundo110 @cosmicjester It’s you. You’re the weakest shit. Surprised you were allowed to take the dick out of your mouth long enough to type up that reply, and that you were able to see the screen clearly through the tears from gagging.
English
@vanishinggradi2 @cosmicjester I don’t think you’re in any position to say what’s the weakest shit, you anonymous coward
English
Fascinated by these short guys that mention Ben Roberts-Smith large height as some sort of defence or justification for his crimes. It’s like they are genetically predisposed to just submit to guys taller than them. Weird behaviour.
English
@mrr78504 @StatisticUrban If only they’d killed themselves before they went invading and murdering we’d all be better off.
English
@StatisticUrban And about a dozen of the SASR men who served with him will likely testify in his defence.
The taxpayer has burned $100 million on this. For two Indictments, neither of which are likely to result in safe convictions.
And veterans suicide will spike.
Terrible, terrible policy
English
Man, he's being charged with 5 counts of war crime murder, i.e. the killing of unarmed, detained civilians/prisoners.
He's alleged to have shot an unarmed teenager in the head.
21 of his own soldiers, from the elite Special Air Service regiment, have come forward against him.
Drew Pavlou 🇦🇺🇺🇸🇺🇦🇹🇼@DrewPavlou
The Australian government just arrested our most decorated veteran from the Afghanistan War for alleged war crimes. Ben Roberts-Smith was awarded the Victoria Cross for Australia after singlehandedly storming two Taliban machine gun positions by himself in order to save his own SAS team. And this is how we repay him. I spoke to an Australian Afghan war veteran who told me that the Australian government literally paid for billboards in Afghanistan offering to pay random Afghans to come forward with war crime allegations. What country on Earth would do this to itself? Tell me: When does the Taliban plan on holding war crimes tribunals into their own people? When do the Taliban plan on holding themselves accountable for terrorism? Absolutely retarded to purge your own men when fighting a brutal barbaric enemy that respects no rules whatsoever. We are a cucked nation. Free Ben Roberts-Smith.
English
@StatisticUrban Why were they too chicken shit to stop him murdering children in the first place?
Cowards to the last man.
English
@kimmonismus Given meta is burning through trillions of Claude tokens we know this model will be absolute garbage.
English
Big update: Meta is gearing up to release its first LLM built under Scale AI founder Alexandr Wang very soon.
-And while not all of it will be open, the company plans to eventually open-source versions of the new family.
-Meta knows these models won't beat OpenAI or Anthropic across the board, but it's betting on specific areas of consumer strength to stay relevant in an increasingly crowded race.
Ina Fried@inafried
New @axios Scoop: Meta will open source versions of new models set to be released soon - the first under @alexandr_wang. But open versions won’t be right at launch. Meta wants to remove some proprietary elements and address potential safety risks
English
@shaunrein What are you doing to stop it? Oh literally nothing? Continue fucking off then.
English
@MsSuzeeQue @keithedwards You don’t join the military because you’re brave or courageous. Quite the opposite in fact.
English
@keithedwards Yet they refuse to speak out while regular citizens worldwide risk everything to speak out.
English
This is why Pete Hegseth fired more than a dozen top generals and senior military officers btw
English
@DeItaone “Mission accomplished”? Where’s the banner?
English
*VANCE: THE MILITARY OBJECTIVES OF THE IRAN WAR HAVE BEEN COMPLETED
English