MarsDoge

4K posts

MarsDoge banner
MarsDoge

MarsDoge

@kamapapaaa

AI Tech - Better Cleaner Future. Best Retweets from the Biggest faces on the 𝕏 app. ₿ Blockchain, Metaverse, NFT, Crypto, 𝕏 spaces content creator

Mars Katılım Şubat 2013
1.5K Takip Edilen550 Takipçiler
MarsDoge retweetledi
DogeDesigner
DogeDesigner@cb_doge·
To the moon! 🚀
English
379
628
3.2K
109.9K
Rumi
Rumi@rumilyrics·
You are alive. You have a home. You have a bed. You have food. Be grateful.
English
360
2.1K
22.6K
782.8K
MarsDoge retweetledi
Science Postcard
Science Postcard@Sciencepostcard·
The evolution of computers [1940-2100] 🤯
English
169
2.1K
7.8K
968.7K
Andrej Karpathy
Andrej Karpathy@karpathy·
Three days ago I left autoresearch tuning nanochat for ~2 days on depth=12 model. It found ~20 changes that improved the validation loss. I tested these changes yesterday and all of them were additive and transferred to larger (depth=24) models. Stacking up all of these changes, today I measured that the leaderboard's "Time to GPT-2" drops from 2.02 hours to 1.80 hours (~11% improvement), this will be the new leaderboard entry. So yes, these are real improvements and they make an actual difference. I am mildly surprised that my very first naive attempt already worked this well on top of what I thought was already a fairly manually well-tuned project. This is a first for me because I am very used to doing the iterative optimization of neural network training manually. You come up with ideas, you implement them, you check if they work (better validation loss), you come up with new ideas based on that, you read some papers for inspiration, etc etc. This is the bread and butter of what I do daily for 2 decades. Seeing the agent do this entire workflow end-to-end and all by itself as it worked through approx. 700 changes autonomously is wild. It really looked at the sequence of results of experiments and used that to plan the next ones. It's not novel, ground-breaking "research" (yet), but all the adjustments are "real", I didn't find them manually previously, and they stack up and actually improved nanochat. Among the bigger things e.g.: - It noticed an oversight that my parameterless QKnorm didn't have a scaler multiplier attached, so my attention was too diffuse. The agent found multipliers to sharpen it, pointing to future work. - It found that the Value Embeddings really like regularization and I wasn't applying any (oops). - It found that my banded attention was too conservative (i forgot to tune it). - It found that AdamW betas were all messed up. - It tuned the weight decay schedule. - It tuned the network initialization. This is on top of all the tuning I've already done over a good amount of time. The exact commit is here, from this "round 1" of autoresearch. I am going to kick off "round 2", and in parallel I am looking at how multiple agents can collaborate to unlock parallelism. github.com/karpathy/nanoc… All LLM frontier labs will do this. It's the final boss battle. It's a lot more complex at scale of course - you don't just have a single train. py file to tune. But doing it is "just engineering" and it's going to work. You spin up a swarm of agents, you have them collaborate to tune smaller models, you promote the most promising ideas to increasingly larger scales, and humans (optionally) contribute on the edges. And more generally, *any* metric you care about that is reasonably efficient to evaluate (or that has more efficient proxy metrics such as training a smaller network) can be autoresearched by an agent swarm. It's worth thinking about whether your problem falls into this bucket too.
Andrej Karpathy tweet media
English
962
2.1K
19.5K
3.6M
MarsDoge retweetledi
SMX 🇺🇸
SMX 🇺🇸@iam_smx·
Elon Musk on How He Built His Massive Net Worth “I build a lot of things with awesome people, and the companies that build those things get assigned a big value”
SMX 🇺🇸 tweet media
English
144
231
1K
25K
MarsDoge
MarsDoge@kamapapaaa·
@BillyM2k I disagree with this. It should be the most important thing for the minimal wages ones... no one can predict the future. Only Trump 😂
English
0
0
0
32
MarsDoge retweetledi
MarsDoge
MarsDoge@kamapapaaa·
@naval I never heard about this. Interesting
English
0
0
1
11
Naval
Naval@naval·
A “computer” used to be a job title. Then a computer became a thing humans used. Now a computer is becoming a thing computers use.
English
1.3K
2.9K
29.8K
39M
MarsDoge
MarsDoge@kamapapaaa·
Oil stocks at 200x long… are started to drain BTC!
English
0
0
0
12
MarsDoge retweetledi
Elon Musk
Elon Musk@elonmusk·
@Vhoyde Banger
Indonesia
88
19
427
18.1K
MarsDoge retweetledi
Elon Musk
Elon Musk@elonmusk·
The lunatic left that took over Twitter was Wormtongue to the World. Firing @Jack was the final straw. He was the last bulwark. Now Bret Taylor is chair of @OpenAI
Matt Van Swol@mattvanswol

@C_3C_3 The graph of people, but especially kids, identifying as transgender literally nosedives as soon as Elon buys Twitter. Almost like it was a social contagion… …and people couldn’t talk about it because they were being censored Imagine that.

English
3.9K
19.7K
126.7K
19.7M
MarsDoge retweetledi
Elon Musk
Elon Musk@elonmusk·
@Anc_Aesthetics @nikitabier We are actively working on shutting down bullshit. Fighting the supersonic tsunami that is AI-generated content is a tough battle.
English
338
328
5K
159.5K
MarsDoge retweetledi
DogeDesigner
DogeDesigner@cb_doge·
Happy International Women’s Day! 💖💖
DogeDesigner tweet media
English
377
266
2.2K
53K
MarsDoge retweetledi
Mario Nawfal
Mario Nawfal@MarioNawfal·
🇺🇸 🇮🇷 Phase 2 of Operation Epic Fury: the underground war Iran hid its ballistic missiles in tunnels carved deep into mountain rock, shielded by hundreds of feet of reinforced concrete. Standard bombs won't even scratch it. So four B-2 stealth bombers flew 6,000 miles from Missouri, slipped through Iranian airspace undetected, and dropped dozens of bunker busters that penetrate concrete, then detonate from the inside out. Cost per bomb: $4,000. Cost of what it destroyed: billions. Source: AiTelly on YT
Mario Nawfal@MarioNawfal

🚨🇺🇸🇮🇷 Operation Epic Fury has reportedly gutted Iran's military infrastructure. Officials say the country's missile program, nuclear capabilities, and navy have been virtually eliminated, with proxy networks severely degraded across the region.

English
371
1.5K
7.5K
1.5M