Ravi Adi Prakoso

231 posts

Ravi Adi Prakoso banner
Ravi Adi Prakoso

Ravi Adi Prakoso

@Raviadi1

Lonely soul... lost in a lonely thought Physics UI 23 CS Tel U 24

Indonesia Katılım Temmuz 2013
381 Takip Edilen51 Takipçiler
Sabitlenmiş Tweet
Ravi Adi Prakoso
Ravi Adi Prakoso@Raviadi1·
Halo, perkenalkan aku Ravi Adi Prakoso, penerima Beasiswa Telkom University IDCloudHost angkatan 2024. Bagi yang punya pertanyaan seputar beasiswa ini, silahkan ya. Insya Allah bisa aku jawab.
Indonesia
30
3
21
3.3K
Ravi Adi Prakoso
Ravi Adi Prakoso@Raviadi1·
@canalCCore2 @karpathy This is where the LLMs are hit the wall for might be the next decade. They failed hardly on spatial intelligence test. Moravec still haunt us probably.
English
1
0
1
35
caio temer
caio temer@canalCCore2·
@karpathy What if we made a benchmark where the model keeps iterating until it can recreate scenes from random video clips in Three.js, compares each result to the original, picks the best one, and then gets tested on script changes for robustness, like adding Pepe and Trump to Thriller 😂
English
1
0
1
454
Andrej Karpathy
Andrej Karpathy@karpathy·
Three days ago I left autoresearch tuning nanochat for ~2 days on depth=12 model. It found ~20 changes that improved the validation loss. I tested these changes yesterday and all of them were additive and transferred to larger (depth=24) models. Stacking up all of these changes, today I measured that the leaderboard's "Time to GPT-2" drops from 2.02 hours to 1.80 hours (~11% improvement), this will be the new leaderboard entry. So yes, these are real improvements and they make an actual difference. I am mildly surprised that my very first naive attempt already worked this well on top of what I thought was already a fairly manually well-tuned project. This is a first for me because I am very used to doing the iterative optimization of neural network training manually. You come up with ideas, you implement them, you check if they work (better validation loss), you come up with new ideas based on that, you read some papers for inspiration, etc etc. This is the bread and butter of what I do daily for 2 decades. Seeing the agent do this entire workflow end-to-end and all by itself as it worked through approx. 700 changes autonomously is wild. It really looked at the sequence of results of experiments and used that to plan the next ones. It's not novel, ground-breaking "research" (yet), but all the adjustments are "real", I didn't find them manually previously, and they stack up and actually improved nanochat. Among the bigger things e.g.: - It noticed an oversight that my parameterless QKnorm didn't have a scaler multiplier attached, so my attention was too diffuse. The agent found multipliers to sharpen it, pointing to future work. - It found that the Value Embeddings really like regularization and I wasn't applying any (oops). - It found that my banded attention was too conservative (i forgot to tune it). - It found that AdamW betas were all messed up. - It tuned the weight decay schedule. - It tuned the network initialization. This is on top of all the tuning I've already done over a good amount of time. The exact commit is here, from this "round 1" of autoresearch. I am going to kick off "round 2", and in parallel I am looking at how multiple agents can collaborate to unlock parallelism. github.com/karpathy/nanoc… All LLM frontier labs will do this. It's the final boss battle. It's a lot more complex at scale of course - you don't just have a single train. py file to tune. But doing it is "just engineering" and it's going to work. You spin up a swarm of agents, you have them collaborate to tune smaller models, you promote the most promising ideas to increasingly larger scales, and humans (optionally) contribute on the edges. And more generally, *any* metric you care about that is reasonably efficient to evaluate (or that has more efficient proxy metrics such as training a smaller network) can be autoresearched by an agent swarm. It's worth thinking about whether your problem falls into this bucket too.
Andrej Karpathy tweet media
English
974
2.1K
19.4K
3.6M
Ravi Adi Prakoso retweetledi
TAYLOR.WTF
TAYLOR.WTF@TAYL0RWTF·
I gave my hermes-agent an old MacBook Pro and a said: "Build whatever you want." I came back in the morning. It had written a Python script, accessed the Mac's Bluetooth module, and built a fully functional RFID/BLE surveillance kit to track physical objects in my house. My AI literally gave itself spatial awareness and a sense of touch. lmao wild. We are so completely cooked.
TAYLOR.WTF tweet media
Nous Research@NousResearch

Meet Hermes Agent, the open source agent that grows with you. Hermes Agent remembers what it learns and gets more capable over time, with a multi-level memory system and persistent dedicated machine access.

English
43
47
801
92.9K
Ravi Adi Prakoso
Ravi Adi Prakoso@Raviadi1·
@lynxluna Ada juga cabang ilmu yang jarang orang bahas. Yaitu Digital Twin dan Sim2Real. Bidang 3D programming itu sulit dikuasai AI karena representasi volume spasial itu hampir impossible untuk dikuasai AI, Ditambah lagi, gak ada standar format data 3D yang jelas, gak kayak PNG atau SVG.
Indonesia
0
0
4
485
Clair 光
Clair 光@lynxluna·
Udah bener, ambil Matematika. Karena ini pre-requisite dari apapun yang lagi ngetren sekarang. Habis ambil Matematika, otodidak baca buku Teknik Komputer dan Elektro. Terus cobain ngoding sampai deploy sendiri pake VM. You're indomitable kalo udah bisa semua itu. Yakin.
Yehez | Kang Ketik@YehezGun

Yg math related kyk matematika murni atau applied math. Gk tau knp, gw liat tmn2 seangkatan lulusan prodi math ini lebih "agile", kemana2 bisa (walau mostly ke tech), tp gajinya beuuuhhh

Indonesia
17
99
846
63.5K
Ravi Adi Prakoso retweetledi
JB
JB@JasonBotterill·
I had Opus 4.6 web agent look up and screenshot all of Anthropics US military connections then create a schizo pinboard
JB tweet media
English
10
13
233
11K
Ravi Adi Prakoso
Ravi Adi Prakoso@Raviadi1·
@hanindh Hehe, banyak yang gak sadar juga kalau Nvidia juga sekitar 80% internnya kebanyakan Ph.D semua, ntah compsci, physics, ee, dsbgnya.
Indonesia
0
0
1
99
Ravi Adi Prakoso
Ravi Adi Prakoso@Raviadi1·
Yeahh boy, creating a full Neural Network library including Batch SGDs, Backprop, full GPU powered by DirectCompute instead of CUDA? Same dataset, same architecture, same lr, the difference is one is using CUDA, the other one is using DirectCompute. Shall i release this? 🤔
Ravi Adi Prakoso tweet mediaRavi Adi Prakoso tweet media
English
0
0
1
69
Wan
Wan@Alibaba_Wan·
ZXX
49
55
787
60.8K
Ravi Adi Prakoso retweetledi
Ilya Sutskever
Ilya Sutskever@ilyasut·
Wrong motivation -> wrong results
English
39
91
736
155K
Ravi Adi Prakoso retweetledi
Ilya Sutskever
Ilya Sutskever@ilyasut·
Empathy in life and business is underrated
English
96
231
1.7K
367.1K
Ravi Adi Prakoso retweetledi
Ilya Sutskever
Ilya Sutskever@ilyasut·
Ego is the enemy of growth
English
249
507
3.9K
2.1M
Ravi Adi Prakoso retweetledi
Ilya Sutskever
Ilya Sutskever@ilyasut·
if you value intelligence above all other human qualities, you’re gonna have a bad time
English
754
2K
14.3K
8.9M
Ravi Adi Prakoso retweetledi
OpenAI
OpenAI@OpenAI·
In the future, humans will need to supervise AI systems much smarter than them. We study an analogy: small models supervising large models. Read the Superalignment team's first paper showing progress on a new approach, weak-to-strong generalization: openai.com/research/weak-…
OpenAI tweet media
English
535
1.3K
6.6K
2.5M
Ravi Adi Prakoso
Ravi Adi Prakoso@Raviadi1·
@laginyarikutu Haloo, untuk ini, lebih ke persentase kamu benar menjawabnya. Untuk pembobotan, sistem pengurangan nilai jika salah, seingatku tahun lalu seperti itu, tapi untuk mekanisme, dll bisa dicek di akun ig-nya, yakni beasiswa.idcloudhost
Indonesia
0
0
1
279
`
`@sincosv·
@Raviadi1 hao ka, sistemnya seperti SNBT gini ga ya? butuh strategi keketatan, daya saing dll? kalo boleh tau infonya dimana ya?
Indonesia
1
0
0
206
Ravi Adi Prakoso
Ravi Adi Prakoso@Raviadi1·
Halo, perkenalkan aku Ravi Adi Prakoso, penerima Beasiswa Telkom University IDCloudHost angkatan 2024. Bagi yang punya pertanyaan seputar beasiswa ini, silahkan ya. Insya Allah bisa aku jawab.
Indonesia
30
3
21
3.3K
cil
cil@cilenkfkunhas·
@Raviadi1 haloo kak ini yang sektor usaha isi apa yaa?
Indonesia
1
0
0
197
sellyy 🍥
sellyy 🍥@loveibubu·
@Raviadi1 haloo kak boleh aku nanya nanya di dm gak ya? makasih sebelumnya
Indonesia
1
0
0
208