Ravi Adi Prakoso

Halo, perkenalkan aku Ravi Adi Prakoso, penerima Beasiswa Telkom University IDCloudHost angkatan 2024. Bagi yang punya pertanyaan seputar beasiswa ini, silahkan ya. Insya Allah bisa aku jawab.

@canalCCore2 @karpathy This is where the LLMs are hit the wall for might be the next decade. They failed hardly on spatial intelligence test. Moravec still haunt us probably.

@karpathy What if we made a benchmark where the model keeps iterating until it can recreate scenes from random video clips in Three.js, compares each result to the original, picks the best one, and then gets tested on script changes for robustness, like adding Pepe and Trump to Thriller 😂

Three days ago I left autoresearch tuning nanochat for ~2 days on depth=12 model. It found ~20 changes that improved the validation loss. I tested these changes yesterday and all of them were additive and transferred to larger (depth=24) models. Stacking up all of these changes, today I measured that the leaderboard's "Time to GPT-2" drops from 2.02 hours to 1.80 hours (~11% improvement), this will be the new leaderboard entry. So yes, these are real improvements and they make an actual difference. I am mildly surprised that my very first naive attempt already worked this well on top of what I thought was already a fairly manually well-tuned project. This is a first for me because I am very used to doing the iterative optimization of neural network training manually. You come up with ideas, you implement them, you check if they work (better validation loss), you come up with new ideas based on that, you read some papers for inspiration, etc etc. This is the bread and butter of what I do daily for 2 decades. Seeing the agent do this entire workflow end-to-end and all by itself as it worked through approx. 700 changes autonomously is wild. It really looked at the sequence of results of experiments and used that to plan the next ones. It's not novel, ground-breaking "research" (yet), but all the adjustments are "real", I didn't find them manually previously, and they stack up and actually improved nanochat. Among the bigger things e.g.: - It noticed an oversight that my parameterless QKnorm didn't have a scaler multiplier attached, so my attention was too diffuse. The agent found multipliers to sharpen it, pointing to future work. - It found that the Value Embeddings really like regularization and I wasn't applying any (oops). - It found that my banded attention was too conservative (i forgot to tune it). - It found that AdamW betas were all messed up. - It tuned the weight decay schedule. - It tuned the network initialization. This is on top of all the tuning I've already done over a good amount of time. The exact commit is here, from this "round 1" of autoresearch. I am going to kick off "round 2", and in parallel I am looking at how multiple agents can collaborate to unlock parallelism. github.com/karpathy/nanoc… All LLM frontier labs will do this. It's the final boss battle. It's a lot more complex at scale of course - you don't just have a single train. py file to tune. But doing it is "just engineering" and it's going to work. You spin up a swarm of agents, you have them collaborate to tune smaller models, you promote the most promising ideas to increasingly larger scales, and humans (optionally) contribute on the edges. And more generally, *any* metric you care about that is reasonably efficient to evaluate (or that has more efficient proxy metrics such as training a smaller network) can be autoresearched by an agent swarm. It's worth thinking about whether your problem falls into this bucket too.

I gave my hermes-agent an old MacBook Pro and a said: "Build whatever you want." I came back in the morning. It had written a Python script, accessed the Mac's Bluetooth module, and built a fully functional RFID/BLE surveillance kit to track physical objects in my house. My AI literally gave itself spatial awareness and a sense of touch. lmao wild. We are so completely cooked.

This cunt destroyed a school for girls in his “preemptive strike” less than a week ago, KILLING OVER ONE HUNDRED LITTLE GIRLS. Lying sadistic fucking prick. Come on cancer do your job.

@lynxluna Ada juga cabang ilmu yang jarang orang bahas. Yaitu Digital Twin dan Sim2Real. Bidang 3D programming itu sulit dikuasai AI karena representasi volume spasial itu hampir impossible untuk dikuasai AI, Ditambah lagi, gak ada standar format data 3D yang jelas, gak kayak PNG atau SVG.

Udah bener, ambil Matematika. Karena ini pre-requisite dari apapun yang lagi ngetren sekarang. Habis ambil Matematika, otodidak baca buku Teknik Komputer dan Elektro. Terus cobain ngoding sampai deploy sendiri pake VM. You're indomitable kalo udah bisa semua itu. Yakin.

I had Opus 4.6 web agent look up and screenshot all of Anthropics US military connections then create a schizo pinboard

@hanindh Hehe, banyak yang gak sadar juga kalau Nvidia juga sekitar 80% internnya kebanyakan Ph.D semua, ntah compsci, physics, ee, dsbgnya.

Minggu lalu meeting sama sebuah computing startup (not an academic institution) di EU. Di introduction slidenya: 355 employees, 136 of them are PhDs. ... Yaah beda dunia mungkin... 🤣

When CS Students ignore physics and math.

Yeahh boy, creating a full Neural Network library including Batch SGDs, Backprop, full GPU powered by DirectCompute instead of CUDA? Same dataset, same architecture, same lr, the difference is one is using CUDA, the other one is using DirectCompute. Shall i release this? 🤔

@Alibaba_Wan High quality LoRAs coming soon

Wrong motivation -> wrong results

Empathy in life and business is underrated

Ego is the enemy of growth

if you value intelligence above all other human qualities, you’re gonna have a bad time

In the future, humans will need to supervise AI systems much smarter than them. We study an analogy: small models supervising large models. Read the Superalignment team's first paper showing progress on a new approach, weak-to-strong generalization: openai.com/research/weak-…

@JeffDean Are there any plans for students outside US?

Colab Pro is now available for free for verified US students and faculty for one year.

@laginyarikutu Haloo, untuk ini, lebih ke persentase kamu benar menjawabnya. Untuk pembobotan, sistem pengurangan nilai jika salah, seingatku tahun lalu seperti itu, tapi untuk mekanisme, dll bisa dicek di akun ig-nya, yakni beasiswa.idcloudhost

@Raviadi1 hao ka, sistemnya seperti SNBT gini ga ya? butuh strategi keketatan, daya saing dll? kalo boleh tau infonya dimana ya?

Halo, perkenalkan aku Ravi Adi Prakoso, penerima Beasiswa Telkom University IDCloudHost angkatan 2024. Bagi yang punya pertanyaan seputar beasiswa ini, silahkan ya. Insya Allah bisa aku jawab.

@cilenkfkunhas Haloo, ini opsional ya. Bisa diisi dengan strip (-).

@Raviadi1 haloo kak ini yang sektor usaha isi apa yaa?

@whosrfamr27 Ini opsional ya, bisa diisi dengan strip

@Raviadi1 halo kak izin bertanya ini wajib di isi

@loveibubu Halo, boleeh yaa. silahkan :)

@Raviadi1 haloo kak boleh aku nanya nanya di dm gak ya? makasih sebelumnya

@itsumeeee Haloo, boleeh.. silahkan yaa :)