restricted_ptr

@mctweetsthis С чего вы взяли про поменьше заплатить? У Теслы топовый компенсационный пакет.

Никогда не пойму любовь технарей (прогеров/исследователей и т.д.) к Маску. В этом суровом капиталистическом мире его типаж — это капиталист-барыга который заинтересован максимально из вас выжать соки и поменьше заплатить, в идеале заменить на ИИ. Откуда такая любовь к сапогу? 🤔

@PetrosyanRob А откуда вы знаете за кого они голосовали?

Тысячи армян приехали из России голосовать не для того, чтобы самим впервые обрести права, свободу и чувство собственного достоинства, а для того, чтобы лишить этих благ тех армян, которые здесь живут.

@zephyr_z9 Wait, what? Cerebras used to partner with Cadence for developing chip design tooling

I can assure u Nvidia has better AI/ML systems for chip design than Anthropic or OpenAI Ant/OpenAI lack the decades of proprietary data for it (Nvidia has lots of that) Also, Clive is hired cuz Anthropic & Broadcom are working on a custom TPU and likely other ASICs Ant/OAI need a partnership with Cadence or Synopsys if they want data access

@itsclivetime @bubbleboi When the baseline is sufficiently bad, 100x speedup is a reasonable improvement.

@bubbleboi in the right places certainly yes! replacing JAX is not the right place, replacing NCCL is. this won't get you 10x > how do you coordinate and schedule compute between nodes efficiently this is handled by the user (writing code in JAX) for the most part. not something C related

Improving CPU speed by 10x should not affect training speed essentially at all. The CPU's main job is to kick off the real work on the GPU. If your kernels are sane (fused etc), the time to launch a kernel on the CPU is <<1% of the kernel runtime, even in Python.

@yacineMTB It used to take a few minutes to train the breakout in pufferlib CPU backend on Mac Book Pro m2. On GPU it's probably trained in an instant!

Pufferlib is insane. You can train neural networks to play games out of the box if you have a CUDA GPU. Like breakout, Atari games, continuous action space problems. You can go to the website right now and they have neural nets running in wasm

@charles_irl Who's the winner?

ℂ is for ℂUDA

@mitsuhiko I am placing comments in the file to make it more explicit. Then let it cook. Although, still, some cleanup is often needed. (usually Codex, latest, xhigh)

Expected a change to be 10 lines of code. Clanker made a 300 lines diff. In moments like this I want to throw the damn thing against the wall.

@charles_irl Models may even train around bugs. When a small non catastrophic, but random, noise is introduced, it may train and go figure why the loss is slightly worse than without the bug.

my gut says that to solve float numerics problems from nondeterminism x nonassociativity, we need to think bigger than determinism. models could eg be trained with large amounts of "implementation noise" so that the learned network is more robust to implementation skew.

@pravdapi Капитола же?

Обстановка напряженная. Это Calella de Palafrugell. Отъехал от туристов немного.

@joefioti @FelixCLC_ They may serve parts of the model that fit or are well suited for the hardware.

@FelixCLC_ The only way cbrs actually works is if they get their wafer-on-wafer hbm stacking working, which is a crazy alien tech bet. Till then it’s just an sram meme machine.

@pravdapi @tengri_spirit Вся наша цивилизация это продукт короткого периода климатической стабильности, позволившей переход к сельскому хозяйству.

@tengri_spirit Что характерно эти изменения сильно коррелируют с количеством СО2 в атмосфере. Вулканы, большие пожары, активность солнца, циклы Миланковича все это может влиять на колебания климата. Такое количество СО2 которое люди сожгли однозначно приведет с существенному изменению климата.

Для всех, кто встревожен тем, как люди нагревают планету. Самолеты мол летают, коровы пукают много. Оранжевые участки графика - потепление, синие - похолодание. График описывает последние 10 000 лет. Кто, спрашивается, нагревал планету 4000 лет назад? Коровы?

@carmichaeljr @Robotbeat @andrewmccalip There is a large variability in efficiency between designs. Although, that goes slowly but works is probably better than the one that fails.

@Robotbeat @andrewmccalip Rim-driven thrusters work great for this apparently! CPS drone on youtube talks about them, little pricey but you could design a 3D printed version with coils and permanent magnets.

Sigh. Seaweed. Again. Now to go find him...

@joefioti No fishing pole, the day was wasted.

where your email finds me

@suchenzang Imagine the exasperation of a professional, designing the optics and postprocessing stack to get a great picture, and all this effort screwed up by AI slop on top.

how $125 billion market cap company came to thoroughly enjoy the smell of their own AI garbage must be studied

@hellogugunim @tvguidemsk And do not chew! That may release sugars.

@tvguidemsk 과일에 있는 당이 해로울 수도 있군요! 과일 주스 보다는 껍질째로 먹어야겠네요

러시아 친구들아 오늘 나 이 주스를 샀는데, 여기 진짜 설탕이 없는거야? 이게 가능해?

@HotAisle @0xSero @ptremblay A balance, beautiful balance.

@restricted_ptr @0xSero @ptremblay What’s more important for inference… cores or memory?

21 petabytes of memory bandwidth.

@HotAisle @0xSero @ptremblay Over 48KB. It's 900k cores lol!

@0xSero @ptremblay Over what…44GB?

@hellogugunim @J0hn_3Volta Looks great, is it dried salty or sweet? Lots of Japanese dried snacks also have a sweet taste. While Russian ones are salty.

@J0hn_3Volta 한국에서는 오징어 말려서 맛있게 먹습니다. 맥주 안주 입니다.

Dried fish is a popular beer snack in the CIS countries. I think in the rest of the world it seems very strange

@Grady_Booch Meaning, there's a lot of room for improvement!

The human brain runs on about 20 watts of power; certain of the frontier large language models require a few gigawatts just to train them and then multiple gigawatts to serve them to their clients. Mind you, this is not exactly a fair comparison, for the former measures one brain while the latter attends to multiple instances in a global elastic fabric. That notwithstanding, the vast difference between the two tells us that evolution is cunning and resourceful while our present architectures are blunt and clumsy.

@severnorth_ А членто тут при чём?