Sam Gijsen

@_ueaj Actually pretty relevant for one of my projects; dropout you mean good old attn+ffn dropout?

It would be very fun to apply all the data augmentation/data effeciency techniques we've learned over the years but weren't applicable to LLMs due to the abundance of data + all the new ones. dropout, layer looping, massive overparameterization, muon, optimal hparams, etc.

@drmichaellevin @davidasinclair @ScienceMagazine I thought it was predominantly a telomere story?

The other part of the heart/cancer story is control of resting potential. Cardiac cells have a very strong control of their bioelectric state, and are extremely resistant to the kind of persistent depolarization necessary for the cancer phenotype. pmc.ncbi.nlm.nih.gov/articles/PMC42… thoughtforms.life/cancer-and-cel… Neural cells too, but they can be forced into mitosis etc. by persistent depolarizarion: sciencemag.org/cgi/reprint/19…

Wondered why we don’t hear about heart cancer? Contraction-sensing Nesprin-2 protein is discovered to prevent heart cancer By causing cells to pulse (or adding in Nespirin) we might be able to treat cancer in other organs 👏 @ScienceMagazine

@vikhyatk @BelieveOnJesus I’m in neither of these categories but wouldn’t both like language models?

@BelieveOnJesus there's a big divide between people who became programmers because they wanted to make video games, and people who joined the industry because someone told them it would make them money

"AI is going to wipe out programming jobs." Then why is every programmer I know working 20 hours a day ever since they started using AI? I thought this technology was going to free us from the toil of labor.

@teortaxesTex There’s a whole cohort of Elon-bucks farming accounts which aim to get these kind of quote tweets via sycophancy

Not bad, my ass Elon is somewhat in denial of xAI's current position I think

Happening now :D #iclr2026 - Hall 3, poster 722 Come talk about variational flow matching over manifolds for proteins and materials

@vikhyatk They need to introduce daily plans so we can keep up with these recommendations

if you’re still using opus after 5.5…

Come check out our poster on training brain foundation models using self-distillation at @iclr_conf Friday 10:30AM!

@yacineMTB Nice val loss bro

@kalomaze @celestepoasts Why is the bs so critical here?

@celestepoasts doing this at a bs approaching pretraining CLT sounds absolutely brutal at small scale. i have low faith both in the concept of "doing it on a model smaller than ~32b" and "doing it without per step batch sizes in the thousands" and ofc you have to be VERY rigorous wrt goodhart

wonder if pretraining scaling laws are holding at all, seems like something only a frontier lab would find out if they don't, and they obviously wouldn't publicize this

@kuberdenis Go to the lakes south-west, go for walks in the forests there or ferry tour over the lakes or visit Potsdam

what should i do in Berlin? i am stuck here for two more days

@sedielem Never got a semanticist setup to work with neural data, wondering if there's something about natural image statistics thats particularly favourable to this kind of approach.

FlexTok/Semanticist provided an elegant recipe to learn semantically coarse-to-fine sequence representations of images. This works for video as well: preserve the temporal axis, replace the spatial axes with a semantic coarse-to-fine axis. Promising for long video generation!

@vikhyatk @yacineMTB Distilling the errors

distilling from 1B to 400M is going well

@vikhyatk You’re missing out

haven't watched youtube shorts ever since the training cluster came online

@robustus 50 cents to the miners, 100 bucks to nvidia. Jensen wins again

This is pretty cool (tho way beyond my knowledge base). I would not have imagined qsafe txns possible without a new key type, but here we go apparently. Obv this isn't practical today (~$100 in GPU time needed to construct the tx), but it at least opens up the possibility space.

@francoisfleuret bottom-to-top, right? ...right?

@dearmadisonblue @sporadica @ClementDelangue but anthropic was for-looping over individual files as well

@sporadica @ClementDelangue yeah, once you have a spotlight on where to look you've solved most of the problem

"But here is what we found when we tested: We took the specific vulnerabilities Anthropic showcases in their announcement, isolated the relevant code, and ran them through small, cheap, open-weights models. Those models recovered much of the same analysis. Eight out of eight models detected Mythos's flagship FreeBSD exploit, including one with only 3.6 billion active parameters costing $0.11 per million tokens. A 5.1B-active open model recovered the core chain of the 27-year-old OpenBSD bug." aisle.com/blog/ai-cybers…

@_ueaj Palantir spoke of this

Worryingly, ontologically, all the zero-days were already in the training data

@thetrocro @LynAldenContact They only talk about exploits that have already been fixed, but currently bitcoin isnt mentioned in the 244 page model card

@LynAldenContact Did they find a bug in bitcoin?

Someone should write a book about this.

Reading claude mythos model card and slotted in this activity for tomorrow morning