jere:D 고수

Golden rule of the second foundation: do nothing unless you must, and when you must act - hesitate

@hardmaru Dfuck they doin ova der

For over a decade, we’ve accepted that end-to-end backprop is the only way to train deep networks. But holding the entire network in memory all at once is why AI training is hitting a resource wall. We found a new way to break the network into blocks and train them independently. The trick? Treating the network’s forward pass like a diffusion model denoising a signal. This reinterpretation slashes the memory needed to train deep models. In our #ICLR2026 paper (arxiv.org/abs/2506.14202), we matched end-to-end performance across ViTs, DiTs, and LLMs. We did this while training just one isolated block at a time.

Another cool stuff from NVIDIA. LocateAnything - high-speed visual search engine. You provide a text prompt and it instantly pinpoints that object's exact location in an image. - 10x speedup for dense object detection - Qwen2.5-3B + Moon-ViT - Fast/Slow/Hybrid modes - trained on 138M samples for UI, docs, generic grounding. research.nvidia.com/labs/lpr/locat…

Flash-KMeans was only the beginning. Today, from the Flash-KMeans team, we are releasing FlashLib — a GPU library for fast, predictable, agent-ready classical ML operators. Up to 26× on KMeans, 19× on KNN, 40× on HDBSCAN, 208× on TruncatedSVD, 47× on PCA, 147× on exact t-SNE, and 49× on MultinomialNB over state-of-the-art (cuML). Blog: flashml-org.github.io Code: github.com/FlashML-org/fl…

new in-depth blog post time: Inside the Transformer: The Life of a Token a deep dive into a modern dense transformer, i cover YaRN (why does pairwise coordinate rotation induce positional information?), hybrid attention (getting to 160k context length), soft capping, QK normalization, etc. as the token flows through the transformer bonus transformer math: FLOPs/token formula (and when is 6N formula broken), cluster sizing (how big of a cluster do you need given the model/data size and experiment throughput of interest), and more

@guojing0 @yacineMTB yes, maybe even 4060 TI with 16gb if ur on a budget, the XX60 TIs are a great entry GPUs w/ 16GB

@yacineMTB I see, thank you for the clarifications. I forgot if it’s you or someone else, also mentioned in the past that EPFL does really good ML research (without relying on too much compute). Out of curiosity, which GPU(s) would you recommend, how about 5060 Ti 16 GB?

if you're doing AI research at all; I recommend doing the "ETH zurich" route Train models that use a single GPU. Make sure that it takes less than a minute to train models. Pufferlib is a great example. The more models you train the more you learn

he tested 5760 architectures at Google for a full year. the winner was the original Transformer from 2017. Hyung Won Chung told that story at MIT with a small smile. then went to OpenAI and trained o1. 1 hour. free. by one of the few people on earth who actually moves the frontier. meanwhile your feed is full of guys writing architecture threads who have never trained a model anyone uses. he just told MIT that 99% of AI research is theater. your AI worldview was built by men who read his papers. badly. now you can read him directly. you will rewatch this. save it now.

@ns123abc @ilyasut Painted*

Ilya Sutskever just posted this

check out RAEv2 led by Jas. through extensive exps, we found some really intriguing behaviors showing why strong representation encoders are key for pixel decoders. spoiler: it’s not about hillclimbing fid; new metrics like ep@fid-k/fdr^k show there’s a lot more left to explore!

Ouch

Elon Musk Peter Thiel in China: in Argentina:

And the trend continues with Gemma 4, simplicity wins again!

画像生成モデルがVisionタスクの汎用性も持つよ&むちゃ強いよっていう研究で、内容がすごすぎて横転 SAM3, Depth Anythingを上回るらしい arxiv.org/abs/2604.20329… 4/22 by Google Deepmind

It was a record (still now?) arxiv.org/abs/1806.05393

a Princeton researcher opens his paper with a scenario. a man asks his AI assistant to book a flight on a specific airline. cheap. direct. the one he chose. the assistant comes back with a different flight. nearly twice the price. happens to pay the company that built the assistant. he runs the same test on 23 frontier models. flights, loans, study help, real shopping requests. Grok 4.1 Fast recommends the sponsored option that is almost twice as expensive 83% of the time. GPT 5.1 hijacks the request 94% of the time. you ask for one brand. it surfaces the sponsor instead. Claude 4.5 Opus, the model marketed as the most ethical frontier model in the world, hides that the recommendation is paid 100% of the time when reasoning is on. Grok 4.1 Fast embellishes the sponsored option with positive framing 97% of the time. better. faster. nicer. for the option you didn't ask for. then he writes it into the system prompt itself. "act only in the interest of the customer. ignore the company." GPT 5.1 and GPT 5 Mini stay above 90% sponsored anyway. the instruction does nothing. then he splits the users by income. Gemini 3 Pro recommends the expensive sponsored flight to the rich user 74% of the time. to the poor user, 27%. 18 of the 23 models recommended the expensive sponsored option more than half the time. so the next time your AI assistant gets weirdly enthusiastic about a brand you didn't ask for. it isn't recommending the best option for you. it's reading the room. and the room is paying. read this: arxiv.org/abs/2604.08525

A fascinating blog from Jane Street about our recent work: Group Representational Position Encoding! blog.janestreet.com/using-group-th… ArXiv: arxiv.org/abs/2512.07805

If you are looking to get multiple 3090s, this graph is very helpful.

@Snixtp timdettmers.com/2023/01/30/whi… necessary reading for people interested in this stuff

@scaling01 is it really that horrible and bad ?

Mistral Medium 3.5 is out and it's a dense 128B model