!Briann🗿

@Ji_Ha_Kim Same their work is soo cool and i read the 100 hours in kimi and i loved everything @Kimi_Moonshot

I would love to join Kimi, they do amazing research strongly aligned with my interests

@charmainemahach 6, 7 times

Out of curiosity, how many times do you read a research paper before implementing it?

The Society (2019) really had something going, that whole “kids left to run everything” setup spiraling fast, and the fact it got cancelled right when things were opening up still stings.

@gpt_alex @Kimi_Moonshot shooters will shoot😂

Come introduce yourself to the team, we have your slippers ready. Reach out at: talent@moonshot.ai

@rasbt Training recipe carrying the model🔥🔥

Flagship open-weight release days are always exciting. Was just reading through the Gemma 4 reports, configs, and code, and here are my takeaways: Architecture-wise, besides multi-model support, Gemma 4 (31B) looks pretty much unchanged compared to Gemma 3 (27B). Gemma 4 maintains a relatively unique Pre- and Post-norm setup and remains relatively classic, with a 5:1 hybrid attention mechanism combining a sliding-window (local) layer and a full-attention (global) layer. The attention mechanism itself is also classic Grouped Query Attention (GQA). But let’s not be fooled by the lack of architectural changes. Looking at the benchmarks, Gemma 4 is a huge leap from Gemma 3. This is likely due to the training set and recipe. Interestingly, on the AI Arena Leaderboard, Gemma 4 (31B) ranks similarly to the much larger Qwen3.5-397B-A17B model. But as I discussed in my model evaluation article, arena scores are a bit problematic as they can be gamed and are biased towards human (style) preference. If we look at some other common benchmarks, which I plotted below, we can see that it’s indeed a very clear leap over Gemma 3 and ranks on par with Qwen3.5 27B. Note that there is also a Mixture-of-Experts (MoE) Gemma 4 variant that is slightly smaller (27B  with 4 billion parameters active. The benchmarks are only slightly worse compared to Gemma 4 (31B). I omitted the MoE architecture in the figure below because the figure is already very crowded, but you can find it in my LLM Architecture Gallery. Anyways, overall, it's a nice and strong model release and a strong contender for local usage. Also, one aspect that should not be underrated is that (it seems) the model is now released with a standard Apache 2.0 open-source license, which has much friendlier usage terms than the custom Gemma 3 license.

JAX is what a well-designed low-level machine learning framework looks like. Good design lets you deliver much greater performance with much lower effort. Bad design is the exact opposite.

@fchollet For real jax breaks down the whole process easily and in a very learnable way

Rono’s drunk uncle clips are finishing me tbh

Those videos are funny 😂

our brain rot probably started with sanjay and craig.

long ass weekend imenipata na mia

@space_colonist @AnthropicAI You got a slot for one more?

I’m pleased to announce I will be leading a new team on VLA interpretability @AnthropicAI

parents love giving you job hunting and career advice that are like “Have you tried destroying and betraying yourself for nothing”

@IamIronLAN Damn they even learning mech interp😐🙂

Stanford is kinda crazy because as a CS undergrad this term you’re choosing between: - CS336: 0 to hero on frontier model training - CS224R taught by Chelsea Finn (founder of Pi) - CS231N taught by Fei Fei Li (Imagenet, WorldLabs CEO) - CS221M Mech Interp Intro taught with Goodfire And a host of personal podcasts delivered by $T CEOs.

i tried in january i tried in february i tried in march and i will try again in april

btw i’d swallow 20 cursed fingers for you, but you’re not nerdy enough to understand

This is why elite universities are elite universities.

No way I am discovering As You Are by the Weeknd rn...he will ask for water when I am done with this song

>72B MoE MNIST classifier

"claude usage limit reached. your limit will reset at 7pm"