elana

paper backlog progress 0-0

1/4 New paper with @weijie444! We introduce a symmetry-compatible principle for LLM optimizer design and, as a byproduct, get an end-to-end layerwise optimizer stack where every major matrix-valued parameter (embeddings, LM heads, SwiGLU MLPs, MoE routers) has its own principled update! 📝 arxiv.org/abs/2605.18106 💻 github.com/timlautk/equiv…

NLAs are claimed to verbalize model activations. But can they faithfully interpret steered activations? In our latest paper, we show that steering moves activations into non-invertible regions; and almost surely, no prompt maps to steered activations! NLAs fail to interpret steered activation states faithfully, supporting our results! ↓ @anqi_liu33 @DanielKhashabi x.com/AnthropicAI/st…

@indefeasible_ also personal bias & not really “intro to proofs” at all but these lecture notes ~follow Apostol’s Calculus Volume II (more rigorous than typical Calc 3 & LinAlg), both that and the notes here could be useful depending on where you are mathematically math.columbia.edu/~mtwang/teachi…

@indefeasible_ longformmath.com/proofs-book/ The Book of Proof and Velleman’s How to Prove It are both also commonly recommended, but based on my friends’ experience this one seems pretty fun & unique :)

can anyone recommend me a intro to proofs book/lecture notes to read this summer?

MoEs are everywhere, but the design space is confusing: total vs active experts? expert size? shared experts? routing? token dropping? We train >2000 MoE LMs 🫠 to investigate and bring you: 📄🔪🍰 Slicing and Dicing MoEs Tl;dr: it's all about expert size and count [1/9]

I'm in the process of DMing every mutual that I know plays Minecraft rn, but if I don't DM you and you want to join a Supersymmetry (GregTech) server hmu!! curseforge.com/minecraft/modp…

@usr_bin_roygbiv @t3nsor “shape rotator” “wordcel” is a dumb dichotomy, the latest weschler tests do verbal comprehension fluid reasoning quantitive reasoning visual spatial working memory processing speed which is way more effective, lots of fri+qri+vcimaxxed ppl slaying in CS&math

@t3nsor you do for systems design. if you're a wordcel gpt will do a much better job

Hot take: you don't need much shape rotation ability at all to do most coding tasks, and writing "good code" is a wordcel ability.

This is devastating for skids everywhere, I got so much social mileage out of being able to set up a decent server

@TW1NKD3STR0YER To get the opposite of this experience, go to Jersey for superior bagels

in california. miss nyc already. get a bagel egg and cheese. it’s 10 dollars. fine. ask for saltpepperketchup. they dont have. ok. they pressed the bagel into a panini. this is where i draw the line

@atzydev mostly programming / software shenanigans, cannot share too much but im so hyped

@elelkhouri exciting stuff!! what will you be working on?

i beat teen pregnancy AND unemployment today 😼

@elelkhouri congrats!!!

i like my short bio because i think it is auraful but i wanted this screenshot so i will keep it like ts for a bit

I got a real transformer language model running locally on a stock Game Boy Color (thanks Codex)! No phone, PC, Wi-Fi, link cable, or cloud inference. • The cartridge boots a ROM, and the GBC runs the model itself. • The model is @karpathy’s TinyStories-260K, converted to INT8 weights with fixed-point math so it can run without floating point. • Built with GBDK-2020 as an MBC5 Game Boy ROM. • The model weights live in bank-switched cartridge ROM. Prompt entry happens on-device with the D-pad/buttons and an on-screen keyboard. • The prompt is tokenized on the Game Boy, then the ROM runs transformer prefill + autoregressive generation. The KV cache is stored in cartridge SRAM, because the GBC’s work RAM is tiny. It is extremely slow, and the output is gibberish because the math is heavily quantized/approximated, but the core thing works! Hardware: stock Game Boy Color + EZ Flash Junior + microSD. No soldering, no internal mods.

@multimodali I was live watching on call and said “I feel like he’s gonna win” before it even started, was like “Man I suck at predictions” after round 1, then was vindicated

@elelkhouri well this wasn't on my bingo card

let's gooooo

@leothrix nah i see reckon as belonging to slightly pretentious british schoolboys

One of my favorite language idiosyncrasies is that Europeans use “reckon” frequently and my American brain always associates the word with grizzled old gold prospectors that ride mules

@1owroller twewy (not a 3ds game necessarily but the combat is fun imo)

what are the best 3ds games? besides the obvious pokémon/mario series

i dont like sean very much but khamzat is incredibly boring to watch so i believe sean can still win