chris chiu

@AnikaSomaia How would an independent evaluation avoid getting goodharted?

imo independent evaluation is the most important part of this proposal. all my thoughts on today’s benchmarks are summarized by goodhart’s law: “when a measure becomes a target, it ceases to be a good measure”

@xdotli What if all one eats is slop? 🤔

eating a lot is necessary condition for acquiring a good taste

@airkatakana You’re going to have hoards of PhD students asking for internship opportunities

next time i go to an academic conference im listing my affiliation as anthropic or openai so i get honeypotted

@shiraeis Are we just FPGAs

We often describe learning as neurons changing how strongly they connect - the weights get updated - but a biological neuronal connection has second parameter almost no artificial neural network actually touches - delay. A spike arrives earlier or later depending on things like axon geometry (axons are the neuron tails that conduct electrical impulses), myelin thickness (myelin is a fatty insulation layer that wraps around the axons), sheath length, and the gaps between sheaths. This is measured in milliseconds, which sounds like a rounding error if you don’t realize/know that most neural computation depends on several signals arriving within the same time window. Changing a weight changes the loudness of a neuron’s communication. Changing a delay, however, changes WHO it can talk to - which other neurons it’s capable of communicating with synchronously. Yes, the brain is learning whether 2 neurons should communicate, but it’s also learning WHEN their messages should meet. It turns out myelin isn’t a childhood fixture you install once and forget about. Oligodendrocytes (the cells that make myelin) react to new activity, & new myelin turns out to be NECESSARY for some learning. When you block new oligodendrocytes in a mouse, it struggles to pick up a hard motor skill, with the gap between treated and untreated mice showing up within the first hours of practice, much closer to the timescale we associate with synaptic learning than slow structural maintenance. When researchers imaged single axons while mice learned a reaching task, during learning, existing sheaths in the circuit actually retracted, meaning the exposed gaps got longer. Modeling suggests this should temporarily slow down conduction of electrical impulses, intuitively. After learning, new sheaths grew in and created longer stretches of myelination, which produces faster and steadier learning signals. Essentially, the circuit loosens its own timing, hunts for a better arrangement, then physically bakes the winner into the wiring. **** NOTE: this decompile / recompile read is mine and not necessarily the authors’, but it fits along with another finding from the visual cortex - when you block adolescent oligodendrocyte development, adult mice actually maintain juvenile levels of plasticity, which is supposed to close after the critical period. This is adjacent to the perineuronal net I posted about in my earliest post about critical periods, but it adds another mechanism. I initially mentioned the brain installs a molecular brake around the inhibitory neurons, but it is also potentially stabilizing a circuit by making its communication timing more precise. This means myelin does 2 things that are in tension with one another. First, it lets distant circuits talk fast with almost no jitter, BUT second, that same reliability is exactly what makes them hard to reconfigure with respect to timing. Stability is something the brain actively builds and pays a price to maintain. This is also what is happening when a skill becomes automatic. Early on, the coordination happens in effortful synaptic activity, but with repetition, part of the solution migrates into the TRAVEL TIME of the network. This renders it such that you can stop micromanaging the movement you’re performing because your neural hardware actually gets scheduled to cooperate without your conscious effort. Now, for my favorite part: the juicy ML connections - in most AI networks, we learn weights and treat propagation time as fixed. Even transformers run in locked synchronized layers & learn WHAT should interact, but never WHEN one path's output should show up relative to another. There are experiments with spiking nets with trainable delays, finally testing the missing delay parameter. As it turns out on temporal tasks like speech, learned delays DO get you performance gains while reducing the need for recurrence or larger network size. Your big takeaway from this should be that time is representational capacity we’ve been neglecting to address - NOT the simpler read (which is that we should seek to mimic biology in AI architectures). Imagine a fast learner updating weights and routing in tandem with a slow compiler that makes note of which computations keep needing to converge & quietly tunes their latency until they arrive together. Learning, as it turns out - at least biologically - is actually destabilizing the circuit, detecting useful choreography with respect to time, and then actually embedding that choreography in how long the signals take to arrive. TLDR - you practice in software, but eventually biology moves the skill into hardware.

I turned myself into a ballot box at #ICML, asking “would you upload your brain?” After 3 days of awkwardly turning my back on people so they could vote: 31 yes, 30 no, 1 undecided. I expected a bit less, but turns out half of us are open to living on the cloud as AI agents

Can LLM agents coordinate in long-horizon, open-ended worlds? We evaluate 13 modern LLMs in a new benchmark where agents must work together to explore, communicate, trade resources, craft tools, build structures, and fight mobs. TL;DR: Most agents struggle, averaging only ~6% normalised return. Yet on the hardest setting, zero-shot Gemini 3.1 Pro performs comparably to the best MARL agent trained for 1 billion environment steps. More broadly, we find coordination is a distinct bottleneck beyond long-horizon task competence, with communication having the largest effect in our harness ablations. 🧵👇

IMO the part about elaborating probability distribution, as well as the investigation over modalities, was the most novel bit The headliner now reads like “we did this prompt trick and it expanded diversity”, but instead maybe “we found that diversity in output can be activated by a prompt trick, and investigated why” is a stronger narrative In general getting (good) awareness to your work is difficult though, something I’m trying to figure out myself 😂

@CHYChiu @simon_ycl Thanks for that! Hmm... I'm starting to suspect we're due for a couple of blogposts to make the "thinking around the thinking" more accessible. Did anything in particular stand out to you as needing more / clearer coverage?

80 pages. 5 pages of proofs. >100 experiments (even with RL!). a whole analysis of why post-training collapses the distribution. → "just some prompt-engineering paper" 🤡 x.com/shi_weiyan/sta…

@dch @simon_ycl Sending some support here as well - I spoke with you guys at ICML and read the paper afterwards. Very thorough work! The “why” was interesting and I think it was highly underrated

@simon_ycl And after the amount of insanely careful work we actually did... 😮‍💨 I guess it's understandable if you skim. I did write a comment unpacking things, but it seems to have gotten buried: reddit.com/r/MachineLearn…

@realchillben I find it varies a lot by country / by state, but in general I found it easier to approach Americans for a first conversation, but easier to build lasting friendships with Europeans

curious if europeans actually have a higher bar for friendship vs americans...

Let the regulatory capture begin

@mathyouf Seoul ready for the furry uprising

They really do train everyone here for military service

wait i think ive seen this episode

@dioscuri Consider fibre as well my dude

This is my nutrition regime these days

@airkatakana Saw you only liking one but not the other as well

do you think he’s confused as to why his student’s research is getting 100x more attention

@mooncat_is Yeah but you didn’t tell everybody that your cousin is a transcendent genius that will simultaneously cure cancer and bring doom to humanity

Breaking: I gave my cousin access to every paper ever written last Christmas and he also failed to invent general relativity. Humans cannot jump. I even gave him the ones written after general relativity and he still could not do it. Human are a bubble. 😭

@MachJustine But what if I’m an idiot at rest

an idiot in motion goes further than a genius at rest

@iScienceLuvr We have LLMs that can solve 50 year old math conjectures and most people use it to convert pdf to markdown

Well yeah that's the point of the movie. People really have no media literacy smh.

@scaling01 x.com/chychiu/status…

The AI 2027/2040 guys' highest probability outcome is a bad AI future with ASI 2031, with anything from doom to insane power concentration and wealth inequality but keep scrolling on your phone. no biggie. it's still 5 years away, so why worry, right?

My Twitter feed today

@iScienceLuvr Which startup had the most effective conversion rate so far??

AI startups are actually wrappers to turn funding into lit socials/parties