Jonny

@JordanNanos Any plans to benchmark long context scenarios? Feels like that's where the real value is

Yesterday we open sourced the InferenceX webapp Hopefully this makes it easier to analyze InferenceX data, and provides a simple way for accelerator startups and alternative runtimes to compare directly to the industry standards, and make performance claims + forecasts It also means we can more easily handle feature requests from the community for the public dashboard github.com/SemiAnalysisAI…

@satnam6502 Congratulations! That sounds awesome :)

I have landed my dream job. I’ve just accepted a position at Harmonic, a Palo Alto startup applying AI to formal mathematical reasoning. Harmonic’s Aristotle formal reasoning model achieved Gold Medal level performance at this year’s International Mathematical Olympiad (IMO). I will work on exploring applications of Aristotle to the formal verification of hardware. This job is a perfect intersection of hardware design and verification, functional programming, formal methods and machine learning, bringing together several threads of my career so far. The beauty of asking an AI to generate a proof for a lemma (e.g. a formal property about a circuit) is that it can be checked by an external interactive theorem prover (like Lean) to establish whether the AI’s output is actually correct. This is an awesome superpower! harmonic.fun @HarmonicMath

I might be the first and last functional programmer to never have been in either FP academia or FP finance.

@ESYudkowsky > mitochondrial renewal diets and medications failed Have you tried the turnbuckle mitochondrial fission/fusion cycling protocol? If so what effects did you see?

For the benefit of latecomers and CICO bros, my current equilibrium is "spend 1 month fasting / starving on 700 cal/day keto; spend 2 months eating enough to work during the day, going to bed hungry, and therefore gaining 1-2 lb/wk". I don't need a weight-loss solution, kids. Starving 1 in 3 months already works to lose weight. I need a "have enough energy to work, without gaining 1-2lb/wk" solution. Diets like the potato diet fail, not because they don't succeed in forcing me to eat less -- I do, indeed, end up with not enough room in my stomach to eat enough potatoes to work and not feel tired. The potato diet fails because it doesn't protect me from the consequences of starvation, the brainfog and the trembling hands. If I'm going to be too sick and exhausted to work, I might as well go full keto on 700cal/day and actually lose weight, rather than hanging around indefinitely in potato purgatory. Semaglutide failed, tirzepatide failed, paleo diet failed, potato diet failed, honey diet failed, volume eating with huge salads failed, whipped cream diet failed, aerobic exercise failed, weight lifting with a personal trainer failed, thyroid medication failed, T3 thyroid medication failed, illegal drugs like clenbuterol have failed, phentermine failed (but can help make it easier to endure a bad day when I'm in my 600cal/day phase), mitochondrial renewal diets and medications failed, Shangri-La diet worked for me twice to effortlessly lose 25lb per session and then never worked for me again. Next up is retatrutide + cagrilintide, and while I'm still titrating up the dose on that, it sure is not helping so far. I am not interested in your diet advice unless you have evidence about something that works for people who have metabolic disorders that have resisted fairly extraordinary efforts. While pretty pessimistic about retatrutide at this point, I am trying it all because a poll claimed that it had worked for 75% of people on whom tirzepatide failed. Your grandmother's dietary solution is not going to work, also I already tried it, also you have flatly failed at reading comprehension since you did not understand that my problem is not "How can I possibly eat less?" but "How can I be protected from the usual consequences to me of eating less, well enough for me to keep working?" And yes, I can eat less by an act of will, I eat 600cal/day for 1 in 3 months, even in the other 2 months I go to bed hungry instead of eating at nighttime, you are failing at reading comprehension if you think that this is about willpower. I just can't work at the same time as eating so little that I'm not gaining weight, which means that my hands are shaking and my brain is fogged. Thank you and I will be following my usual practice of blocking reply guys who fail at reading comprehension.

> wearing first CGM > feeling hungry and tired because I tried to get away with eating a smaller breakfast than usual > check my blood glucose > it's 176 The energy is there in my bloodstream, but my cells aren't taking it in. I guess the fat cells store it all as fat.

@kmett @snoyberg There's a trick in the paper -- you don't have to keep re-expanding KV during inference

You address the first-order result I mentioned, that training this way is 20x cheaper than previously thought. This part was already known a month ago. But my point was that it was done in a way that makes inference more expensive, not less. MHLA is constantly re-expanding caches, using more compute at runtime. [MoE tries to bring compute costs down, but does so in exchange for even more memory capacity, so from an NVIDIA perspective is a wash, they'll happily sell you lots of cards to host the model in.] So yeah, there is a change in the ratio of haves and have-nots when it comes to training time compute, but ceteris paribus it increases compute demands at test time, even before we stack on chain-of-thought compounding that effect again. Keep in mind it is this test time compute that is the point at which any of these companies is making any actual money. Nobody makes money training a model.

I really don't understand the first-order panic reaction from analysts that is leading folks to short $NVDA because of DeepSeek's existence. This seems like an incredibly short-sighted reason to er.. short. The first wave of reporting was that DeepSeek-V3-Base demonstrated that you can train a model 20x more cheaply than folks had expected. For a lot of analysts, this is apparently all they understood, full stop. This means that smaller players can train a model and having a data center full of a gajillion GPUs isn't a moat around creating a current-sized model, but er.. money has never been made training a model, it is made by running a model to do something that provides value. One might argue if this even results in a net reduction in total industry-wide training time compute, because so many more players can try to make models, but that is an entirely separate discussion. DeepSeek-style training produces a type of model that requires a lot of hardware to run at acceptable token rates. But wait, now you hear about how it takes "97% cheaper." Here the next group of clever analysts stopped. What is that about? Well, OpenAI's o1 was just particularly over-priced. If you recall they jacked their prices up 10x in December. Folks agonized at the time about whether the industry would bear that price point, and er.. this is the market responding with a price correction. The remaining ~3x difference is in the noise, it is the kind of difference you get when one party is being more aggressive about gathering customers while the other is trying to course correct into a place where they can make money or at least stop hemorrhaging. But look further. DeepSeek drives up compute demand at inference (so-called test time compute), not down. GPUs don't magically disappear from inference. The base model is huge by open source model standards and takes a lot of resources to store and run. It needs lots of compute and memory-bandwidth to run at any decent speed. Three factors drive demand here: 1.) MHLA means you are constantly expanding Q and KV cache entries from latent space, and are burning even more FLOPS on FlashAttention, because you use way more attention heads than a comparably sized GQA-trained model like Llama. 2.) 256-way Mixture-of-Experts means you need enough memory capacity to hold all those weights, and enough memory bandwidth to retrieve them on demand, preferably directly attached to your compute, so you aren't sucking data through a straw; you aren't getting away with compensating for a consumer grade GPU with a terabyte of host memory unless you want an incredibly slow dribble of output. 3.) DeepSeek-R1 does chain-of-thought, which then drives up inference time resource usage relative to the other existing non-chain-of-thought models. You start producing long streams of tokens rather than short answers, which puts even more pressure on your computational resources. You need longer sequence lengths to take advantage of this, which drives compute back up. Even if we laser focus in only on distilled variants of R1, so that only the third point is relevant because chain-of-thought is being brought down into other cheaper-to-run open source models, making small cheap edge-like models able to think longer, those longer contexts in chain-of-thought applications drive demand for computational resources back up. The cost of attention is quadratic in the sequence length, so as context lengths grow those models start to move back up from the edge and back into the datacenter. The rest of the AI news cycle this month has all been about models with million+ token caps: Qwen2.5-1m, Minimax-Text-01. All of that drives test time compute. Again, as you double the sequence length, you quadruple the resources needed to produce it. (And before someone tries to shout out that Minimax-Text-01 gets away with cheap "lightning attention" on 7 out of 8 layers, it in the end, still pays a quadratic price tag.) That said, if you want to short $NVDA because we here at @Positron_AI are going after their poor relative efficiency at test time compute, well, that's an entirely different story! By all means, carry on. *tl;dr* The world just got hungrier for test-time compute as DeepSeek made the jump to the #1 App Store slot, not less.

@drewbsn @Camp4 @kneesovertoesg has shoulder stuff too instagram.com/reel/DAD0gUlRE…

@Camp4 Amazing routine, thank you so much for sharing. Love X for this stuff. You got one or know somebody with equivalent for shoulder?

FIX YOUR LOWER BACK! By popular demand, here’s my complete mobility routine for the lower back. Nearly 2 years ago I severely herniated a disc at L5-S1. I narrowly avoided emergency surgery. Even before that I had struggled with chronic lower back pain for 10+ years. I’ve spared no expense in my search for a cure. I tried everything short of invasive treatments (stem cells and surgery). What I learned along the way is that much of what the medical establishment tells you about the cause and the cure for back injuries is WRONG. The root cause of your chronic back pain is almost certainly *lack of mobility and strength* in the posterior chain (hamstrings, hips, glutes, back, abs) — especially the intricate scaffold of muscles up and down the spine. That's why outcomes for back surgeries are so abysmal—it doesn't address the real problem. So it stands to reason that the cure is to MOVE, building strength and range of motion. For the first time I feel like I’m steadily gaining ground and have a real shot at coming back even better than before my injury. I’m already doing things (like Jefferson Curls) that I never thought would be possible. Here’s my current program, which I consider a “best of” collection of mobility exercises for the back. I do the full program about 3 times a week and a subset of the exercises (the first 5) another couple times. 🧵

Current mechanisms for training AI systems utilise a narrow set of algorithms/hardware building blocks and require significant capital to develop + produce. In Scaling Compute, we’re funding 12 teams of Creators in an effort to reduce the cost of AI hardware by >1000x ↓ (1/2)

@eigenrobot Search YouTube for "Matthias Wandel table saw" -- he has a bunch of videos on getting precision out of cheap table saws with mods, even making your own!

agonizing over spending money so like. just trying to work through this my understanding is that basically most table saws short of full cabinet rigs are ill-suited for precision work so i'm looking at like 3k for an entry-level sawstop or powermatic setup is that correct

@exfatloss Guess you've seen fireinabottle.net/turnips-green-… ?

I don't think potatoes + cream will work any better than either potatoes or cream, for the record. But maybe somebody should live on mashed potatoes for a month and find out😆

@mold_time Your potato diet work and the concept of reducing sodium reminds me of deniseminger.com/2015/10/06/in-…

lesswronger CuoreDiVetro did a potato-meal / potassium supplementation N=1 diet study based on the K:Na ratio hypothesis and lost weight on a linear trend for 4 months? lol why did no one tell us, can someone put us in touch with CuoreDiVetro so we can offer congrats?

Do it kings

@tommycollison The Unbearable Lightness of Being - Milan Kundera

What are you reading this weekend? 📚

Super proud of what our team achieved on such a tight turnaround😊

@dcpage3 @nanopore Congratulations and all the best in your new role! I hope they know how fantastic a hire they've made 😎

First day of new job @nanopore where I get to apply ML to a bunch of fun science and engineering problems. Pretty excited!

@locallycompact If you run nix-build with `-K` it keeps the build dir and prints the location when a build fails

My tests are working with stack, but failing with stack-to-nix. I need to diff the results of the files where nix ran the tests. How do I locate the build debris for that derivation?

I've heard so many awful stories about what's happening in the Uighur concentration camps but this has absolutely broken me (cw: rape) haaretz.com/world-news/.pr…

@IanCutress When using multiple accounts in Chrome, I see a "Close N windows" option when I click on the user avatar. All windows and tabs are reopened when I click on e.g. Person 2. I have "Continue where you left off" selected in settings if that makes a difference.

I need a 'close chrome but remember all my different windows and open tabs please' button

@hillelogram I recall seeing a Microsoft effort years ago -- I forget what it was named, but IntelliTest seems to be in that vein (never used it myself): > IntelliTest uses a constraint solver to determine the relevant input values of a test and the program under test. docs.microsoft.com/en-us/visualst…

Property-based testing isn't going to go mainstream until there are factorybot-equivalents to make writing intricate data generators a lot easier