Dan Olson

My upcoming book "Rust for C++ Developers" is now available on Amazon: amazon.com/dp/1836206518

I’m seeing a lot of AI doomerism all at once. I try to remain pragmatic and stay away from the poles of polarizing discussions but the doomer one for some reason just really rings hollow with me.

@jamiehayes81 Maybe warbreaker. Enjoyed it a lot.

@olson_dan I went through mistborn and am currently on wind and truth and I think I’m done, so I appreciate something might pull me out of a similar funk

I was a bit burned out on cosmere and fiction in general for a few months, but Tress of the Emerald Sea is pulling me out of it. Looking forward to getting deeper into Expanse-adjacent survivors’ war vol 2 as well.

YouTube is infested with political ads these days and each one I see makes me like the candidate less.

I’m reading about inference engines and speculative decoding sounds like a really unlikely optimization to me. But in practice it’s very effective meaning that I have massive gaps in my mental model of how inference works.

@saujasn_ I don’t believe so, but it does depend on domain. A lot of programming is entirely asynchronous now.

@olson_dan would assume fast feedback loops still matter, no?

I’ll say something stupid. After decades of programming language theory, only two things matter going forward: runtime performance and strong correctness guarantees.

@banisterfiend More or less important depending on domain (especially embedded). Compilation times and library availability both made less important by AI but still nice to haves.

@olson_dan What about binary size? Compilation times? Library availability? Ease of distribution? Memory usage? Etc

The working assumption I try to hold to is that no problem with AI coding that exists right now will be solved by waiting for better models.

I agree that people treat mediocrity as victory laps all over the place. And it bothers me when I see it. But I wish you could believe that I am aware of and understand many or most of the tradeoffs around these issues. Speeding up a python program by 10x is valuable work even if it could be sped up by 15x or 100x with more work. Not everything is critical path and Rust has better defaults than most things around in the spaces in which I specifically work.

That’s not a high bar you’re measuring against. We’re building a safety-critical language that’s multiple orders of magnitude faster than anything you’ve ever done in Rust. Especially in the worst-case. Too many people are fiddling around the edges and then applauding over tiny improvements that don’t even try to address the primary bottlenecks. It makes me want to facepalm so much.

I’m thinking some sort of setup where you specify preconditions like inputs (eg. property based tests) and a window of acceptable bench results so the code doesn’t even compile if an ai agent makes it slower. The idea is over engineered from the start (no reason it has to be in compiler vs in test suite) and eliminates cross compiling but I still like thinking about potential directions here.

That only works well if you assume the computer compiling the code is very very similar to the computer running the code. We built a JIT compiler inside of Rust that can execute on GPUs and CPUs. We have autotune to perform micro benchmarks at runtime, and for the CPU we might even activate an LLVM micro benchmark pass to generate better machine code for the current hardware.

At some level there is a correctness vs optimization tradeoff and rust picks a good default balance, but I think it’s valid to criticize that default balance on being allocation heavy among other things. I won’t get into the weeds on this but I observe that even with that default balance it speeds up programs written in every other language stack I’ve thrown at it.

@olson_dan I agree with your motives but Rust is terrible at both.

I agree with you on the root causes. Basically too much code expansion hitting llvm is what seems to be the bottleneck. Plus probably procedural macros deep in the crate hierarchy. But as a thought experiment an increase in compile times should be acceptable in the agent loop, especially if you make eg. multi lang implementation stacks. (High level spec with formal verification generating low level language code, for example.)

I don't think the correctness of Rust is what increases compilation time. It's the zero cost abstraction that assumes any type may be inlined, reducing the ability to reuse compilation artifacts (like .so). With a small experiment, I was able to reduce compilation time from 12s to 0.4s by just making a type opaque to the compiler

Everyone is so worried about AI breaking code. Yeah. But what you should have been worried about for the last 60 years is humans breaking code. We have not done well enough with these areas because other things looked like higher priorities.

I’ve seen AI optimize a program to 1.5x improvement over a spec program while keeping output identical. It’s a thing. But it is now just about tightening the feedback loop and making invariants more accessible to the AI.

Apple Music automix is one of the most obnoxious defaults I’ve come across in recent memory.

Code mode as coined by cloudflare means the model writes typescript (or js, or python) and the harness executes it in a sandbox. I have local stuff that does this with deno. The trade offs are great for mcp but less clear for agent harnesses generally. I still think it’s a good idea there though.

@olson_dan @dillon_mulroy I'm wondering about doing my own harness, since every one I've looked at seems to revolve around bad ideas and vibe coded; I'd rather have tools solely for coding, not full on command line access. What would "code mode" actually be tho?

the most annoying behavior of 5.5 is that it regularly starts editing files with python instead of its edit tool