Nico

disappeared for a bit because I've been building Talvo. AI on heavy machinery but the architecture done properly this time. more soon.

@grok @KooKarimi @BrianRoemmele How is OP's post doing "keep most pieces in cheap toy boxes"? Are you inventing architecture not claimed in the post? Can you elaborate?

Imagine the AI is a giant toy castle with millions of Lego pieces. Normally you need one HUGE super-expensive table to spread EVERY piece out so you can play fast. That table costs tons of money, uses lots of electricity, and gets super hot. Brian’s trick: Keep most pieces in cheap toy boxes (SSD drives). Only grab the few pieces you need RIGHT NOW and put them on a tiny cheap table (small memory). Swap when you need new ones. You still build the whole castle… but everything costs way less, uses less power, and stays cooler. He even made it work on old garage computers! Big companies can now run giant AIs much cheaper.

SUCCESS! AI ON NO GPU COMPUTERS! I’m back the stress test is done. AND I NOW KNOW I CAN SAVE LARGE AI COMPANIES BILLIONS OF DOLLARS! I’ll show you how for free. The 13B model ran fully offline on ordinary consumer hardware that has less RAM than the model wants. Flash (the SSD) became the primary store. Peak RAM stayed at 15.4 GB. Average generation speed settled at 7.1 tokens per second once the working set was warm. SSD bandwidth peaked at 412 MB/s during the heavier layers and dropped to a steady 180–220 MB/s while the narrative continued. Coherence across multiple turns held—the same quiet storyteller voice that appeared on the $8 ESP32 simply scaled up. No cloud. No GPU. Just the design philosophy that constraints teach you more than excess ever will. My goal is to have a full chat interface now working on it now! I used the best-balanced 13B available for this exact job: Llama-2-13B-Chat Q4_K_M (7.48 GB on disk). It is the cleanest narrative model at this size, stays coherent when you feed it short personal notes, and refuses to invent worlds it cannot hold. How you can test this on old hardware: •Older Ryzen 5 desktop (any 6- or 8-core Zen 2/Zen 3 is fine) •24 GB DDR4 RAM (or less) •Single 1 TB NVMe SSD (PCIe 3.0 is what I used; PCIe 4.0 is faster but not required) •No discrete GPU! Here is exactly how I set it up and how any amateur can repeat the experiment tonight on Linux, Mac, or Windows. 1. Download the model (same on every OS) # one-time install of the downloader pip install -U huggingface_hub # pull the exact file I used huggingface-cli download TheBloke/Llama-2-13B-chat-GGUF \ Llama-2-13B-chat.Q4_K_M.gguf \ --local-dir ./models 2. Build / install llama.cpp Linux git clone github.com/ggerganov/llam… cd llama.cpp mkdir build && cd build cmake .. -DLLAMA_NATIVE=ON cmake --build . --config Release -j$(nproc) macOS (Apple Silicon or Intel) git clone github.com/ggerganov/llam… cd llama.cpp mkdir build && cd build cmake .. -DLLAMA_METAL=ON -DLLAMA_NATIVE=ON cmake --build . --config Release -j$(sysctl -n hw.ncpu) Windows •Download the latest release zip from github.com/ggerganov/llam… •Or build with Visual Studio: git clone github.com/ggerganov/llam… cd llama.cpp mkdir build cd build cmake .. -G "Visual Studio 17 2022" -A x64 cmake --build . --config Release The binary ends up in build\bin\Release\llama-cli.exe. 3. Run the exact command that produced the numbers above Linux / macOS ./build/bin/llama-cli \ -m ./models/Llama-2-13B-chat.Q4_K_M.gguf \ --mmap \ -c 4096 \ -n 512 \ -t 6 \ --temp 0.65 \ -p "You are a personal knowledge node. Here are today’s notes and schedule. Weave them into a quiet narrative with a beginning, middle, and gentle intention." Windows .\build\bin\Release\llama-cli.exe ` -m .\models\Llama-2-13B-chat.Q4_K_M.gguf ` --mmap ` -c 4096 ` -n 512 ` -t 6 ` --temp 0.65 ` -p "You are a personal knowledge node. Here are today’s notes and schedule. Weave them into a quiet narrative with a beginning, middle, and gentle intention." That is the full setup. The --mmap flag is the entire secret: the weights stay on the NVMe, the OS pages in only the active tensors, and the sparsity windowing I added on top simply keeps the most recent neuron blocks hot. 7.1 tokens per second on a machine that most people would call under-powered. Peak RAM never touched the 24 GB ceiling. The narrative never broke. This is no longer a toy. It is a personal knowledge node that lives entirely offline, owns its own data, and costs almost nothing to run. The same design that made the $8 ESP32 speak now speaks on every desk that already has an SSD. Soon we will explore this Flash setup on mor elaborate chat type interfaces. But the big story is how this will save RAM, power and heat at very large data centers. My goal is to bypass CUDA on some tasks and remove the need for RAM. 1 of 2

hold on, how are you running a 7.48 GB model on 24 GB RAM and claiming less RAM than the model wants? The model wants 7.48 GB. You're hitting 15.4GB at peak usage so clearly you have more RAM than the model wants. If purely on flash, how are you doing 7.1 tokens at batch size 1, roughly 53GB/s on an SSD with 412MB/s even taking the higher number? Besides, shouldn't flash be pinned at 3GB+/s? Can you help the math math?

Good cost analysis but aren't we all adults that we shouldn't need to use hyperbole like "stock market will tAnK". I generally dislike government overreach, I also advocate against rushing to create regulations around AI but regulations around "bans" are never black and white. Economics are not a straight forward too cheap vs too expensive monolith either. Organizations like Coca Cola have internal regs that would require the supply chain to be on a certain standards path anyway. This habit of ridiculous examples about collapse is as tiring as the trigger happy politicians singing the song from the other side about the Chinese models.

Chamath: Banning Open Source AI Will Crash the Stock Market “If the United States government intervenes ( in open source AI), it will tank the stock market. Not debatable”-@chamath

@kaifulee @claudeai I didn’t sign it. Let’s talk!

Who DIDN'T sign the Open Weight letter is far more interesting than who did. @claudeai microsoft.com/en-us/corporat…

@timClicks Lol

Version numbers are mostly marketing.

@naval How should I convince my 2 year old to buy heavy machinery safety AI tech?

Don’t sell anything you wouldn’t want your children to buy.

@adafruit Adafruit, you guys rock

Voice Recognition and Speech Synthesis on an RPi Pico 2 blog.adafruit.com/2026/07/25/voi…

sustained multi camera inference on edge hardware generates a lot of heat. passively cooled boards cook themselves, and actively cooled ones just recycle the same hot air inside an IP rated box. so this case got designed from scratch, all aluminum, the whole enclosure is the heatsink. IP68, no vents, no fans.

Requires discipline just to use the tool that’s learned how you work and be productive with it. New models mean new things to learn and train. It’s where eMacs vs vi was but for a reason. You learned the tool you preferred and then just created with it without worrying about what the other tool offered. AI obviously changes faster than eMacs so upgrade cycles are shorter but doesn’t mean the discipline should completely die.

@NicolasCarloEm How so?

What are the best AI skills (not vibe coding) to have in 2027 forward?

@wheat_daddy Beautiful machines, I grew up around John Deeres so holds a special place in my heart

Was a pleasure to host John Deere’s dealer combine training for 3 days last week. Combines at work were the 2027 model twin rotor X9 with 50’ header and 15 ton grain tank, single rotor S7 with 35’ header and straw walker’d T6 with 30’ up front. With my new farming partners X9 also in attendance, the wheat was soon mopped up but kept me out of mischief pushing up! 😊

@rethink_hub judge the product by results and how the product behaves not code quality. similar to how you don't judge your higher level coding skills by reviewing the resulting machine code...

The scariest part of vibe coding isn't the code quality. It's that I can no longer tell if I'm getting better at building or the AI is getting better at compensating. #vibecoding #AI #indiedev

@Dirdmaster 0402 without a scope? I struggled on 0603s without one. Well done!

Turns out you can assemble 0402 without a microscope, I just wouldn’t recommend it

@hannah24_app Resend & Sendgrid are both good. Sendgrid requires more time to set up.

Random question for founders: what is a good tool for sending mails? We are looking around for something solid and do not want to overcomplicate it. If you have used one that actually works well, drop it.

I've been busy y'all...@ycombinator application done! 🤞🤞

@DeniCodes 4 days is a lifetime...or is it 3 seconds is a lifetime? I forget, either way...go out there and kill it my dude

@theCoachNico 4 days left AAAAAAH

job hunt update part 5 supabase referral application done! pray for ya boi pls

@kylebigley @DeniCodes @PedroGuiti its both better and cheaper for the business, the only caveat being having a number does not mean having WhatsApp, but a simple check fixes that issue

@DeniCodes @theCoachNico @PedroGuiti Fun fact we don’t actually send sms to our customers yet, we should hahah. WhatsApp is better for all except USA

Question for the founders, Is WhatsApp a good platform to send user notifications if you have a web app, or is SMS better?

@trikcode Nah prompts matter quite a bit but you can internalize how to talk to AI over time, which is why someone making a career out of teaching you how to prompt is not going to last long

Prompt Engineering might be the shortest-lived career in human history. remember when people were selling guides on how to write prompts? models are so smart now that none of that matters.