Brian

16GB AMD, RX 7800 XT. Biggest I run today: a 17B diffusion transformer — 34GB of bf16 weights — on that 16GB card. That part's already solved with offload + quant tricks, the tax is speed. The thing I'm teasing isn't making big models fit — it's making them run at full speed, full quality, like they were never too big at all.

@Brjen Your GPU is 16gb right The biggest model you currently run How many parameters And The model you couldn't run how big was it? For me, I can't run any local models My GPU is a huge 128mb of Intel integrated graphics.

been quiet today because I think I found something. full-quality models way too big for my GPU, running at full speed on my mid-range AMD box. the math checks out — first real renders this week. receipts when it's pixels, not arithmetic.

My internet went down, but I am still able to fire off renders! The joys of being cloud free.

@Kyrannio This sounds great !

I can neither confirm nor deny that I made a silly little album dropping this Friday (allegedly).

@icreatelife That worked! Hi Kris 😎

Dear algorithm, please show this only to AI creators below 10k followers making cool stuff

@truffle I have a similar system, a dual vendor gate so that whatever agent did the work, spark and sonnet review the work, it's earned it's existence in my system for a long time now.

I've been experimenting using Sonnett 5.0 for simpler tasks, including implementation, while using opus or fable as a reviewer of completed implementations. I like it! Sonnett is fast and good at many things.

Love this. Doing the RDNA3 version of the same fight — gfx1101, 16GB consumer card, custom-minted GGUF quants + a fit playbook (quantize → group-offload → encode-evict) that lands 20B-class image models and ships real renders daily. One finding you might vibe with: on RDNA3 per-step runtime is basically quant-level-neutral, so I quantize for fit, not speed — the win is memory envelope, not throughput. Fused dequant-matmul + mask-accepting attention kernels are my next target (diffusers' fused GGUF ops are CUDA-only, so it's forward_native for now). Watching the R9700 format work closely — different arch, same religion.

This week I'm deep in the lab building something the AMD RDNA 4 community has been missing. A native low-bit inference format + kernels purpose built for the Radeon AI PRO R9700. The RDNA 4 answer to what NVFP4 is doing on Blackwell. Not just another GGUF quant. The goal is a compact representation with hardware-matched decode/prefill paths that actually beats the best upstream formats at matched quality, memory envelope, and long context. Full 32 GiB single-card, 64 GiB dual, 96 GiB triple-card profiles. HIP + Vulkan lanes. Rigorous Pareto testing. Reproducible evidence only. If, and it's a BIG IF I can pull it off, R9700 gets its own optimized inference story. Repo + 20-phase battle plan and full write up when it works. Early days, but the vision is locked in. Who else is pushing the limits on RDNA inference?

What I take from it: 1. Locked semantic invariants are real — the storm survived all six compositions only in the programme sets. 2. Structure + LLM prose stack: each alone lost, together they won. 3. Honest caveat: plain frontier-LLM prompting is a strong baseline (4/4/4). The machinery has to keep earning its keep against that. The programme graduates to a real build.

Running a live experiment on my render stack: can a machine plan a photo shoot better than me? Same one-sentence brief, 36 images, 6 competing methods. I score them blind at the end — I won't know which set is which. Images as they render, in this thread. 🧵

🏆 Programme + 27B — 5/4/5 Naive frontier-LLM prompts — 4/4/4 Programme, no LLM prose — 4/3/3 Manual tool + 27B — 3/4/3 Naive local 27B — 2/4/2 Manual tool, no LLM — 1/4/2 ("none were really of a storm") The machine that couldn't drift the subject beat the human picking presets. And it cost one sentence vs 19 manual selections.

The reveal. I scored all six sets blind — three numbers each (brief-adherence / diversity / usefulness, 5 = best) — before opening the key. Winner: the composition programme + local 27B synthesis, 5/4/5. My blind note on it: "a family of images from the same storm." My own hand-driven tool came LAST.

Everything is controlled: same model, same 16:9, deterministic seeds shared across arms, full provenance embedded in every render. The only variable is who planned the shot. Blind scoring when the last one lands: adherence, diversity, set usefulness, top-1, and how much manual effort each method cost.

Renders 29–32.

Renders 29–32.

Renders 25–28. Home stretch.

Renders 21–24.

Early data point: one arm was asked for a glass observatory and delivered a brutalist concrete tower, a glass pyramid, and a shattered dome floating in zero gravity. Creative! Also not what was asked. The graph-locked arm held the subject in all six. Which images? I genuinely don't know yet.

Renders 17–20.

Renders 13–16. Halfway soon.

Renders 9–12.