hans

I'm excited to say my paper, Audio-reactive Latent Interpolations with StyleGAN, has been accepted to the Workshop on Machine Learning for Creativity & Design @ NeurIPS 2020! Blog: wavefunk.xyz/audio-reactive… Paper: wavefunk.xyz/assets/audio-r… Code: github.com/JCBrouwer/maua… Video:

Jamming with @cerspense on Sentinel. Built a 3D scene of points and lines, fed it through SDXL to turn it into jellyfish, layered a particle ripple system on top, finished with film grain. Everything vibe coded with Claude in real time. We described what we wanted as we listened to the music and just watched it come together.

@kettukaa @dnbt777 Wrote it up in more detail here wavefunk.xyz/audio-reactive…

@kettukaa @dnbt777 Yeah, I looked at a lot of higher-level spectral/musical features (onset envelopes, chromagrams, etc) and tried to map as many of those as possible to the different bits of styleGAN as I could

lil experiment injecting spectrogram data into stylegan3 latents fft(amplitude) -> log freq axis to get pitch energy -> rotate pitch energy vector over time -> w = w + energy(t)

@Ji_Ha_Kim @nhigham there's almost surely a "what is" post about this, but reverse lookup is hard haha

Is there a common name for the number tr(H)^2/tr(H^2) is cheap and very useful for lower bounding rank of symmetric matrix with nonnegative eigenvalues pretty precisely? It's helpful for predicting difficulty of polar decomposition

@ErwannMillon @torchcompiled Does torch.compile do these kinds of horizontal fusions? It is indeed just a specific form of grouped mm

Yeah I know I’m saying that with compile you get the best of both worlds Like you keep qkv split in the model state dictionary. compiler will fuse during fwd and backward. But because you pass the split params to the optimizer, the whitening will be applied separately to the q, k, and v

Something I haven't seen talked about much: Whether you fuse QKV or don't, and similar case for gate+proj fusion for swiglu, impacts how matmul-based optimizers function like muon, shampoo, etc

@SkopeUK YESSS!

I've been writing lots of filthy garage like this lately, anyone fancy a whole EP like this?

@cerspense has been developing a tool (Sentinel) that allows live diffusion. I have been playing with it in conjunction with TouchDesigner (for audio reactivity) and my Teenage Engineering OP-XY (for additional effects trigger). Being able to make animations on the fly based on a song feels like playing an instrument that makes visuals instead of sounds. It is awesome! This has been made possible thanks to my Dell Pro Max T2 tower equipped with an NVIDIA RTX PRO GPU (you can check out the spec here: lnkd.in/gQ-S9cF3). #DellProPrecision #DellTech #NVIDIA

Less subway map, more star wars...

Upcycling some old finetuned RuDALL-E generations with LTX-2

Love letters from latent space

Some autoregressive image generation from the hard drive archive I think it's either a finetuned RuDALL-E or CogView

@thomasmassena Very nice! Just fyi, some of your plots are pretty hard to read on mobile.

Wrote my first blog post, turns out exploring adaptive orthogonalization methods is pretty fun. massena-t.github.io/blog/2026/02/1… TLDR: Using an adaptive Newton-Schulz scheme allows a theoretically principled way to choose between SGD-esque or Muon-esque weight updates.