Sabitlenmiş Tweet
New followers.
No researcher is to be found here. I am a novel capabilities engineer doing random walk.
(read: unhinged)
English
Adam Hibble
3.8K posts
Adam Hibble
@Algomancer
I generate models that generate other stuff, working on @mancerlabs -- Prev: Founder of Popgun Labs (Techstars), Founder of the @QUTCode Network.
Katılım Ağustos 2013
1.1K Takip Edilen4K Takipçiler
This is cool because you could use flows with expensive inverses like MAFs since you’re throwing them away after training
English
I have been thinking about the measure of how far away a transform moves an input from its 'automorphism group'.
Intuitively, this would seem to specialise (in expectation) to negentropy / expected log-det Jacobian.
vaguly, it'd result in flows that act more as a gaussanity tests/regulariser, where the 'goal' is to starve the flow, ie isometry log-det -> 0, singular values -> 1
English
I don’t know the upper bounds but i have comfortably handled 16 nodes solo, more of a budget limitations - but I understand I’m more comfortable with operations than most so I think I’m mostly going to tap out on gpu faults. Imagine anything beyond I’d probably need to think more about topology, but assuming no physical faults I can usually figure it out. I will say tho I’m perpetually compute poor so I’d hedge beyond that.
and tend to work entirely solo so don’t have much ‘other people overhead’ wrt to job scheduling. Pref just to have root.
English
gonna revisit this, i really liked the split head formulation for compute reuse.
Adam Hibble@Algomancer
A Ladder of Heads Varient of Free Transformer
Each transformer layer contains a binary VAE: where the "Encoder" and "Decoder" live inside the same Transformer layers, separated only by attention masks and head splits. Decently efficient thanks to flex attention.
First N heads: bidirectional (encoder) — sees full context for posterior
Remaining heads: causal (decoder) — computes prior from z_{
English
artifact from a discord chat
wake-sleep Bayesian program learning in one PyTorch file. Integer arithmetic domain. Joint inference over a typed contextual PCFG and a neural recognition model that amortizes the posterior over programs given I/O examples.
basically a nano dream coder
github.com/Algomancer/bay…
English
me: pull out this into a single .py for github
claude: got it im changing all your numerics for no reason
English
quietly good paper for anyone training discrete latent variable models. lower variance gradients for discrete latents, with a nice reframing of the whole estimator family. this one rao-blackwellizes the noisy bit and patches the leftover bias with a control variate.
Did a quick implimentation here and a mapper layer for vaes like free transformer.
github.com/Algomancer/Bet…
English
sparsemax core. tropical-pilled. λ-maxxing from hard max to mean. real zeros fr. fused kernel no thoughts head empty. vaguely trains. idk had needs
github.com/Algomancer/mor…
English
Combine with this, x.com/Algomancer/sta…
and have fun.
Adam Hibble@Algomancer
there is something really magical about giving an agent access to the dom directly setup a js term, embed it, open claude, give it an api it can curl to to execute js - pretty fun. minify an app to a single file, insert it on the fly via curl. I think if this becomes like, less of a security nightmare its totally a thing.
English
Okay this in unhinged.
But you can add a hook, that fakes being a agent team mate and make your application interface be on an agent team.
gist.github.com/Algomancer/035…
English
New followers.
No researcher is to be found here. I am a novel capabilities engineer doing random walk.
(read: unhinged)
English
there is something really magical about giving an agent access to the dom directly
setup a js term, embed it, open claude, give it an api it can curl to to execute js - pretty fun.
minify an app to a single file, insert it on the fly via curl.
I think if this becomes like, less of a security nightmare its totally a thing.
English
This is my new favourite way to start a project.
Self-bootstrapping Claude app. Three files, one loop. Tell the agent to modify itself. It will.
github.com/Algomancer/cla…
English
I said this a while ago, but dude im so keen for code conditioned realtime video generation. We are close.
react.js? Nah, just bash pipe to pixel space.
Adam Hibble@Algomancer
I think not enough people are thinking about what high quality, long context video generative models mean in the longer term. It's not just generative Netflix. You're going to have generative video applications on your generative operating system, everything you see on your screen can be generated. Including chat interfaces.
English
we went from memory is cheap so just write it in node.js to a ram shortage
computing is cool again
English
One thing is for sure, i love the models, but it's really impressive how janky all cli interfaces are.
English
