Evgenii Egorov

An interlude about structure computation, SSM and attention. Myosotis: arxiv.org/abs/2509.20503 I hope to tell more in this line of work later. See poster at SPIGM workshop.

A new way of doing. If link is established, but computationally algorithm doesn’t change only names, than there is no new knowledge. Example: if I say that linear system solving is the instance of probabilistic inference: finding marginals of Gaussian, than there is no new knowledge I just renamed Schur complement but will do all the same states. But if I add and hence on can use sampling for this problem – this is new knowledge, algorithm is different.

@eeevgen @norpadon Why not? What is a discovery?

The main goal of Bayesian ML research is to show that all methods which have previously been shown to work well in practice are somehow approximately Bayesian

@hr0nix @norpadon My point is with out this step this “something is something” is not a discovery.

@eeevgen @norpadon The point is that, sometimes, when you discover a non-obvious connection of a method to some concept, it might hint at a way to improve the method even further. Worked well for diffusion.

@zhaisf I have a note where describe flow matching, but was thinking that who needs this if we have BigGAN

Found this half page note I wrote ~6 years ago. Describes basically linear attention but half a year before the “Transformers are RNNs” paper came out. Sadly I didn’t take it too seriously at the time because I didn’t have any use cases for it and was also too busy with GANs.

I guess everybody know this but for same snails as me: while neurips.cc doesn’t have 2026 year, direct neurips.cc/Conferences/20… works

@nblqbl Anna actually did not long ago reasoning with latent states, without decoding in prompts arxiv.org/pdf/2510.02312 , also I think some works in similar lines was from meta. In principal than you can try to also make this latents more explicit memory, not only for faster inference

@eeevgen right! but in your eyes is this "old-fashioned" continual learning continuously connected to trying to make an LLM that has a genuine persistent memory instead of cobbling together a bunch of stored prompts? i imagine the latter problem is still a big one?

as part of my ongoing education in ML, i've been reproducing for myself basic phenomena in deep learning. in case they help other ML-curious people get started, i've decided to start writing blog posts on my investigations. (link below). first: catastrophic forgetting, or --

I think it was one of motivation of switching architectures. Like it was more rigid parametric architectures and CL was more about replay/weight penalties. Than people realized that better way is a “functional view”, so there are a lot of papers with ideas of something like support vector points from SVM but for neural networks also more like Gaussian process point of view. And this is already quite close to attention-like and KV-cache etc

@norpadon @acidglxtter Ну ладно, у sigraph +- тоже

@acidglxtter Какая же охуенная типографика! В первый раз такое вижу в журнале. Надо будет спиздить шрифты

> The paper is published in the journal Publications mathématiques de l'IHÉS мне этот журнал всегда внушал опасение и уважение. если бы я был аспирантом и мне надо было что-то прочесть и понять оттуда, я бы испугался сначала. а статья прикольная. link.springer.com/article/10.100…

I think field switched to different paradigm with large pretrain models + lora, so I don’t think it is a problem anymore. Also transformer layers are a bit different, more close to non-parametric things. So for me looks like CL is a bit dead 😵 on the other hand, any autoregressive net is kind of CL…

@eeevgen ooh nice! will read it carefully, i feel like this is the principled way to update the VAE that i was looking for. are you thinking about continual learning in general these days?

@nblqbl We did a bit about how to train vae in this fashion with latent memory proceedings.neurips.cc/paper/2021/fil…

the fact that neural networks generally forget how to do old tasks when trained on new ones. i studied this and tried to fix it in a toy benchmark, in a way that is probably not very efficient, but the most fun. i used hopfield memories and a VAE. open.substack.com/pub/nabiliqbal…

Хотелось бы поесть борща и что-то сделать сообща: пойти на улицу с плакатом, напиться, подписать протест, уехать прочь из этих мест и дверью хлопнуть. Да куда там.

@norpadon you excluded me, I will take this personal

Btw this my personal criterion of AGI: if a system can take a *description* of a language as an input and become fluent in that language “Language” here includes things like programming languages or mathematical objects

One thing that can entertain a person forever is a mirror.

On this International Women’s Day, we celebrate the incredible contributions of our women who help shape Russian Chess School every day. In a male-dominated industry, we’ve built a top-notch product together, and this is just the beginning!

Somehow, on the Dutch society knowledge exam, there were no questions about either Erasmus of Rotterdam or Benedictus de Spinoza. Nevertheless, I passed, but I was disappointed.

@materion Can you describe please

Golden rule: when using visual proofs of algebraic identities, never forget the circle. Example for 1/2+1/4+1/8+1/16+1/32+...=1

@maxxxzdn Example

Starting to review GDL papers with obscure branches of math applied just for funsies and seeing Policy A (Conservative)

@norpadon 1. Quadratic form with preconditioners, so white with it top-k from svd unweight back 2. power of 2, but what is not too big not too small idk 3. some quick select with buckets Conclusion – I don’t know algorithms in gpu

Some fun ML interview problems

Imagine that “Europe” made several incredible huge startups in AI. And where will they do their IPO? :)

@srboljubbosanac @maxxxzdn Capital by USA and first author by USA en.wikipedia.org/wiki/John_M._J…

@eeevgen @maxxxzdn Most authors of AlphaFold2 are European and it was created in London, UK, Continent of Europe.

I find the argument of Europe lagging behind the US in AI overly reductive. Over the time, the focus of Europe clearly shifted towards AI4Science, where its lead over the US is comparable to the US's lead over Europe in LLMs (AlphaFold, GenCast, ML force fields, equivariant networks -- to name a few). It's not obvious to me what will bring humanity further in the long run, very likely the combination of both, hence we should keep collaborating and try to build a better world together.