DelocalizedDanny

812 posts

DelocalizedDanny

DelocalizedDanny

@DelocalizedD

🧙‍♂️🖥️:=⚗️⚛️🤖 Comp. materials scientist, PI #QuATOMs group @UHasselt, delocalized between #physics& #chemistry, gluing theory&experiment, torturing #DFT& #AI

Katılım Şubat 2020
123 Takip Edilen117 Takipçiler
BURKOV
BURKOV@burkov·
This paper really is groundbreaking. It solves a long-standing embarrassment in machine learning: despite all the hype around deep learning, traditional tree-based methods (XGBoost, CatBoost, random forests, etc) have dominated tabular data—the most common data format in real-world applications—for two decades. Deep learning conquered images, text, and games, but spreadsheets remained stubbornly resistant. This paper's (published in Nature by the way) main contribution is a foundation model that finally beats tree-based methods convincingly on small-to-medium datasets, and does so very fast. TabPFN in 2.8 seconds outperforms CatBoost tuned for 4 hours—a 5,000× speedup. That's not incremental; it's a different regime entirely. The training approach is also fundamentally different. GPT trains on internet text; CLIP trains on image-caption pairs. TabPFN trains on entirely synthetic data—over 100 million artificial datasets generated from causal graphs. TabPFN generates training data by randomly constructing directed acyclic graphs where each edge applies a random transformation (using neural networks, decision trees, discretization, or noise), then pushes random noise through the root nodes and lets it propagate through the graph—the intermediate values at various nodes become features, one becomes the target, and post-processing adds realistic messiness like missing values and outliers. By training on millions of these synthetic datasets with very different structures, the model learns general prediction strategies without ever seeing real data. The inference mechanism is also unusual. Rather than finetuning or prompting, TabPFN performs both "training" and prediction in a single forward pass. You feed it your labeled training data and unlabeled test points together, and it outputs predictions immediately. There's no gradient descent at inference time—the model has learned how to learn from examples during pretraining. The architecture respects tabular structure with two-way attention (across features within a row, then across samples within a column), unlike standard transformers that treat everything as a flat sequence. So, the transformer has basically learned to do supervised learning. Talk to the paper on ChapterPal: chapterpal.com/s/a1899430/acc… Download the PDF: nature.com/articles/s4158…
BURKOV tweet media
English
77
392
2.6K
331.4K
DelocalizedDanny
DelocalizedDanny@DelocalizedD·
Fortunately, no-one will ever see this post ;-)
English
0
0
0
8
DelocalizedDanny
DelocalizedDanny@DelocalizedD·
1984 update to 2025? Looks like a lot of people never read this classic, though it seems to be unfortunately relevant and all to realistic. Maybe time to so now. There is more relevant dystopian fiction (minority report and Gattaca come to mind.)... goodreads.com/book/show/6143…
English
1
0
0
17
DelocalizedDanny retweetledi
Fabio Cortes
Fabio Cortes@fabiojcortes·
Molecular dynamics simulations in mixed reality! With @labriataphd and @lucien_krapp we’re scaling our multi-user WebXR platform to enable immersive molecular simulations, fully based on web technologies. Stay tuned, lots of cool stuff coming soon! @threejs #WebXR #Quest3
English
51
598
2.8K
268.9K
DelocalizedDanny retweetledi
DFT2024 Paris
DFT2024 Paris@DFT2024·
Group picture is here! #DFT2024
DFT2024 Paris tweet mediaDFT2024 Paris tweet media
English
0
7
34
2.8K
DelocalizedDanny
DelocalizedDanny@DelocalizedD·
Big thanks to the organisers of #dft2024 for the opportunity to present our recent work on diamond. As usual it was a lot of fun presenting without a single equation 😈👨‍💻
DFT2024 Paris@DFT2024

Day 3 of #DFT2024 is ending! Let's thank all the speakers of the day for their exciting talks! See you soon for the poster session!

English
0
1
5
410