David Cardozo 🇨🇦

“Geography has made us neighbours. History has made us friends. Economics has made us partners. And necessity has made us allies." "From the fields of Flanders to the streets of Kandahar." Vive la Canada, Team Canada The Maple Leaf Forever

This algorithm uses one of my favourite theorems in math, the Johnson-Lindentrauss Lemma, which says you can drastically reduce the dimensionality of n points to just log(n) dimensions and still preserve pairwise distances

@himanshustwts GAN was coded in a night that too by a drunk Ian Goodfellow.

AI agents can act. But can you see why they acted? That gap makes trust and governance harder in critical workflows. At #NVIDIAGTC, we introduced Kiji Inspector™, a new open source framework for AI agent explainability, starting with NVIDIA Nemotron. dataiku.com/stories/blog/i…

NVIDIA just announced Nemotron 3 Super, a new open model for agentic AI at scale. Dataiku is proud to be part of the ecosystem helping enterprises operationalize multi-agent systems. Learn more — and see us at #NVIDIAGTC booth 3108: blogs.nvidia.com/blog/nemotron-…

Diffusion models clicked for me when I started seeing them through the lens of particle motion. I built this interactive playground where you too can clickety-clack to understand how drift, noise, and other hyperparams control diffusion. I hereby submit this as penance for the sin of YouTube edu-tainment 😇 Link in the first comment.

is grain and arrayrecords the new tfrecords? I cannot believe that in the era of LLMs google still cannot produce more than this post: developers.googleblog.com/building-high-… for grain and arrayrecords :(

Tailscale is my most Canadian proud company. Best tool ever

I just got a new job where I got to play with a gb200, missing Jax but now I have compute to do cool stuff

i love jax and tpus, i think they are elegant and fit my mental model of how compliers and systems should work. that said, you'd have a _very_ hard time getting me to go back to using jax and tpu instead of pytorch (really cuda and CuTeDSL) and GB300NLV72s

Here I stand, I can do no other.

A guide of Pallas on TPU SparseCore is out! Things could still be wacky, but these are the paths we've cleared out, and more to come soon. Have fun playing with it! docs.jax.dev/en/latest/pall…

a beauty for anyone interested in mechanistic interpretability or getting into LLMs. interesting to look at small algorithms and their "neural implementations" to get a sense of how neural nets implement various functionality. unless the minification really creates "esoteric" solutions that you wouldn't encounter in practice, which might be more based around distributed representations, helixes etc. i tried training the same arch briefly from scratch and gradient descent didn't find the solution, would probably work with more degrees of freedom and enough effort.

How come Google does not provide an Chrome arm64 package in linux.....

Introducing 575 Lab: an open-source initiative for production-ready AI tooling. As AI systems take on greater autonomy, trust must be earned in production, giving teams visibility into AI behavior and the controls needed to govern it at scale. Join us: dataiku.com/open-source/?u…

1/ Transformers don't count like computers. We assume they have hidden "registers" to track variables. We were wrong. New research by @AnthropicAI reverse-engineered Claude 3.5 Haiku and found it works with 6D helical manifolds. It's geometry, not math. 🧵

this is now officially a GB300NVL72 fan account. sorry in advance

@RisingSayak or the test suits, sometimes I have learned of upcoming features by looking at the tests.

I think going through an important PR of a repository like PyTorch, even as an external observer, is an under-appreciated learning resource. I learned how to test for recompilations from one of those PRs and it has been a great debugging tool since then!

Today I just renewed my subscription to @che_shr_cat this is the way I keep up with arxiv. If you are an MLE or AI practitioner this is a must, and I really like

@ariG23498 Amazing material

The YouTube lecture on Mixture of Experts (MoEs) Routing Algorithm is out now! Here is what I cover: > Foundation of MoEs (MoE) > The routing algorithm > A toy problem > Issues with routing (oversub experts) > Conclusion Please swamp me with feedback. It is a big video, so if you don't have time this week, do not worry. Bookmark this, and come back to it in the weekends. A cup of coffee, a lazy saturday/sunday morning and this video 🤌🏻❤️. A fresh brain would easily grasp every concept and also tell you that you could have understood the concepts yourself. Link: youtu.be/CDnkFbW-uEQ

Some notes on what someone can do for building chops in ML x open source x modeling: • Take a popular pre-trained model implementation, profile it, spot the bottlenecks, & try to improve its speed-memory trade-off -- it's a valuable skill that any sane hiring manager should understand and credit (they are probably not legit if they don't). github.com/meta-pytorch/s… is a good example of this. • GPUs are in short supply. So, try reimplementing it in JAX, leveraging its strengths. Make it run on TPUs, blazing fast 🔥 -- this will help you establish that you care about performance and are comfortable switching stacks when needed. github.com/sanchit-gandhi… is an amazing example of this. • In the context of an organization, communication is the key. Make sure you document your experience in an easily digestible way so most folks would understand what you achieved. Provide numbers on benchmarks, mention assumptions, and whatever limitations you faced and how you approached them. • Get a pro subscription to whatever AI coding assistant you think works the best for your stuff. Make it a part of your workflow, but DO NOT become overly reliant on it. Have enough juice in the process so that you can build muscle memory and objective evidence of your intellect over time. • Have fun! It gives me a sense of joy and relief to know that back in the days, we did all of it happily WITHOUT any AI coding assistance. Lots of fun, despair, and anxiety -- but all worth it; 10/10 -- would do it again!