Sergei Lebedev

Want to improve GPU compute/comms overlap? We just published a new short tutorial for you! A few small changes to the Pallas:MGPU matmul kernel is all it takes to turn it into an all-gather collective matmul that overlaps NVLINK comms with local compute: docs.jax.dev/en/latest/pall…

@WCelhen @apaszke Curious what you are looking to do with SparseCore. Can you share more?

@apaszke 🤣 Was looking at tpu sparsecore with jax and couldn't find any tutorial for it (apart from a repo with no docs🥹)

Curious how to write SOTA performance Blackwell matmul kernels using MGPU? We just published a short step-by-step tutorial: docs.jax.dev/en/latest/pall… At each step, we show exactly what (small) changes are necessary to refine the kernel and the final kernel is just under 150 lines.

@AliesTaha Thanks for sharing, Ali! Can you link to the kernel and benchmarking code, please?

We beat Nvidia’s cuBLAS kernels on B200s in 170 LOC. Using zero CUDA. Just pure Mojo. Here’s exactly how we went from 1% to 106% of Nvidia benchmark perf from scratch (with code) 👇🧵

@gallabytes @sharadvikram @finbarrtimbers @IlyasHairline @_xjdr OOC what kind of documentation are you missing most? Is it GPU-specific guides, API docs, "the sharp edges", something else?

@sharadvikram @finbarrtimbers @IlyasHairline @_xjdr both, tpu situation is definitely better but I highly doubt you've documented all the little things that *don't* work in pallas.

the performance of jax + GPU is abysmal compared with an equivalent pytorch implementation. This is sad but unsurprising. jax might be relegated to TPU only (for me) for a little while longer. that said, there is nothing that touches jax + TPU for large scale perf (405B bby)

I've finally landed my first proper JAX feature since joining the team: a supported "foreign function interface", which makes it easier to call into external libraries from within JAX code. Check it out: jax.readthedocs.io/en/latest/ffi.…

@youlovegriszka получается, это уже традиция

посмотрел на календарь и выяснил, что между событиями из твитов выше прошло ровно два года

уже сутки не могу перестать угорать с того факта, что в воскресенье вечером сам себе по лицу съездил, причём сочно так, прям кулаком в глаз

Many of you are excited about H100 attention, so it’s a good time to show you Mosaic GPU: a Python DSL for H100s. The attention example matches FA3 performance, while being only ~200 lines of Python: #L146-L354" target="_blank" rel="nofollow noopener">github.com/google/jax/blo… It's easy to install too! Latest JAX packages have it.

Our team is looking for a strong research engineer, hardware background is *not* obligatory. Please, share and recommend someone! boards.greenhouse.io/deepmind/jobs/…

@DrLangstrand Whenever there is a LaTeX alignment/spacing problem, the answer is usually minipage and its friends hspace and vspace :)

@vincenthz @ChShersh Curious to hear what crufts you came across?

@ChShersh The honeymoon phase will pass for you. Best to reserve judgment until you actually use the thing for many years, and see all the crufts. (Source : done many years of OCaml personally and professionally)

Choose 👏 OCaml 👏 today 👏 The hype might end. But the amazing language will stay with you forever.

@mitsuhiko @robsmallshire That's sadly "by design". PEP-484 decided that int should be a subtype of float, even though it isn't at runtime...

@robsmallshire Numbers are all weird in mypy :-/

On the topic of Python typing: One of my biggest qualms with the system is that once you are past having everything typed, it lets you do stuff like this and does not even complain.

@mitsuhiko I think a lot of the ergonomics were sacrificed to the idea of annotations being used for things other than types. So, most typing features are designed to abuse existing syntax to the benefit of nobody.

Python typing makes my blood boil. How the hell does anyone enjoy that experience?

The trilogy is complete! My "Advanced Topics" book is officially released today. Buy it on Amazon, or get it for free at probml.github.io/pml-book/book2….

Friend reminder that it’s all regression twitter.com/zfurnas/status…

Are you a PhD student interested in the interface between generative AI, LLMs and audio? Our team at Google behind AudioLM, MusicLM and AudioPaLM is looking for a talented student researcher! See details and apply google.com/about/careers/… and send your cv to bmcw@google.com.

@yminsky @ezyang You can use it in CI as well, though, right? You don't have to use mypy there.

@ezyang Have you considered pyright? It's supposed to be faster, but people mostly talk about it as something to use with vscode. It seems weird to me to use a different checker in the editor than in CI, but this seems like normal behavior in Python-world.

So...does anyone have advice for picking between the various and sundry Python type systems? mypy, pyright, pyre, pytype...how do you pick?

@yminsky The Python type system has no spec, so I can guarantee there will be plenty of things they disagree about. Pyright is also usually faster to adopt new type system features, so it's possible to get type errors simply because mypy doesn't fully support some feature (yet).

Sounds like lots of people use mypy for CI, and pyright for the editor (because, VSCode, and mypy isn't zippy enough). But...is using two linters really the right thing? Does anyone have experience using pyright for CI as well as the editor?

@yminsky Is there a way to learn more about what Jane Street is doing in this space without applying? :)

Re-upping this. The Python dev-tools team has grown since I posted this in December, but we're still looking for people with experience building a delightful developer experience for Python. Apply here! janestreet.com/join-jane-stre…

OCaml wins the 2023 SIGPLAN programming languages software award! 🎉sigplan.org/Awards/Softwar…