Abhinav Bhatele retweetledi
🧠 From early programming lessons to scaling #AI on supercomputers, Abhinav Bhatele (@bhatele) shares his path and research at @UofMaryland's Department of Computer Science.
Read more: go.umd.edu/Bhatele-4-2026
English
Abhinav Bhatele
502 posts
Abhinav Bhatele
@bhatele
Prof @UMDCS and @umiacs, Lead Parallel Software & Systems Group @hpc_group. @IITKanpur & @UofIllinois Alum. Ex-@Livermore_Lab. Views are my own.
Katılım Haziran 2009
275 Takip Edilen681 Takipçiler
Abhinav Bhatele retweetledi
Congratulations to Dr. Joy Kitson for successfully defending her PhD dissertation last Friday (April 10)! That makes her the fifth PhD student to graduate from our group.
Dissertation title: "Scaling Agent-based Epidemic Diffusion on HPC Clusters"
English
Congratulations to Dr. Cankur and Dr. Davis! Best wishes for a bright future!!
Parallel Software and Systems Group@hpc_group
On Mar 25, we set a personal record with 2 PhD defenses in a day! Congrats to @onurcankur & @jhdavis_josh for their excellent work: Onur's title: "Longitudinal Data Analytics of HPC Systems & Applications" Josh's title: "LLM-driven Development of Performant & Portable GPU Codes"
English
If you are interested in collaborating with us or you are an undergrad planning to apply for a PhD, come and talk to us (don't feel shy!). You'll find me or PSSG students at the events below. #SC25
Parallel Software and Systems Group@hpc_group
We have arrived in St. Louis for @Supercomputing 2025. Learn more about our group's research through the various talks, panels and tutorials below. We will also be at the @UofMaryland booth (3123) at various times. #SC25 #UMDCS #HPC #AI #MLSys #AI4Science
English
Abhinav is a solid researcher and a great PhD advisor. Work with him if you wanna get into GPUMode! :)
Abhinav Bhatele@bhatele
A large number of PhD students in my group have graduated or will be graduating by Spring, so I am recruiting several PhD students for the next admission cycle (Fall 2026). If you want to work with us, apply by Dec 5 and drop me a short email. Please repost/share widely. #HPC #AI
English
Abhinav Bhatele retweetledi
Fine-tuning massive LLMs requires reliability at scale. The Lightning open source stack gives you just that.
Check out Democratizing Al: Open-source Scalable LLM Training on GPU-based Supercomputers.
✅ 405B param MoE language model
✅ Fully finetuned on 32,768 GPUs on a massive 1.4 exaflop/s cluster
✅ All running on LitGPT
Led by @siddharth_3773 and @bhatele at @UofMaryland.
Read the paper ➡️ go.lightning.ai/44EjLFu
Use LitGPT ➡️ github.com/Lightning-AI/l…
English
Congrats @siddharth_3773! Siddharth is going to join NVIDIA after he graduates to continue working on parallel training and inference.
Follow him to keep track of what interesting stuff he’ll do next 😊
Parallel Software and Systems Group@hpc_group
We are on a roll, second successful dissertation defense in a week (March 28)! Congratulations to @siddharth_3773 on becoming the second PhD graduate from PSSG!! Dissertation title: "Optimizing Communication in Parallel Deep Learning on Exascale-class Machines" #HPC #AI #HPC4AI
English
Abhinav Bhatele retweetledi
Riken, including our center R-CCS, is hiring researchers big time, from PIs to postdocs to interns. We have the best support for international researches; in fact the % of non-JP researchers at R-CCS is approaching 50 % now. Come work with us! riken.jp/en/careers/ope…
English
Incredibly proud moment for me as an advisor. The first cohort of students in @hpc_group started in 2020 and they are now starting to graduate. Also a bittersweet moment as we bid adieu to some senior students this Spring & welcome new ones in the Fall. Congrats @DanielNichols10!
Parallel Software and Systems Group@hpc_group
We present our very first newly minted Dr. Daniel Nichols (@DanielNichols10), who successfully defended his dissertation today. Congrats & best wishes for a bright future ahead!! Dissertation Title: "On Learning Behaviors of Parallel Code and Systems Across Modalities" #HPC #AI
English
Abhinav Bhatele retweetledi
We have a postdoc opening in our group on the intersection of HPC and AI, specifically on developing and applying Code LLMs to HPC software development. Please help us spread the word:
umd.wd1.myworkdayjobs.com/en-US/UMCP/det…
Position will remain open until filled. #HPC #AI
English
Abhinav Bhatele retweetledi
Poster session in progress … if you miss us and want to chat, you can find more information on pssg.cs.umd.edu.
English
Abhinav Bhatele retweetledi
Come see our #NeurIPS2024 poster today on"
Loki: Low-rank Keys for Efficient Sparse Attention"
Paper: arxiv.org/abs/2406.02542
Code: github.com/hpcgroup/loki
Loki is a sparse attention method that reduces the computational and memory costs of LLM inference! By exploiting the low-dimensional space of key vectors in self-attention, Loki achieves faster attention computation without compromising model performance.
Parallel Software and Systems Group@hpc_group
Can we get away w/ reducing attention keys to a lower-dimensional space to optimize compute during inference? @prajwal1210 & @siddharth_3773 investigated using PCA on key vectors & found that the rank of attention keys is much lower than the full dimensionality. #NeurIPS2024
English
@prajwal1210 will be presenting this work today (Dec 13) @NeurIPSConf in the poster session from 11 am-2 pm in East Exhibit Hall A-C (Poster# 2000). Come and say hi!
#NeurIPS2024 #HPC #AI
Parallel Software and Systems Group@hpc_group
Can we get away w/ reducing attention keys to a lower-dimensional space to optimize compute during inference? @prajwal1210 & @siddharth_3773 investigated using PCA on key vectors & found that the rank of attention keys is much lower than the full dimensionality. #NeurIPS2024
English
Abhinav Bhatele retweetledi
Why is addition hard for next token predictors? Come hear about our fix for this at #NeurIPS!
We’re presenting Abacus Embeddings 🧮 tomorrow (Friday) from 11-2 in the East Exhibit hall at poster 2907.
Drop by to see how we can improve your language model.
Sean McLeish@SeanMcleish
Introducing 🧮Abacus Embeddings, a simple tweak to positional embeddings that enables LLMs to do addition, multiplication, sorting, and more. Our Abacus Embeddings trained only on 20-digit addition generalise near perfectly to 100+ digits. 1/n
English
Abhinav Bhatele retweetledi
🏆 Researchers from the @UofMaryland received the HPC Innovation Excellence Award from @Hyperion_HPC for their work on AxoNN—a scalable framework that’s pushing the limits of Large Language Model (LLM) training! 🚀
💫 bit.ly/3CsmjKM
🏆 bit.ly/3V9HFTw
English
Abhinav Bhatele retweetledi
Congratulations to the winners of the #HPC User Forum Innovation awards at #SC24
o Siddharth Singh and Abhinav Bhatele
o Eric Nielsen
o Nils Helset
hpcuserforum.com/innovationawar…
English
Abhinav Bhatele retweetledi
A team of parallel computing and machine learning experts from @UMDscience and @umiacs has been named a finalist in a global competition recognizing outstanding contributions in high-performance computing.
The team—led by Abhinav Bhatele and Tom Goldstein—is one of six vying for top honors in this year’s ACM, Association for Computing Machinery’s Gordon Bell Prize competition.
umiacs.umd.edu/about-us/news/…
English
Abhinav Bhatele retweetledi
Hyperion Research Announces Winners Of 2024 HPC Innovation Excellence Awards
ow.ly/cBuC50Uc6Qt #HyperionResearch #HPC #TCIwire
English