Soheil Kolouri

@hyhieu226 While studying LLM compression, we also confirmed that the gradients are indeed low-rank with often very small rank. (ZS-SVD ICML 2026) mint-vu.github.io/ZS-SVD/

Big if true. Have we investigated gradients of the Transformer? I did. They are sparse (in certain ways), so exploring them in the low-rank world makes sense.

Iran’s internet is shut down. Silence is how mass murder happens. Support the people of #Iran. Speak out now. #DigitalBlackoutIran

@HamidRezatofigh 😂😂😂

@SKolouri 😂😂

Anyone from #iran looking for a phd/postdoc/research internship in statistical learning theory, deep learning theory etc, contact me. Please retweet.

Huh! Really?!!

Few things in life are more rewarding than sparking ideas in students and helping them see the beauty of complex mathematical concepts.

Introducing NOLA (#ICLR2024) as a parameter efficient finetuning (PEFT) for LLMs: LoRA reduces the number of parameters in finetuning LLMs by limiting the rank of the weight matrix so that dW=AxB. However, the rank cannot go below one, which is still large on large LLMs.

Researchers from @lborouniversity, @MIT, and @Yale introduced the concept of "Collective AI," aiming for AI systems to work together seamlessly. It's AI reimagined as a symbiotic network and could change our approach to global challenges. dailyai.com/2024/03/resear… Read how this innovation could redefine technology's future ⬇️ #AI #CollectiveIntelligence #AIInnovation #FutureTech #MIT #Yale #MachineLearning

@rad_niazadeh @DominikRehse Is it, though? 😅

@DominikRehse Well, that’s almost the same as comparing a Chevy with a BMW; both have four wheels and get you to work, but one is just different 😅

Took less than 15 queries and less than 30 minutes to make ChatGPT generate Tikz code of my hand-drawn figure, with all of the details! (I uploaded my drawing)

Having a storyboard for papers and proposals and writing the document around the figures with comprehensive captions is definitely the way to go.

Kicking off 2024 on a high note! Thrilled to announce that I've received the NSF CAREER award, advancing my research in mathematical ML and optimal transport theory. Immensely grateful to @NSF for this outstanding opportunity! @Vanderbilt_CS Learn more: nsf.gov/awardsearch/sh…

@rdnowak @docmilanfar Yes! Also some earlier yet less known work: ieeexplore.ieee.org/document/91278…

@docmilanfar And you can use the RT to study neural networks arxiv.org/pdf/2301.09554…

Radon Transform (RT) was formulated in 1917 but remained useless in practice until CT scanners were invented in the 60s But RT isn't just for CTs. You can use it for measuring motion! RT g(p,θ): Shoot rays at θ+90 & offset p, measure line integrals of f(x,y) along the ray 1/6

In this #ICCV23 paper, we reparametrize deep models to compact them without changing their architecture. We train the model to be a linear combination in the weight space of many (e.g. 1000) frozen, random models (basis). Paper:web.cs.ucdavis.edu/~hpirsiav/pape… Code:github.com/UCDvision/PRANC

@gabrielpeyre Your visualizations are just fabulous!

The integral curves of a vector field tangent to a manifold stays on the manifold. en.wikipedia.org/wiki/Integral_…

#CVPR2023 decisions are now available on OpenReview! This year, we received a record number of 9155 submissions (a 12% increase over CVPR 2022), and accepted 2360 papers, for a 25.78% acceptance rate.

We are very excited to be hosting Dr. Brandon Amos (Meta AI) as our next guest speaker this upcoming Monday (02/13/2023). He will talk about the integration of optimization and ML methods, with an emphasis on differentiable/amortized optimization. Please join us at 12:10PM(CT).

It was a blast to host @kbatmang in our ML Lunch Seminar at @Vanderbilt_CS. Kayhan talked about 'Bridging Between AI Models & Medical Insights,' and now you can watch his talk and many of our previous seminars here: youtube.com/playlist?list=…

Congratulations to the Best Paper Award Winners!!! #ICASSP2022

Our paper "Generalized Sliced Probability Metrics" won the Best Paper Award at ICASSP 2022! A big shout out to my coauthors: Kimia Nadjahi, @sh2_shahin, and @umutsimsekli, and to my academic home @Vanderbilt_CS.