Parthe Pandit
217 posts
Parthe Pandit
@PartheP
Thakur Family Chair Assistant Professor @ C-MInDS, IIT Bombay
Powai, Mumbai शामिल हुए Şubat 2014
524 फ़ॉलोइंग334 फ़ॉलोवर्स
@mybmc No action taken on this blocked & broken drainage chamber on LJ Rd despite formal complaint, follow-ups. Please take immediate action. Serious health menace for residents. Local sewage not draining into municipal gutters ⇒ clogged sewage ⇒ rats, mosquitos, cockroaches...
English
@dbeagleholeCS @etienne_barnard @etienne_barnard
very good question!
here is a (hopefully satisfactory) answer:
twitter.com/dbeagleholeCS/…
Daniel Beaglehole@dbeagleholeCS
We identify that Conv Nets implement a variant of the same general mechanism of feature learning as in fully-connected networks. The covariances of the filters in CNNs again recover the average gradient outer-product (AGOP) of the model, additionally averaged over input patches.
English
What is the nature of feature learning in deep networks? We propose that neural networks recover a statistic known as the average gradient outer product (AGOP).
Github: github.com/aradha/recursi…
arXiv: arxiv.org/abs/2212.13881
English
@dbeagleholeCS @_onionesque easier to come up with new iterative algorithms rather than throwing away previous estimates
English
@_onionesque Yeah I think @PartheP may have mentioned this? what is the advantage of a variational formulation?
English
At the risk of sounding repetitive: Everyone should check this paper and line of work out! First, because it's cool, and second, they somehow (still not sure how) found some old paper of ours, which I really liked*, that literally no one cared about! 😜
columbia.edu/~skk2175/Paper…
Daniel Beaglehole@dbeagleholeCS
We identify that Conv Nets implement a variant of the same general mechanism of feature learning as in fully-connected networks. The covariances of the filters in CNNs again recover the average gradient outer-product (AGOP) of the model, additionally averaged over input patches.
English
Super proud that my student Nami Matsumoto just won an NSF Graduate Research Fellowship. This is sooo well-deserved!
English
@andrewgwils 2. arxiv.org/abs/2302.02605
How to train kernel models with large number of centers (or inducing points)?
A new algorithm, EigenPro3, to train such models requiring only O(p) memory for p centers. Prior work such as FALKON needed O(p^2) memory, ie, infeasible for large models
English
...and 3) conjugate gradients! :). I look forward to reading this. I suppose "pure kernel methods" means no kernel learning? I've always wanted to combine these infinite-width methods with kernel learning... perhaps a promising next step.
Jasper@latentjasper
Really excited to share this new paper: Kernel Regression with Infinite-Width Neural Networks on Millions of Examples. We found that the recipe for success with kernels is 1) highly expressive kernels (infinitely wide deep nets) and 2) lots of data. arxiv.org/abs/2303.05420
English
@andrewgwils You might find these papers interesting too:
1. arxiv.org/abs/2212.13881
recursively adapting kernel functions inspired by how neural networks learn features
These models, Recursive Feature Machines, are SotA on tabular datasets and bridge the gap to fully connected networks
English
English
Extremely delighted to join @Cornell_ORIE @cornell_tech @Cornell. Heartfelt thanks to my amazing advisors, @LesterMackey, Susan Murphy, Devavrat Shah, Martin Wainright, & Bin Yu, the wonderful collaborators, friends, & family for making this possible. The chapter starts Jan 1st!
Susan Murphy lab@SusanMurphylab1
Postdoc @raazdwivedi has accepted a tenure-track assistant professorship at @Cornell, Operations Research and Information Engineering @Cornell_orie, @Cornell Tech in NYC! Congratulations, Raaz!!!
English
@nmallinar you can't just get a single dream of Costco. you have to get a 32 pack
English
@PreetumNakkiran i am hoping someone creates a generative model that converts hand drawings to professional figures
English
don’t know tikz, so I have to waste time the old fashioned way (drawing unnecessary figures by hand)
English
Indonesia
Thrilled to share that our HPCA'23 paper "Scalable and Secure Row-Swap" has been selected for the Best Paper Award!
This was a collaborative work led by @JeonghyunW and @Prashxnt_Nair at @UBC !
JH will present our paper at @HpcaArchConf on 27 Feb in Montreal!
English
We now have a simple, powerful, and stable alternative to Deep Nets
A challenge in classical kernel models is choosing the 'right' kernel
By examining NN training, we identified the modification necessary to empower kernels - FEATURE LEARNING
#NeuralNetworkFreeSince2023
Daniel Beaglehole@dbeagleholeCS
What is the nature of feature learning in deep networks? We propose that neural networks recover a statistic known as the average gradient outer product (AGOP). Github: github.com/aradha/recursi… arXiv: arxiv.org/abs/2212.13881
English
@Yizhezhu_ How many samples do I need to generate to achieve the rate? If there are n true samples, can you generate O(sqrt(n)) samples and still achieve small W1-distance?
English
@ysbhalgat Paragraph structure is very important for technical communication.
The same text should perhaps be divided into 3-4 separate paragraphs, each with a purpose.
English
@ysbhalgat Not a good summary
- Needs bullets. Currently, I need to read an entire sentence with obscure 10-word-named schemes to get to the start of the next sentence
- The Capitalization Of Words In The Sentences Makes It Harder To Read Easily
- Topics can be segregated based on themes
English
IMO one of the most useful applications of #ChatGPT is summarizing or explaining lengthy documents which usually require domain expertise to really make any sense of.
This could be scientific papers, legal paperwork, financial documents, etc. (1/n)
#ai
AI Breakfast@AiBreakfast
Use ChatGPT on your own files This is going to be big: humata.ai lets you upload a .pdf up to 60 pages long and allows you to ask questions about it in plain English ↓
English
@_onionesque whats the difference between Bayes theorem and Bayes rule?
English
A poster by Larry Wasserman to put on your refrigerator.
English
I just used humata.ai to explain the Union Budget recently released by the Indian Govt.
I have a minimal understanding of Economics/Politics, but I think it did a great job summarizing the key points in a 58-page doc. (2/n)
#UnionBudget2023
English
Parthe Pandit रीट्वीट किया
Rahul Parhi and I wrote a tutorial style article to explain modern theory of deep learning and neural function spaces via elementary signal/image processing concepts (Fourier transform, Radon transform, L1 regularization, sparsity) arxiv.org/abs/2301.09554
English
@DimitrisPapail the difference between
"I could have written that" v/s "I can't paint that" ?
English
Nobody calls denoising bullshit generation, but suddenly if it happens using a transformer then it is?
English