Shyam Tailor

Please sign up! Benefits include free registration :-) If you have any questions please feel free to shoot me or @egctong an email using sv-chairs-2021@ubicomp.org

We're not *quite* as good at pruning at init as the lottery ticket hypothesis (LTH) says is possible, but we've made a big jump in this work! A good question is: how can we get it to work better for ImageNet? We make progress but there's still work to do here 🙂

9/9 Hope this has been interesting for you 🙂. Code: github.com/shyam196/egc Paper: arxiv.org/abs/2104.01481 Thanks especially to @chaitjo!

8/n @yitayML et al. found that pretrained convolutions performed well relative to pretrained (anisotropic) transformers. And @liuzhuang1234 et al. found that convnets are still competitive with ViTs. Is it really unbelievable that similar results apply for GNNs?

Question: How confident are you that we need attention mechanisms or full-fat message passing in our graph neural networks? If your answer was "of course" then my ICLR paper *will* make you think again! @FelixOpo @pl219_Cambridge @niclane7 arxiv.org/abs/2104.01481

@chaitjo I'll send you an email today with some stuff :-)

@satailor96 What are some key papers on hardware accelerators for GNNs? I'd like to add this aspect to the list, too.

This a great resource for anyone who is interested in getting into efficiency aspects of GNNs. Thanks for putting this together Chaitanya! 😀

@PeterWBattaglia Simple and elegant - thanks for sharing! 😀

In new work led by Jonathan Godwin, Michael Schaarschmidt, et al, we show how very deep GNNs can be trained effectively using a simple noise/denoising scheme, and achieve top performance on several challenging molecular benchmarks: arxiv.org/abs/2106.07971

@chaitjo @PetarV_93 @urialon1 @yahave I think this changed in a recent commit for PyG 🙂. Worked when I tried it for the EGC paper, with a fair bit of headroom

@PetarV_93 @urialon1 @yahave Sidenote on scaling: Another GAT implementation detail I found curious/interesting is that DGL's GAT is significantly more optimized than any other library. E.g. You cannot train full-batch PyG GATs on ogbn-ARXIV yet; current top performing models use DGL GAT (+ many tricks).

Important read of the day: GATv2 (Brody, @urialon1, @yahave): arxiv.org/abs/2105.14491 The exact attention mechanism I used in the GAT paper was intentionally 'weakened' to make it work on the easy-to-overfit datasets of the time. It was never meant to be a 'silver bullet'... 1/2

Our DGN paper was accepted as a long oral presentation (top 3%) at the ICML2021 conference of machine learning!! In this paper, we generalize the notion of directions beyond Euclidean spaces arxiv.org/pdf/2010.02863… @Saro2000 @vincentmillions @GabriCorso @pl219_Cambridge Will H.

Join me at the poster session at 3pm UK time! 🙂

@weihua916 Thanks for sharing, this was a fun read! Out of curiosity -- do you have any insight into why Swish helps so much? In fig5 it looks like helps to nearly close the gap when using no edge features (!)

Excited to give a contributed talk on "ForceNet: A GNN for Large-Scale Quantum Chemistry Calculations” at Deep Learning for Simulation Workshop at ICLR 2021 Paper: simdl.github.io/files/62.pdf Talk: #collapse-sl-3685" target="_blank" rel="nofollow noopener">iclr.cc/virtual/2021/w… Done as part of Open Catalyst Project opencatalystproject.org

Join us now at the ICLR poster session! 🧑‍💻 iclr.cc/virtual/2021/p…