Russell Tsuchida

Squared families can be viewed as simplified versions of previous models, such as squared probabilistic circuits (@loreloc_ ), or squared neural families. We fix the "hidden feature" for extra tractable theory, while still allowing for rich representations.

Squared families are in some precise sense "rare" families which always have tractable (often closed-form): 1. Normalising constant 2. Fisher info 3. Statistical divergence all linked through a tractable convex Bregman generator and integral. Applied to MLE for params/density.

Come chat with us about "Squared Neural Families: A New Class of Tractable Density Models" tonight on Discord. 6pm UTC, ykilcher.com/discord

Squared Neural Families: A New Class of Tractable Density Models [spotlight], with @RussellTsuchida, @ChengSoonOng, a simple neural network density model with closed form normalising constants, generalising exponential family. arxiv.org/abs/2305.13552 neurips.cc/virtual/2023/p… 4/4

We find latent variables under an exponential family model using SGD. We call inner products of these latents Deep Equilibrium Kernels (DEKers). DEKers are built from and generalise NNGP components. DEKers may admit fixed points and are kernel versions of deep equilibrium models.

Pleased to share our freshly accepted paper at TMLR! If you are interested in neural network kernels (NNGP, NTK), implicit neural networks, SGD, or exponential families, you might enjoy this paper. youtu.be/Ab2KEE9OqEk

@le_anh_phuong @aistats_conf @ChengSoonOng Hi Phuong. I didn't realise you had started a PhD. Congratulations to you too!!

@RussellTsuchida @aistats_conf @ChengSoonOng Congratulations Russell!! This is exciting!

Nice to be recognised as a Top Reviewer (top 10%) by @aistats_conf! Also looking forward to attending in April, where I will be presenting arxiv.org/abs/2211.05943, which views exponential family PCA in light of deep equilibrium models. Joint work with @ChengSoonOng.

The Legendre transform can be approximated in O(n*log(n)) using a Gaussian convolution. en.wikipedia.org/wiki/Legendre_… en.wikipedia.org/wiki/Lower_con…

Some tricks of the trade in #MachineLearning Kindly add in comments missing tricks please. Thanks. @thegautamkamath twitter.com/thegautamkamat…

Camera ready paper available here: openreview.net/pdf?id=q4HaTeM…

Suk Yee, Lars, Cheng Soon and myself are part of @CSIROMLAI.

Happy to announce that work emerging from a successful collaboration between myself, @sukyeeyong, Ali Armin, Lars Peterrson and @ChengSoonOng has been accepted into @iclr_conf!

Excited to present our work Gaussian Process Bandits with Aggregated Feedback (accepted by #AAAI2022) at #NeurIPS2021 #WiML2021 poster session #1 (neurips.cc/virtual/2021/a…) at 3:45-4:45pm AEST . Pre-print mengyanz.github.io/files/aaai2022… Joint work with @RussellTsuchida @ChengSoonOng

@aaronsnoswell I see you found the answer yourself :) Cool stuff! The question was tagged as recreational so I'm curious; do you have an application in mind or is just for fun?

Help me solve a Polyomino problem! "Counting elements of Polyomino coverings" at math.stackexchange.com/q/3578100/9803…

Does anyone want to guess what this is?? 🤔🤔😷

Back in February, @Tea_Pearce, Chris van der Heide, Fred Roosta, @marcus_marcusg and I did some work on the kernels of infinitely wide deep neural networks with GELU and ELU activations. We also studied the fixed points of these kernels. Check it out here: arxiv.org/abs/2002.08517

@marcus_marcusg Is this about chng.it/7RxHPhXTp8 ? Does "attendance" have to be synced? Do we even need time zones?

If research conferences are held fully online, which time zone do they use?