Maxime Daigle

New paper 🚨 #ICLR26 Most world models predict the future from a past trajectory. But neuroscience suggests that such inference can instead be made from temporally independent experiences. We built the Episodic Spatial World Model (ESWM), a model that does exactly this:

New paper 🚨 "Stable Deep Reinforcement Learning via Isotropic Gaussian Representations" Deep RL suffers from unstable training, representation collapse, and neuron dormancy. We show that a simple geometric insight, isotropic Gaussian representations, can fix this. Here's how 👇

🧠 Can a neural network build a spatial map from scattered episodic experiences like humans do? We introduce the Episodic Spatial World Model (ESWM)—a model that constructs flexible internal world models from sparse, disjoint memories. 🧵👇 [1/12]

Here comes σ-GPT! TL;DR: We added an output positional encoding, which allows to generate tokens in any order chosen on the fly. Tweet 5/6 is sweet.

Have you ever done a dense grid search over neural network hyperparameters? Like a *really dense* grid search? It looks like this (!!). Blueish colors correspond to hyperparameters for which training converges, redish colors to hyperparameters for which training diverges.

@francoisfleuret @zga_aaa I see. Sounds reasonable, thanks

@MaximeMDaigle @zga_aaa There are up to 4 items, for each there is one choice among 6 colors, 6 letters, 6 rows and 6 columns, so that's ~6^(4x4) if we ignore exclusions, that's already 3e12. Then there are the transformations, and then the properties...

One more MyGPT experiment! The GPTs trained on language are very bad at spatial representation. So I tried a very simple experiment to check if it is just a corpus issue (which tbh seems obvious) 1/7

@francoisfleuret @zga_aaa Not sure if the order of transformations matter or if the order does change, so to be safe number permutation = 60 + 20 + 5 + 1 = 86 unique transformations. Number of unique samples = 86*6*6*6*6 = 111,456. 250k samples is at least 2 times the upper bound of unique samples.

@francoisfleuret @zga_aaa I don’t understand how the train and test set is separated? If we quickly compute an upper bound for the number of unique samples. 6*6 locations, 6*6 unique objects, 5 basic transformations (combine between 0 and 3).

Inverse Scaling Prize Update: We got 43 submissions in Round 1 and will award prizes to 4 tasks! These tasks were insightful, diverse, & show approximate inverse scaling on models from @AnthropicAI @OpenAI @MetaAI @DeepMind. Full details at irmckenzie.co.uk/round1, 🧵 on winners:

Brain-computer interface (BCI) 2021 year in review 🧵 2021 was a big year! We saw an intracortical BCI decode handwriting, the 1st country to pass a bill on neuro-rights, & much more Below we cover the most exciting BCI updates across academia, industry, & public policy

How do we learn from numerous experiences throughout the day without major interference? How does the brain optimize rapid encoding based on prior experience? Check our latest article published today @NeuroCellPress: sciencedirect.com/science/articl… @YalePsych @YaleNeuro @YaleMed 1/6

XKCD makes an appearance in a meeting xkcd.com/2347/

“This offers the opportunity to actually understand, in a very concrete way, how the hippocampus contributes to memory storage in the cortex,” says Mark Bear of his and @peter_finnie's new study in @CurrentBiology picower.mit.edu/news/brains-me… #neuroscience @mitbrainandcog @ScienceMIT

NPR’s Jon Hamilton spotlights @DrLiHueiTsai's work developing a noninvasive technique that uses lights and sounds aimed at boosting gamma waves and potentially slowing progression of Alzheimer’s disease. npr.org/sections/healt…

Can we improve chatbots after deployment from natural user feedback? A big YES :) User feedback has rich cues about errors but it cannot be used directly for training. So we use GANs to generate training data from feedback. arxiv.org/abs/2010.07261 #Findings-of-#EMNLP2020 #NLProc.

We found that just as a large transformer model trained on language can generate coherent text, the same exact model trained on pixel sequences can generate coherent image completions and samples. openai.com/blog/image-gpt/

To generate a seemingly infinite number of portraits in a variety of painting styles, @NVIDIA researchers developed StyleGAN2. The new model, trained on NVIDIA V100 GPUs with @TensorFlow, will be presented at #CVPR2020. Learn more here: nvda.ws/2UJ3udu

We're releasing Procgen Benchmark, 16 procedurally-generated environments for measuring how quickly a reinforcement learning agent learns generalizable skills. This has become the standard research platform used by the OpenAI RL team: openai.com/blog/procgen-b…

Interested in learning about multi-armed bandits? My @MSFTResearch colleague Alex Slivkins just published an introductory book and the whole thing is available for free on arXiv. arxiv.org/abs/1904.07272

This feels like a real breakthrough: arxiv.org/abs/1911.08265 Take the same basic algorithm as AlphaZero, but now *learning* its own simulator. Beautiful, elegant approach to model-based RL. ... AND ALSO STATE OF THE ART RESULTS! Well done to the team at @DeepMindAI #MuZero