@_dsevero @cjmaddison Now you get to add a corollary and thank the reviewer for the inspiration
The theorem implies that for all eps>0, f is within eps of g...
English
Richard Lange ๐๐ค๐ง
177 posts
Richard Lange ๐๐ค๐ง
@wrongu
Postdoc with @Kordinglab, interested in the intersection of neuroscience, AI, and philosophy.
Philadelphia, PA ๊ฐ์
์ผ Ocak 2012
247 ํ๋ก์254 ํ๋ก์
Calling this app Twix during these uncertain times of transition
English
@ke_li_2021 Oops โย I missed that when skimming. Sounds like it fits with some philosophical ideas on counterfactual computational results as a test of "representation." Nice!
English
@wrongu Some board states correspond to no valid input sequences. We found that, if we intervene the modelโs board into such states, it can still make legal predictions according to game rule.
English
Preprint: Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task (Othello-GPT: arxiv.org/abs/2210.13382)
How does Large Language Models work? Are they merely memorizing surface statistics or relying on an internal world model?
(1/6)
English
@rmnpogodin @arna_ghosh @gauthier_gidel @g_lajoie_ @tyrell_turing Great work!
At first I was puzzled that the NE potential is convex for w>0 or w<0 but is nonconvex around w=0, while (6) relies on it being convex. But maybe a feature not a bug of the theory? Eq (7) preserves the sign of w. So exp grad respects Dale's law for free - neat!
English
Richard Lange ๐๐ค๐ง ๋ฆฌํธ์ํจ
Counterfactual World Modeling (CWM) is a new project from my group. Our ultimate goal is to build a single unified model that could solve a wide range of human visual tasks in a zero-shot manner -- a kind of pure-vision foundation model. arxiv.org/abs/2306.01828
English
@aliceschwarze I made the switch from mendeley recently. Importing to zotero library seem much smoother, esp with a browser plugin.
English
Remember kids: never plot or inspect your data. That way all of your results can be zero-shot learning!
MMitchell@mmitchell_ai
Remember how Google claimed their system could translate English to Bengali without ever having seen that? And we were like, wait, how do you know that it hasn't? Yeah. Below is evidence that was a lie. Bengali (BN) in bilingual & translation settings was in the training data.
English
@RNogueiraNeuro This work with V1 coarse orientation discrimination is very recently out! journals.physiology.org/doi/full/10.11โฆ
A lot of data available during training that didn't make it into this first paper
English
Neurotwitter please help! I am looking for a dataset with: 1. Sensory discrimination task (drifting gratings, RDM, etc), 2. Population recordings from relevant sensory area (V1, MT, A1, etc), 3. Recordings and behavior throughout learning (the most important part). Anybody knows?
English
@patrickmineault This work is great and we have been extending it!
arxiv.org/abs/2206.10999
Elevator pitch: once you start talking about *distance* between neural reps, you can also start talking about *directions*. We can use tools from geometry to talk about how reps are transformed.
English
What does it mean for a neural network to "be like the brain"? I revisited this paper from Alex Williams and co today, which I think asks the right questions and offers some tantalizing answers. 1/
Alex Williams@ItsNeuronal
A very interesting trend in neuroscience is the emergence of datasets with large-scale recordings repeated across many animals (see e.g. @IntlBrainLab) How to make sense of variability across animals in these data? Our NeurIPS '21 paper has a few ideas arxiv.org/abs/2110.14739
English
@cian_neuro Based on past performance, 3+ years. But deep down I will forever believe that the next one will take two weeks
English
How long does a computational neuroscience project that results in a โgoodโ paper typically take?
English
Richard Lange ๐๐ค๐ง ๋ฆฌํธ์ํจ
Neuromatch soft-launched its mastodon instance last week. We already have a great local feed, you should totally join our merry neuroscientist/hacker spaceship ๐ neuromatch.social
English
@rachel_kurchin Check out the neuromatch course materials! deeplearning.neuromatch.io/tutorials/intrโฆ
Videos + exercises embedded in colab. This works best with a flipped classroom arrangement so students can go through the notebooks, including some exercises ahead of class time.
English
๐จ summoning the hivemind! ๐
I want to have some interactive computational demos in my course next semester, likely in the form of notebooks. I need it to be runnable in a browser without installing anything or waiting excessively for something to spin up, โฆ
English
Richard Lange ๐๐ค๐ง ๋ฆฌํธ์ํจ
declaring a new term:
A Bach Faucet is a situation where a generative system makes an endless supply of some content at or above the quality of some culturally-valued original, but the endless supply of it makes it no longer rare, and thus less valuable
English
Richard Lange ๐๐ค๐ง ๋ฆฌํธ์ํจ
Who of the famous people in the field would be ready to publicly talk about their non-rigorous research in the past? I think that few people have ever published 10 papers without major rigor issues in at least one of them. Let's normalize talking about rigor.
English
@micahgoldblum Testing my understanding: this is hard constraints (EmpiricalRM) vs soft complexity penalties (RegularizedRM).. Less complex (lower norm) functions take fewer samples to learn even if your hypothesis space is huge, and VC/rademacher depend only on size of the space, right?
English
The following statement, while a commonly held view, is actually false! โLearning theory says that the more functions your model can represent, the more samples it needs to learn anythingโ. 1/8
Yann LeCun@ylecun
OK, debates about the necessity or "priors" (or lack thereof) in learning systems are pointless. Here are some basic facts that all ML theorists and most ML practitioners understand, but a number of folks-with-an-agenda don't seem to grasp. Thread. 1/
English
@DimitrisPapail A very similar line of questioning (how to think about composing "simple steps" towards implementing some complex functions) motivated this: arxiv.org/abs/2206.10999
I still have more questions than answers and would be interested to hear what you find out!
English
Say I want to infer a function f() from its evals. That's hard! But what if f(x) is an "algorithm" comprising T iterations, i.e.,
f(x) = f_T(...(f_2(f_1(x)))
and f_i are "simple" (e.g., low-deg poly?). Do we know how do to recover these f_i's?
English
@gershbrain These papers draw some really nice links between prior predictive, posterior predictive, cross validation, and model comparison :
arxiv.org/abs/1905.08737
arxiv.org/abs/2202.11678
English
I've come across an interesting phenomenon, which I'm sure has been discussed before but I don't know the right references.
English
@KordingLab @SuryaGanguli Credit where credit is due! AlHazen was ahead of his time. Not sure if he counts as part of the "direct/transparent" link that @SuryaGanguli asked about. Did Helmholtz know about AlHazen? Did the authors of the Helmholtz machine? Or were ideas rediscovered?
English
@wrongu @SuryaGanguli Although you could ascribe these to AlHazen
English
I want to learn about useful applications of philosophy to science. Other than the scientific method itself, what are concrete examples of philosophy done by โcard carryingโ academic philosophers that lead, thru a direct/transparent chain of causation, to new successful science?
English