Jerry Swan retweetledi
I'm super excited to have a new preprint - Manifold Diffusion Geometry
Diffusion geometry *robustly* measures curvature, dimension, and tangent spaces of data!
arxiv.org/abs/2411.04100
GIF
GIF
English
Jerry Swan
209 posts
Jerry Swan
@RadioFreeMonad
Cyberneticist.
radiofreemonad.bsky.social Katılım Temmuz 2015
489 Takip Edilen222 Takipçiler
@jwesamuel @reasonisfun @DavidDeutschOxf Using QM *and* GR as an example: supersymmetry, loop quantum gravity, and String theory.
Disclaimer: I am not a physicist.
English
@RadioFreeMonad @reasonisfun @DavidDeutschOxf Can you give an example of the kind of counter factual hypotheses a deep theory evokes? Eg using qm or general relativity or some other universal theory as an example?
English
@analytichegel Funnily enough, Section 9.4 of link.springer.com/book/10.1007/9… was Hegel-inspired ...
English
@bleepbeepbzzz See also section 9.3 of link.springer.com/book/10.1007/9… which uses some category theoretic gadgets for abstraction and analogy, with LSA as an example of the latter
English
@RadioFreeMonad Argh, how did I miss this? You have letter-string problems, Structural Information Theory, and GP all in one paper!
English
Jerry Swan retweetledi
I'm hiring a postdoc / research assistant to work in the area of Bongard Problems, Raven's, Hofstadter's toy worlds, and ARC @arcprize.
Using genetic programming driven by visual feedback to find symbolic solutions reflecting structure in objects.
In Uni of Galway, Ireland.
English
@vr4300 >by extension it should be possible to directly build
>a model that understands the innate structure
True, but then these folk never said they wanted to build a *good* synthetic data generation engine, they just know their learning methods need more data from somewhere ... ;-)
English
Synthetic data is an incredibly silly concept. If you can build a synthetic data generation engine, by extension it should be possible to directly build a model that understands the innate structure of the task completely skipping the need for more training data to learn it.
Jacques@JacquesThibs
I keep saying how underrated it is among alignment researchers
English
@kenneth0stanley @pretendsmarts David A. Wilkenfield, 'Understanding as Representation Manipulability': jstor.org/stable/41931582
English
I’m not really advocating any particular approach, just highlighting that focusing on reasoning can obfuscate a deeper issue that may be more instructive to investigate further. There are plenty of potential solutions- lots of room for creativity - but first it helps to diagnose the problem.
English
When LLMs fail at simple reasoning, many posit an inability to genuinely reason as the culprit. But there’s a deeper ingredient that underpins reasoning and many other faculties as well: representation. Before worrying about reasoning, worry about representation.
English
Lot of concerns about definition of understand here. Personally, I can't think of a meaningful definition of "understand" that'd suggest a chess program doesn't understand chess, other than "understands via a human brain," but I appear to be in the minority.
Zach Weinersmith@ZachWeiner
Can a champion chess program be said to understand chess?
English
@analytichegel @gtupinamba @mattecapu @kenwheeler @tangled_zans Also @YistvanPof has a bunch of CT stuff for musical analysis here:
alpof.wordpress.com
English
English
lmao I have no memory of that pic getting taken but here we are
patagucci perf papi@kenwheeler
just remembered the fucking funniest thing that ever happened
English
@analytichegel Russell's 'Problems of Philosophy' and 'History of Western Philosophy' are exquisitely written throughout and obviously contain some analytic sections.
English
Jerry Swan retweetledi
Really pleased to finally share my preprint: Diffusion Geometry
(1/4) Diffusion geometry defines Riemannian geometry for data and probability spaces!
arxiv.org/abs/2405.10858
English
@GaryMarcus >I honestly have no idea how I will answer.
By analogy with NP-completeness, just tell them that LLMs solve the complexity class of "Bullshit-complete" problems.
i.e. great for fallibly performing only the 'bullshit jobs' with output that can be easily verified by a human ...
English
Someday my children will ask me why anyone ever took seriously the thought that systems like this had anything to do with AGI.
I honestly have no idea how I will answer. Mass delusion?
(Example emailed to me by a friend.)
English
I stopped being interested in "complete this series of numbers" problems after learning that it's rather simple to prove that any of those has an infinite number of solutions. Suppose you have N points in a graph, add an arbitrary point, find a polynom that connects them, done.
English
@reasonisfun Marvin Minsky was of a similar opinion: he said something to the effect that "when we think we've 'had an idea', it's because the unconscious mind has yielded the result of its deliberations to the conscious".
See section 7.7 of his book, "The Emotion Machine".
English
Anyone know any studies or theories that have bearing on this conjecture?
Lulie@reasonisfun
I suspect further that breakthroughs (or thought of any kind) *always* start unconsciously — the subconscious is where the creative program lives — and when it comes to consciousness, you’re doing a different operation with it (criticising it, communicating it, comparing it against other ideas).
English
@mattecapu There's a long history in that area that greatly precedes transformers - for example, Lawrence Fogel's work.
English
has anyone trained a generative model (eg a transformer) to generate words from the language of an automaton?
English
@filippie509 >the set of corner cases is "fat tailed"
>and therefore sampling is doomed.
Agreed e.g. as described on p26 of:
link.springer.com/book/10.1007/9…
English
Next version of FSD rolls out, and within days a flurry of silly failures pop out on twitter/YouTube. This is 100% consistent with a hypothesis I put forward in this thread almost two years ago, that the set of corner cases is "fat tailed" and therefore sampling is doomed.
Filip Piekniewski🌻 🐘:@[email protected]@filippie509
For years I've been claiming that our #AI being based in statistics is not equipped for dealing real world data. That data is most likely fat-tailed, and hence not possible to characterize by sampling. Let me explain this in thread below: 👇
English
@PetarV_93 Of possible interest: link.springer.com/article/10.100…
English
This is very much a work in progress, and we welcome any and all comments from the machine learning / applied category theory community! Please feel free to reach out 😊
English
Unveiling the next chapter in our quest towards the elusive theory of *all* neural networks: Categorical Deep Learning! 🐈🤖
categoricaldeeplearning.com
See thread 🪡 for a bucket list 🪣 of concepts our framework can handle!
w/ @bgavran3 P. Lessard @andrewdudzik @tlvg @_joaogui1
English