general abstract nonsense

@Jonathan_Blow Also you might want to check out this recent @ToposInstitute video, I found it nicely explanatory: youtu.be/hsgxoLEzLyo

@damienstanton This has always been how I learn best. I think the problem so far is just that when I read explanations of how CT is supposed to help things I am interested in (like programming), the explanations look wrong to me. But maybe I haven't found the right application area.

Every time I read something to try and learn more about Category Theory, I come away unimpressed. It just always seems like vapid nonsense. If I were to read one and only one book (or paper/video/etc) to convince myself that category theory is good, what should it be?

@Jonathan_Blow Or to put it differently, category theory is only really useful at the interfaces between mathematical systems or paradigms, which is why it is so vapid; outside of those interfaces it’s not going to produce much, but at them it’s a powerful scientific tool.

@Jonathan_Blow Since CT is abstract as a foundational principle (which is weird), to me, the best way to approach it is to marry it with a concrete math you already care about like algebra, type theory, geometry etc. I like Bob Harper’s “computational trilogy” approach. ncatlab.org/nlab/show/comp….

@charles_irl It isn’t not programming, exactly but I have mostly used GPT to explore ideas in type theory (and sometimes I ask for code listings to illustrate our conversations). The most useful thing I have found is synthesizing/comparing/contrasting the content of papers or groups of papers

Anyone using ChatGPT for work tasks _outside of computer programming_ want to share what's worked/not worked for you? I mostly use it for code, where I've developed a sense for where it helps and how to use it, and I'm curious about other areas -- copy-writing, editing, ideation

@dvassallo I’ve volunteered with the TEALS org for a while and the standard HS curriculum is MIT Scratch then onto Python or Java. But personally I would start with Racket! It’s one IDE and place to learn and follows a well structured path with teaching languages: docs.racket-lang.org/drracket/htdp-…

My 9yr old wants to start learning how to make computer games. I was making small games in C at his age, but I’d rather spare him the trauma :) What’s the best tech nowadays for teaching kids to code a Tetris-like game while also learning some programming fundamentals?

@defnotbeka @mattsiegel Harry Partch comes to mind.

what's the most alien, weird, bizarre music you've ever encountered? music that sounds like a nonhuman species created it, but which never the less still feels like music, in that it has clear structure (ie, no noise music or similar stuff that feels random)

@steveklabnik Yes! In particular I love the description of .await as a composition operator. Explained in a clear way like this, I find that thinking of async/await as a certain kind of state monad can make it easier to understand.

@ChShersh It would be a great thing! The more people learn about it, the better. With 5.0, OCaml is the only ~ mainstream language I know of that embraces algebraic effects & handlers as a first class construct.

I wanna make an OCaml course. But now it's not the time. I spent 8 years teaching Haskell, using it in various industries, OSS, blogging, and participating in the community. That's why my Haskell course is THE SHIT. My OCaml course won't be that good. RemindMe! 10 years

@luckymanML @FriedFryingPan @toncijukic @bgolus @bmcnett The more I use LLMs when writing software, the more I agree with @RaeHaskell’s take that it won’t be that AI models will write *all* the code; rather it will change the role of human programmers to one of reviewing/pair programming using rich, expressive specification languages.

@FriedFryingPan @toncijukic @bgolus @bmcnett Have you tried? I’ve used it for like 8 different working projects already. I’m not saying it’s autonomous, but it’s a massive write off. Obviously you need to know core debugging.

So with GPUs now able to read directly from the SSD, and allowing shaders to spawn shaders, what's the ETA on games being 100% run on the GPU and the CPU only existing to ferry user input and networking data around?

Fascinating to see the divergence of “what people speculate LLMs are useful for” and “what LLMs are used for”.

huh? so when gpt4 was thought to be a really really big gpt3 people were like "WOW AMAZING" and now with the rumor of it being 8*220B mixture of experts with small inference trick they are like "Oh, Mixture of Experts? thats what you do when you are low on ideas"?

This is an important aspect of the model-augmented coding workflow, *especially* for students and/or junior engineers.

* People ask LLMs to write code * LLMs recommend imports that don't actually exist * Attackers work out what these imports' names are, and create & upload them with malicious payloads * People using LLM-written code then auto-add malware themselves vulcan.io/blog/ai-halluc…

Can Large Language Models Infer Causation from Correlation? paper page: huggingface.co/papers/2306.05… dataset: huggingface.co/datasets/causa… Causal inference is one of the hallmarks of human intelligence. While the field of CausalNLP has attracted much interest in the recent years, existing causal inference datasets in NLP primarily rely on discovering causality from empirical knowledge (e.g., commonsense knowledge). In this work, we propose the first benchmark dataset to test the pure causal inference skills of large language models (LLMs). Specifically, we formulate a novel task Corr2Cause, which takes a set of correlational statements and determines the causal relationship between the variables. We curate a large-scale dataset of more than 400K samples, on which we evaluate seventeen existing LLMs. Through our experiments, we identify a key shortcoming of LLMs in terms of their causal inference skills, and show that these models achieve almost close to random performance on the task. This shortcoming is somewhat mitigated when we try to re-purpose LLMs for this skill via finetuning, but we find that these models still fail to generalize -- they can only perform causal inference in in-distribution settings when variable names and textual expressions used in the queries are similar to those in the training set, but fail in out-of-distribution settings generated by perturbing these queries. Corr2Cause is a challenging task for LLMs, and would be helpful in guiding future research on improving LLMs' pure reasoning skills and generalizability.

This is also great advice for any discipline, artistic or otherwise. Real, tangible career growth is as much about connecting with people who help motivate, mentor, and inspire as it is about honing the skills.

@patrickmineault Sean Carroll’s Mindscape is excellent.

Favorite podcasts on machine learning, neuroscience or general nerdy matters?

@paulg Remote work when well done is a superior management model. Most of those companies never implemented key aspects of remote work like: - async first communication - documentation first approach - proper feedback loops and performance review

@firasd @goodside Good name 👍

@goodside I'm thinking of a term for this: Oracle Fallacy People really treat ChatGPT like an Oracle so they think it can do things it can't (similar to when people ask "why did you say that?", "what were you trained on?", "is that a real link?" etc)

Friend: I just learned about temperature. Now I use it all the time in ChatGPT! Me: You can't set temperature in ChatGPT. Friend: What do you mean? You just...

@NickADobos @offtheblok There’s a great episode of Signals and Threads that seems to be the only serious treatment of this question I’ve seen on a podcast so far: signalsandthreads.com/future-of-prog…

@NickADobos @offtheblok GPT and similar models seem poor at understanding abstraction boundaries and how software systems should be designed at the level of modules or type theory. They’re going to push the human work into more of a kind of pair programming using rich specification languages (e.g. Agda)

Have seen multiple experienced devs arguing "chatGPT won't replace devs, most coding work isn't writing code" You honestly think an hour long meeting w/10 people at $200k+/yr ea will come up with a better product, design & implementation plan, faster & cheaper than Ai? ok LOL