@glupyan @AndrewLampinen The hype-counterhype incentives in the study of LLMs are making everything around them so hard to understand with a cool head. Very frustrating.
English
Pablo Contreras Kallens
89 posts
Pablo Contreras Kallens
@pcontrerask
Ph. D. candidate, Cornell Psychology in the Cognitive Science of Language lab. Serious account.
Ithaca, NY Katılım Eylül 2020
54 Takip Edilen127 Takipçiler
Can people truly reason? On this definition, no. They can’t even tell you if a number is even or odd reliably - sapir.psych.wisc.edu/papers/lupyan_… pattern matching all around 🤷♂️ (@AndrewLampinen )
Mehrdad Farajtabar@MFarajtabar
1/ Can Large Language Models (LLMs) truly reason? Or are they just sophisticated pattern matchers? In our latest preprint, we explore this key question through a large-scale study of both open-source like Llama, Phi, Gemma, and Mistral and leading closed models, including the recent OpenAI GPT-4o and o1-series. arxiv.org/pdf/2410.05229 Work done with @i_mirzadeh, @KeivanAlizadeh2, Hooman Shahrokhi, Samy Bengio, @OncelTuzel. #LLM #Reasoning #Mathematics #AGI #Research #Apple
English
I've been having a nice time talking to @denizcemonduygu about maps and graphs of philosophical discourse, and thought I'd repost my bibliometric map of the phenomenology literature. Dots are authors, links are citations, and colors are clusters.
English
@JeffYoshimi @denizcemonduygu Such a fun project!!!
English
@denizcemonduygu I should have said _our_ map, since this was thoroughly collaborative with @pcontrerask
English
@Forrest_L_Davis @MH_Christiansen @CSL_Lab There's no paper yet! But yes, we found that in this particular agreement dataset the accuracy of the RLHF GPT-3 diminishes in comparison with non-RLHF models in the two phrase manipulations that humans struggle with the most.
English
@MH_Christiansen @CSL_Lab @pcontrerask Is there a link to the talk or to a paper/abstract? There’s work on agreement errors in English models (partially) overlapping with humans without RLHF or “feedback”. Is the argument they align more closely after fine tuning ?
English
Pablo Contreras Kallens retweetledi
@CSL_Lab alumn @pcontrerask talks about how feedback is crucial for getting large language models to produce more human-like language output, such as making similar agreement errors and being sensitive to subtle semantic distinctions
English
Pablo Contreras Kallens retweetledi
Information density as a predictor of communication dynamics
Spotlight by Gary Lupyan (@glupyan), Pablo Contreras Kallens (@pcontrerask), & Rick Dale on recent @NatureHumanBehav work by Pete Aceves (@peteaceves) & James Evans (@profjamesevans)
authors.elsevier.com/a/1ixK34sIRvTA…
English
@ElmlingerSteven Small correction! It'll be 10 pm EST!!
English
Come see Pablo Contreras Kallens, @pcontrerask a newly-minted PhD 🥳🥳, at CogSci for questions this week @ 7 PM EST / 9 AM AUS on Fri the 28th and Sat the 29th, poster ID: VP-T-557-1812!
CogSci paper link: escholarship.org/uc/item/3wj676…
English
Pablo Contreras Kallens retweetledi
How do infants learn to produce the consonant sounds of their ambient language? To find out, check out our CogSci proceedings paper “Statistical learning or phonological universals? Ambient language statistics guide consonant acquisition in four languages” A 🧵: /1
English
it's me!
Morten H. Christiansen@MH_Christiansen
Huge congratulations 🥳to 👉Dr.👈 @pcontrerask who just passed his PhD defense with flying colors!👏 He defended his dissertation in @CornellPsychDpt THE COMPUTATIONAL BRIDGE: INTERFACING THEORY AND DATA IN COGNITIVE SCIENCE Follow @pcontrerask to see the papers
English
@glupyan Their response to why it's not seems fixated on only one of the interpretations of a very complex intellectual exercise, which focuses on just a portion of the assumptions. Really strange, to me.
English
So everyone knows Bender and Koller’s “octopus test” is Searle’s Chinese room thought experiment, right? And there are decades of critiques showing all the ways it’s wrong..? (Asking for a friend)
English
Pablo Contreras Kallens retweetledi
I'm delighted to announce the publication of our free, open access book, "Horizons of Phenomenology", a collection of essays on the state of the field. A brief thread about the book, and the long and ultimately victorious struggle to publish it open access. 1/
English
Pablo Contreras Kallens retweetledi
Just realized I didn't include the link. Here it is! link.springer.com/book/10.1007/9…
Jeff Yoshimi@JeffYoshimi
I'm delighted to announce the publication of our free, open access book, "Horizons of Phenomenology", a collection of essays on the state of the field. A brief thread about the book, and the long and ultimately victorious struggle to publish it open access. 1/
English
Pablo Contreras Kallens retweetledi
In sum, the ideas that children (a) have innate syntactic categories and (b) NEED them because they can't construct them via distributional analyses alone are NOT straw-men but real and influential proposals in the child language literature 7/n
English
@jbenfalandays I was once told that bacteria don't have cognition because they don't have brains, by people who were staunch Fodorians. Weird feeling
English
@roger_p_levy @glupyan @weGotlieb @xsway_ I don't know if that exchange ever actually happened in published form, so it's just a hunch, and the now "weak" version can be considered now as a contender. But labeling it as a "modern" version, as if the other option was just a strawman, is a bit too revisionist for my taste.
English
@roger_p_levy @glupyan @weGotlieb @xsway_ I fear there's a little bit of squirming in this "weak" POS. I don't think generativists would have told you that the magic "amount" actually exists. I have trouble believing that if you asked "what if they got 2x the data? 3x? 100x? the answer would have been "yes" at any point.
English
I urge interested readers to consult the @weGotlieb et al. in press: we find that the "GRNN" LSTM of @xsway_ et al. 2018 trained on a childhood's worth of English shows substantial success on filler–gap dependencies and the island constraints on them. 1/3 direct.mit.edu/ling/article/d…
Masoud@linguistMasoud
It seems like in the recent discussion of the poverty of the stimulus & large language models two versions of PoS are being conflated. Version 1 says human children do not have enough data to learn abstract grammatical representations from *their* input. LLMs have nothing to say
English
@glupyan They won't say though, in typical Open fashion
English
Naive (?) question: how does OpenAI (and other big players) get access to all the copyrighted books etc?
English
Pablo Contreras Kallens retweetledi
I've seen lots of threads about large language models (LLMs) and their implications for language acquisition BUT not many threads by language-acquisition specialists. So here's my two cents on how LLMs undermine SOME SPECIFIC PROPOSALS for acquisition of syntactic categories 1/n
English
@EvelinaLeivada @glupyan @Timothy0Leary But this framing assumes a view of language that's based on the work being questioned. "Learning rules" is an explanation, not a description, of productive language use. If a model that doesn't do the former can achieve the latter, why assume that that's what people are doing?
English
@glupyan @Timothy0Leary Honest Q: Never mind the language disagreement, would you say it is *learning*? Meaning figuring out of how a rule works, the domain in which it applies, its exceptions, its productivity etc. Do they learn in the sense of critically engaging with rather than absorbing any input?
English
I'm bewildered. The argument here seems to be that if a statistical model of a phenomenon provides accurate predictions, then this is evidence that the phenomenon itself is nothing more than that same statistical model + the means used to fit it.
English