Umesh Patil

How do memory retrieval and prediction work together during sentence comprehension? We use computational modeling + eye-tracking to unpack their interaction in German possessive pronouns. New paper with @_joaoverissimo, @_mesh and @sol_lago doi.org/10.1016/j.jml.…

Contrary to standard prompting advice that you should give LLMs the context they need to succeed, I find it’s sometimes faster to be lazy and dash off a quick, imprecise prompt and see what happens. The key to whether this is a good idea is whether you can quickly assess the output quality, so you can decide whether to provide more context. In this post, I’d like to share when and how I use “lazy prompting.” When debugging code, many developers copy-paste error messages — sometimes pages of them — into an LLM without further instructions. Most LLMs are smart enough to figure out that you want them to help understand and propose fixes, so you don’t need to explicitly tell them. With brief instructions like “Edit this: …” or “sample dotenv code” (to remind you how to write code to use Python's dotenv package), an LLM will often generate a good response. Further, if the response is flawed, hopefully you can spot any problems and refine the prompt, for example to steer how the LLM edits your text. At the other end of the spectrum, sometimes I spend 30 minutes carefully writing a 2-page prompt to get an AI system to help me solve a problem (for example to write many pages of code) that otherwise would have taken me much longer. I don’t try a lazy prompt if (i) I feel confident there’s no chance the LLM will provide a good solution without additional context. For example, given a partial program spec, does even a skilled human developer have a chance of understanding what you want? If I absolutely want to use a particular piece of pdf-to-text conversion software (like my team LandingAI’s Agentic Doc Extraction!), I should say so in the prompt, since otherwise it’s very hard for the LLM to guess my preference. I also wouldn’t use a lazy prompt if (ii) a buggy implementation would take a long time to detect. For example, if the only way for me to figure out if the output is incorrect is to laboriously run the code to check its functionality, it would be better to spend the time up-front to give context that would increase the odds of the LLM generating what I want. By the way, lazy prompting is an advanced technique. On average, I see more people giving too little context to LLMs than too much. Laziness is a good technique only when you’ve learned how to provide enough context, and then deliberately step back to see how little context you can get away with and still have it work. Also, lazy prompting applies only when you can iterate quickly using an LLM’s web or app interface. It doesn’t apply to prompts written in code for the purpose of repeatedly calling an API, since presumably you won’t be examining every output to clarify and iterate if the output is poor. Thank you to Rohit Prsad, who has been collaborating with me on the open-source package aisuite, for suggesting the term lazy prompting. There is an analogy to lazy evaluation in computer science, where you call a function at the latest possible moment and only when a specific result is needed. In lazy prompting, we add details to the prompt only when they are needed. Original text: deeplearning.ai/the-batch/issu…

Interested in doing a PhD in French psycholinguistics 💻? We are looking for someone fluent in French and interested in Romance Languages. 3-year position (extension possible). 📅 Application deadline: December 20, 2024. Details: linguistlist.org/issues/35-3312

There are 2 mistakes you can make about LLMs: ① Thinking everything LLMs say is correct, they can reason, and with a bit more scale they’ll get us to superintelligence ② Thinking LLMs are good for almost nothing—they are FAR better at all #NLProc tasks than previous methods

East/West Germany's phantom borders More below

Interested in doing cross-linguistic comparisons using eye-tracking 👁️? Come do a 4-year postdoc with us! We are looking for someone fluent in Spanish and/or German 📅 Application deadline: March 1, 2024. More info: linguistlist.org/issues/35-369/

Great overview of compression algorithms for LLMs. Covers compression algorithms like pruning, quantization, knowledge distillation, low-rank approximation, parameter sharing, and efficient architecture design. This space is moving so fast. This is just a nice overview which also includes future research ideas and topics.

Can you train an LLM from scratch using only the text that a human might plausibly read? Come find out today at 3:30PM! The organizers of the BabyLM challenge, the #CoNLL2023 shared task, will be presenting 🤩 babylm.github.io

We are happy to announce that we can once again offer Junior (3-6 months) and Senior (1-3 months) short-term fellowships to the #SFB1252. They are available in 2024.🎉 Apply by December 31st 🗓️and find more information on how to apply here. ▶️ sfb1252.uni-koeln.de/sites/sfb_1252…

“Cha" and "te" are both Chinese words for tea. If a territory came into contact with the drink by: • The Silk Road = "cha" • Sea shipping routes (starting with Dutch traders) = "tea"

📢📢CALL FOR ABSTRACTS📢📢 Interested in Naturalistic Approaches to Reference? Then submit an abstract for our workshop at next year's #DGfS2024 in Bochum 🗣️🧑‍💼👥

If IRIS indeed does clustering of papers based on context rather than citations, it seems really useful to bypass the citation bias

People are testing large language models (LLMs) on their "cognitive" abilities - theory of mind, causality, syllogistic reasoning, etc. Many (most?) of these evaluations are deeply flawed. To evaluate LLMs effectively, we need some principles from experimental psychology.🧵

We still hear the cries for help from those trapped under the rubble. Many of our own families and neighbours have not survived. Thousands in #Syria are dead. Thousands more are missing. Please help us in our response by donating. #Earthquake gofund.me/8602b47f

India just passed China as the most populous country in the world. Why? Because of the biggest accident in history Look at where people live in India. What's that band up north?

Announcing the BabyLM 👶 Challenge, the shared task at @conll_conf and CMCL'23! We’re calling on researchers to pre-train language models on (relatively) small datasets inspired by the input given to children learning language. babylm.github.io arxiv.org/abs/2301.11796

Here is a talk a gave at AMLAP summarizing some of the points we make in BBS paper “Deep Problems with Neural Network Models of Human Vision”: youtube.com/watch?v=7C_0vB… Still a few days to write a comment. Good way to spend a blizzard of a Christmas? cambridge.org/core/journals/…

... and that's a wrap on #CoNLL2022 ! Thanks to all the organizers, reviewers, authors, and attendees for making #CoNLL2022 what it was! ♥️ Looking forward to #CoNLL2023 already... It will feature a really cool shared task! 😃😃 babylm.github.io Stay tuned 📺 👋👋🎆🎆

Support: General Linguistics: PhD, University of Cologne, Cologne, Germany: The Collaborative Research Centre CRC 1252 “Prominence in Language” in Cologne, Germany, announces short-term Junior fellowships available between April and December 2023 for a… dlvr.it/Sdv9nf