XiulinYang

@_Suresh2 @weGotlieb Thanks for your question! Based on our current results, it’s hard to interpret whether frequency or position matter more, but we do find that disrupting structural predictability makes the LMs drop the most (Table 2 & Figure 7)

@xiulin_yang @weGotlieb neat angle on function words. did frequency matter more than linear position?

🎉 Happy to share that our paper on function words & language learning (w/ Heidi Getz & @weGotlieb) is accepted to #ACL2026! A little late to the party, but still worth celebrating 🥳 We ask: what statistical properties help a learner abstract grammatical knowledge from linear input? Turns out function words, though often overlooked, play an important role. Check out our updated preprint: arxiv.org/pdf/2601.21191 🧵 1/4

One more interesting finding is that a 5-gram LM shows a different learning pattern as transformers in terms of function words. This suggests that transformers develop syntactic generalization beyond linear heuristic rules. 5/4

Finally, we probe LM internals via attention analysis and ablation experiments to understand how much models actually *rely* on function words during processing. - When all three properties are intact -> models show the largest performance drop when function words are removed - When structural predictability is disrupted -> models shift away from function words and lean on other cues instead I'll be at ACL this summer and happy to chat with anyone interested!😀 4/4

Announcing a new version of our 2024 paper on linguistic hypothesis generation from LMs! @najoungkim and I have systematized our hypothesis generation framework, added stringent criteria for model selection, 10x-ed our learning trials, and included an epigraph from Jeff Elman!

I’m at #HSP2026 at MIT this week! I’ll be giving a talk Friday at 5:25pm entitled “Structural Priming Effects in Language Models are Less Human-like in Languages Other Than English”. Looking forward to chatting to everyone!

👀Ever wondered how visual information quality affects reading and language processing? Our new #EMNLP2025 paper, “Modeling Bottom-up Information Quality during Language Processing”, bridges psycholinguistics and multimodal LLMs. 🧠💡👇 arxiv.org/pdf/2509.17047

I will be recruiting PhD students via Georgetown Linguistics this application cycle! Come join us in the PICoL (pronounced “pickle”) lab. We focus on psycholinguistics and cognitive modeling using LLMs. See the linked flyer for more details: bit.ly/3L3vcyA

I’m in Vienna all week for @aclmeeting and I’ll be presenting this paper on Wednesday at 11am (Poster Session 4 in HALL X4 X5)! Reach out if you want to chat about multilingual NLP, tokenizers, and open models!

If you are interested in details, feel free to check out the paper arxiv.org/abs/2502.18795 & visit our poster on Wednesday at 11am! Looking forward to discussing more on #Typology, #Multilinguality, #Learnability, #CognitiveScience and other related topics!

🥳Thrilled to present our #ACL2025 paper with @t_aoyam, @yuekun_yao, and @weGotlieb on language modeling! Can LMs’ learning dynamics distinguish typologically attested from unattested and from truly impossible languages? We test this across 12 languages and find: yes, but…👀

Tokenization is just a special case of "chunking" - building low-level data into high-level abstractions - which is in turn fundamental to intelligence. Our new architecture, which enables hierarchical *dynamic chunking*, is not only tokenizer-free, but simply scales better.

Come do research with me, my fantastic colleagues, and some of the coolest PhD students I've ever met! #NLProc #nesy #neurosymbolic #AI #ML

@im_samin Thanks! Will check it out🥹

@xiulin_yang Catastrophic forgetting of the language model? It happens when you sequentially finetune with new datasets.

📢 *Looking for a PhD student!* @hedderik and I are starting a super exciting project on using LLMs and adaptive methods to improve learning of complex knowledge, and address implicit biases. Want to do a PhD with us? Get in touch (asap)! @GroNlp @univgroningen @JTSchool_UG

This is how I advise my #PhD students to write research manuscripts (in case someone finds it helpful). General points: 1. Research questions addressed by your manuscript are key and should guide you. 2. Don’t view your manuscript as an article. See it as a STORY. 3. Pick the writing style that is easily understood by a broader community. Make reading easy. 4. Most of data should get into the paper. If some doesn’t support the hypothesis, it still must be in the Suppl. Information. It must show the reproducibility limits. 5. Make the paper shorter, not longer. Cut out things that may sound like ‘bluff’ or ‘decoration’ of the story. Use well-defined terminology, don’t invent it unless clearly necessary. 6. Focus on reporting & explaining the numbers. Minimize discussions of qualitative outcomes and your imagination. ▫️ Specific steps: 1️⃣ First, formulate and polish the key questions that your study addresses. It may take hours or even days (even though you've been doing research in this area for years). A single study should address no more than 1-3 key questions. It’s your perfect start for writing. 2️⃣ Write down the structure of your STORY first: Sections and Subsections that will answer those questions. Into each subsection, put 1-2 sentences that formulate the message(s) from this subsection. It will hugely help you navigate the manuscript later and save a lot of time. 3️⃣ Write approximate messages in the conclusion section. Usually, no more than 1-4 sentences. ▫️ At this point, SHARE your [structure+questions+messages] document with your advisor for feedback. Toss it back and forth until you both converge. You can also include major collaborators if needed. ▫️ 4️⃣ Write the introduction part. Put down the paragraphs that introduce a reader into the key question(s) of the manuscript and the background of your story. 5️⃣ Write the main text for each section, smoothly and firmly. Each paragraph should add a separate value and end with a message-like sentence. Follow the “First… Second… Third…” structure for paragraphs when possible, it gives rigor and readability to your story. 6️⃣ Write the conclusions. Add a broader perspective that is justified and not generic. 7️⃣ Write the abstract. It must have simple terminology and clearly explain what readers can find inside the paper. It also should contain the key conclusions. 8️⃣ Write up 4-5 different titles and spend >30 mins with your team discussing which title sounds best. Finally, iterate on the resulting draft within your team. The number of drafts can easily exceed 20. ▫️ ❗In addition, I always emphasize that a high quality of your research paper: - sharpen your writing and analytical skills. - shapes your reputation. - shows who you are as a researcher and communicator. ▫️ p.s. Everyone has a different style of advising and writing. You can adopt only some specific steps if you find them helpful. p.p.s. Another way that we sometimes use is by starting with figures ('story in figures' style). #AcademicTwitter #AcademicChatter

Now hiring: Three more PhD students and one postdoc for research on combining neural and symbolic/interpretable models of language, vision, and action. Work with world-class advisors at @Saar_Uni, MPI Informatics, @mpi_sws, @CISPA, @DFKI. Details: neuroexplicit.org/jobs

Now hiring: Six PhD students and one postdoc for research on combining neural and symbolic/interpretable models of language, vision, and action. Work with world-class advisors at @Saar_Uni, MPI Informatics, @mpi_sws, @CISPA, @DFKI. Details: neuroexplicit.org/jobs