Miao LI

Major difference in my mind: - an engineer, given a problem, invents and tries multiple solutions and stops when the solution is good enough. The goal is product innovation and shipping. - a scientist asks new questions, proposes various new solutions, compares them (sometimes with old ones), and writes about it. The methodology must be sound or else peers will sneer. The goal is scientific breakthroughs and technological progress. Both can be called "researchers". Many people can do both: these are activities, not identities. Importantly, most product innovations are built on scientific breakthroughs and technological innovations that happened 2, 5, 10, or 20 years earlier.

A seemingly better solution for peer-review: Stage-1: reviewers write comments, no scores; Stage-2: authors provide rebuttal/clarification for multi-round discussions; Stage-3: reviewers give scores with supporting evidence based on the discussions at least responses.

I've talked about this in various panels, keynotes and forums: foundational model providers are likely going to be similar to large telecom providers. They provide crucial infrastructure, they're very expensive to build and maintain, they create a lot of value in the ecosystem but they might not capture that value long term. You can't build an Uber or Google maps or tiktok without good pervasive internet. But telcos don't get the majority of that value. Foundational model providers are likely similar and that's why we haven't invested in such companies at AIX Ventures. It's also why I'm very bullish on the future of ydc and the big partnerships we're working on with larger enterprises and publishers. We're helping them with accurate answers, agents and AGI over their own and public data. Just like the coal of Jevons paradox (im glad to see others have picked that up also) and internet bandwidth before it, more efficient intelligence will lead to us using it in more and unexpected places.

Can #LLMs truly reason over loooong context? 🤔 NoCha asks LLMs to verify claims about *NEW* fictional books 🪄 📚 ⛔ LLMs that solve needle-in-the-haystack (~100%) struggle on NoCha! ⛔ None of 11 tested LLMs reach human performance → 97%. The best, #GPT-4o, gets only 55.8%.

This is one reason why I still keep working on language generation while people saying it is ‘solved’ by LLMs or not interesting anymore.

I just felt all benchmarks are like this when I read LLM benchmark papers, e.g., the HELM evaluation crfm.stanford.edu/helm/, and evaluation in LLM technical reports, e.g., the Llama3.1 report arxiv.org/pdf/2407.21783.

As a final-year PhD student, I really appreciate the analysis on the sources of our anxiety and frustration about careers and future!

My rule in meetings: “you are not allowed to say the word agents. Talk about the problem you are trying to solve”

Tree search, the key idea in classical AI, has little to do with true intelligence or reasoning, no matter which fun puzzle / games are well solved by search eg game 24. Search is just a tool usage. Surprised to see so many regard search as reasoning to pursue.

In today's publication culture, most authors are after being SOTA, showing tables with 𝐛𝐨𝐥𝐝 numbers, and writing the minimum viable paper! The goal of a scientific paper should be to push the field forward with new intuition/insights on how to think about solving a problem.

If you want to know how to fuzz the full program environment, check out our latest work with @GJ_Duck and @AbhikRoychoudh1, accepted by @acm_ccs. This work is designed to fuzz everything with program environment fuzzing. 👉Preprint: mengrj.github.io/pdfs/EnvFuzz-C… [1/2]

OLMo supports Intra-Document Causal Masking now 🤗

How do black-box neural networks transform raw data into predictions? Inside these models are thousands of simple "components" working together. New MIT CSAIL research (bit.ly/473lcfE) introduces a method that helps us understand how these components compose to affect model behavior — a key step in making neural networks more interpretable. 🧵

We won an Outstanding Paper Award at #ACL2024NLP for our efforts to bring some order into the field of media framing research. Massive congratulations to my wonderful colleagues @YuliaOtmakhova and Shima @shinyemimalef 👉 frermann.de/dataFiles/Medi… 👉 github.com/julia-nixie/aw…

I will present our poster "Analysing The Impact of Sequence Composition on Language Model Pre-Training" at 10:30am🚀🚀🚀 (Oral presentation had presented on Monday) Welcome to have a chat if you are interested in language model pre-training🤗 #ACL2024 #ACL2024NLP

Here is a better one! 🏴󠁧󠁢󠁳󠁣󠁴󠁿🇹🇭 #ACL2024NLP

🚀

If your #hatespeech research is motivated by a desire to make the web a safer place, you don't want to miss our new #ACL2024NLP paper: ✨"Explainability and Hate Speech: Structured Explanations Make Social Media Moderators Faster"✨ @Edin_CDT_NLP @EdinburghNLP @Snap #NLProc 1/6

- A Sentiment Consolidation Framework for Meta-Review Generation, arxiv.org/abs/2402.18005 - NewsBench: A Systematic Evaluation Framework for Assessing Editorial Capabilities of Large Language Models in Chinese Journalism, arxiv.org/abs/2403.00862

Excited to be giving in-person presentations of two papers at ACL'24 on Tuesday next week in Bangkok. Look forward to meeting old friends and making new ones!