Jinu Lee

🧑‍⚖️Rubrics proven to be powerful in legal NLP, too! Introducing LEGIT (LEGal Issue Trees): a novel legal reasoning dataset with rubrics for evaluating the reasoning traces. arxiv.org/pdf/2512.01020 (1/7)

🚨 Medical AI Research Alert! 🚨 How do we ensure AI patient simulators in mental health training truly reflect diverse, realistic human behavior? @UofIllinois presents 𝗣𝗦𝗜-𝗕𝗲𝗻𝗰𝗵, an interpretable, clinically grounded framework for evaluating depression patient simulators. By Nguyen Khoi Hoang(@hknguyen20 ), Shuhaib Mehri(@ShuhaibMehri ), Tse-An Hsu, Yi-Jyun Sun(@jyun_sun), Quynh Xuan Nguyen Truong, Khoa D Doan(@khoaddoan), Dilek Hakkani-Tür(@dilekhakkanitur ) Now you can watch and listen to the latest Medical AI papers daily on our YouTube and Spotify channels! YouTube Deep Dive: youtu.be/WzfCw8QeH6M YouTube Shorts: youtube.com/shorts/2A7_RE4… Spotify: open.spotify.com/show/4edRuSTOu… Here's why it's exciting: 👇🧵 1/n #MedicalAI #Healthcare #MentalHealthAI #SimulatorEvaluation [1/9]

@Bollegala @emnlpmeeting @sunipa_dev The case you mentioned is already impossible due to the double submission policy. Per my understanding, what you *can't* do is to opt in for AACL at submission time, and get better scores than expected so end up committing to EMNLP instead

@emnlpmeeting @sunipa_dev Does this mean a submission that selected EMNLP and then committed but got rejected will not be able to commit to AACL?

📢 Please note a critical update to the original #EMNLP2026 Call for Papers: since EMNLP and AACL share the ARR May cycle, authors will need to explicitly select a target conference at submission time. This choice will be binding! For more details see: #paper-submission-information" target="_blank" rel="nofollow noopener">2026.emnlp.org/calls/main_con…

@dela3499 Nice puzzle! Some more: S _ _ _ _ _ _ _ _ word 1: happiness word 2: horror _ _ _ _ _ O word 1: organizing things word 2: organized moves B _ _ _ _ _ word 1: found at mountains word 2: found at rivers _ _ _ _ _ E word 1: throw out word 2: scratch out

Trying to make a new genre of word puzzle. Fill in the blanks!

wrote a guide on getting compute grants as a student, something I wish I did more at the beginning of my PhD. It's honestly one of the highest ROI things you can do as a student (we've gotten 100k+ gpu hrs for roughly 2 weeks of work writing). nightingal3.github.io/blog/2026/04/1…

@RyoKamoi ahh that's unfortunate... hope to see you soon

@jinulee_v Thank you! But I will not be able to attend the conference 😭

🎉 Our paper has been accepted to ACL 2026 Findings! We propose FoVer, an efficient method for creating accurate PRM training data using formal verification tools 🚀 Our tool-annotated formal synthetic training data improves PRMs on math, NLI, and BBH 😳 arxiv.org/abs/2505.15960

Do all reasoning tokens matter equally? We study the functional importance of reasoning tokens implicitly encoded in LLMs. Work w/ my cool advisor @dilekhakkanitur at @convai_uiuc; to appear in ACL Main 2026. #ACL2026 Paper: arxiv.org/pdf/2601.03066 Demo: greedy-token-pruner.github.io

Finally, *Scaling evaluation-time compute* got accepted to ACL Findings! See you all at SD 🏖️ Since this paper was released, many have sought scaling evaluation-time compute via "reasoning" PRMs and LLM-as-a-judge+rubrics. Still more to explore in 2026!

*Legal Issue Trees* got in ACL Main! If you are interested in reasoning evaluation, legal NLP, or both, let's chat in San Diego 🏖️

Working on speech and language processing in Midwest? Submit your abstract and join us at UIUC mid April!

@teortaxesTex I'd say it is a minor revision of existing papers, e.g., R1-thoughtology, Thought anchors, etc. Section 5.2. seems new in the context of reasoning model distillation, but the fact that many of their analyses are trivial will not change.

@jinulee_v It's a cool paper even if you remove all the molecular framing it's going viral because of the framing though I think the framing is excusable because Seed also publishes heavily in protein structure prediction

Honestly, the analogy between the logical structures of CoT and molecular forces is very stretched. They just measure the distance between two paragraphs, using attention weights, and distilling from two teachers. Can anyone explain to me why this is going viral?

@DrDatta_AIIMS Visuals are indeed cool, which bothers me even more 🤣

@jinulee_v I am a cool guy - and the visual was cool - with a cool caption 🤣

Thank you for shouting out my preprint on rubrics for legal reasoning! Evaluating reasoning traces with rubrics is a promising direction, but constructing a good rubric for expert-level tasks remains a significant challenge.

@cwolferesearch I believe designing rubrics for expert domains (e.g., law) still requires domain-specific approaches to achieve *useful* improvement. My project emphasizes designing and validating legal rubrics from the experts' perspective for RaR: arxiv.org/abs/2512.01020

I've been reading a lot about rubrics-as-rewards (RaR) for RL. Some of my favorite papers (so far): 1. arxiv.org/abs/2507.17746 2. arxiv.org/abs/2508.12790 3. arxiv.org/abs/2510.07743 4. arxiv.org/abs/2511.19399 5. arxiv.org/abs/2507.18624 Most of the added technical complexity of RaR is less related to RL and more related to reward modeling. If we can get a reliable reward signal, RaR works well, but teaching a model to perform granular / instance-level evaluation is tough. Generalizing these evaluation capabilities across arbitrary domains is even tougher (especially those that are highly subjective). Our reward model also needs to avoid hacking in large-scale RL runs. In my opinion, new developments in this space are likely to come from advancing the frontier of (generative) reward models rather than RL. So much to be done.

To all midwestern #NLP-ers: 2026 Midwestern Speech & Language Day (MSLD) returns to UIUC! Best chance for searching new ideas and making new friends 🚀 - Abstract submission due: March 2 (Non-archival) - Events: April 15-16, UIUC

[Long Tweet Ahead] Faculty Interview Tips & Common Questions: 🧘‍♀️0. Firstly, do not be nervous - Almost everything can be prepared in advance:) - Be grateful for everyone's time. - Think of it as an opportunity to share your research with others -- exciting, right? - Technical issues WILL happen -- no worries. - Try meditation! (seriously, it helps me tremendously with the interview marathon) 🚀 1. The MOST crucial part: Research Vision This is what keeps me up at night (literally!), trying to distill my entire research agenda into one powerful sentence. It is like crafting your research tagline/punchline/slogan. What is your unique contribution? What stands you out? Here is the thing: it is fundamentally why the university wants to hire you. They want to see you as a rising star for the next few years, someone who can make the university name become associated with some impactful research. Think about it: when people want to learn about a specific topic, they immediately think "Oh, I should check out X's work because they are THE person for this". The university is not just hiring a researcher; they are investing in a vision for the future of the field. The key is to come up with a punchline that captures your research identity and repeat it all the time during the talks and onsites. Ask yourself: - What will your name be associated with in the next decades? - Where is your field heading? (Is RoboGPT the future? Is Transformer really the final architecture?) - What are the REAL unsolved challenges? (Not just throwing more data at problems) Get ready to discuss: - Are large models really the future? Can we achieve true intelligence just by scaling up? - What about data bottlenecks? Is synthetic data reliable? What are effective ways to collect data? - Are models really do reasoning? Do we need symbols/structures? - Is Transformer is the final answer? What is the bottleneck of Transformer? - What are the new tasks we really need to focus on? - How do you think of the current research trend of creating evaluation benchmarks? - What is still fundamentally missing in current research? 🤓 2. The BIG question: "Why Academia?" This is actually what you should confirm multiple times with yourself. It is really about your passion and motivation: - What your happiest moments (I talked about those late-night breakthrough moments haha) - Where do you see yourself in 50 years? (Dream big! Talk about the research institute you want to build, the problems you want to solve, the leader you want to become in your field) - What is your ideal group size and resources you will need? (Be concrete!) - Are you also looking at industry jobs? Here is the real talk: we are in the age of large AI models requiring infinite GPUs. So you need to have solid answers about: - Why choose academia NOW? - How you will position yourself in this large model era - The practical stuff, like how you will handle GPU needs (I will concretely mention XX research directions that don't need massive compute, XX research require GPUs but I have XX potential funding sources, and collaboration opportunities with XX) 💼 3. Logistics about Application Materials 3.1 Application Materials: - Use figures. It is always what people firstly check when reading long documents. - DO NOT miss deadlines! You can usually update materials after submission. 3.2 Personal Websites: - CV and websites are very important (I personally feel it is even more important than research statements, or at least equal) - Two Must-Haves for your website: (1) CV (fresh and updated!), research statement, teaching statement, diversity research statement (since people may not be able to find your package quickly during onsites; the research statements can always be updated if you have better ideas of your storyline) (2) your email address: make it OBVIOUS. ⏰ 4. Logistics about Timeline 4.1 My actual timeline: - First phone interview: Dec 14 - First onsite: Jan 11 - Last onsite: Apr 7 4.2 If you are asked to choose an interview slot: - Most importantly, figuring out whether it is "rolling-based" or not. - Rolling admissions? Interview earlier! - Non-rolling? Later interviews = more practice = better performance 4.3 Timing matters: - Schedule your dream schools for mid-Feb to early-Mar (Most universities I interviewed with after mid-March did not extend an offer, but almost all my January and February interviews led to offers, while the universities are similar ranks) - Health tip: Protect yourself from COVID during Jan-Feb interview season (I have to reschedule several interviews, learned this one from experience!😷) 🎙️ 5. Logistics about Phone Interviews Let us talk about something that makes everyone nervous: the interview process. I have learned that preparation is KEY. Let us go through step by step. 5.1 The "Why THIS School" Question (some universities even ask this as the first question, so I started to do more preparations on this part): - First think about what makes the university special (Is it known for something unique? What research centers do they have?) - Name drop (respectfully!) potential collaborators in the department - Track their recent wins (I always check department news before interviews) - Think about location benefits (research collaborations, funding opportunities, industry connections) Pro tip: Keep a cheat sheet with specific details for each school. Trust me, it helps when you're on your 5th interview and the details start blurring! 5.2 Research Vision 2.0 (School Edition) This is where you customize your research vision for THEIR context: - Show how you fill a unique gap in their department - Paint an exciting picture of future collaborations (for example, when listing your future direction, you can say: I am excited about this future direction xx and xx university is perfect for me since I can collaborate with faculties xx, and research centers xx. ) 5.3 Teaching Plans: - Specific course numbers (both undergrad and grad levels) that you could teach - Your dream course ideas (I actually created a full syllabus for a Multimodal Machine Learning course and put it on my website. Having concrete materials ready shows you are serious about teaching) 🎤 6. Logistics about Onsite (The Big Show) Alright, let us talk about the main event, the onsite interview! This is where the real magic happens, and a lot of things can be prepared as always. 6.1 The Job Talk: Your Moment to Shine ✨ Let's be real: this is THE most important part of whether you can get the offer (others are all minor). Still, your research vision is the most important: - Boil down your idea to ONE powerful sentence (and repeat it strategically!) - The first 10-15 minutes are GOLD. Some dept chairs only stay for this part. Be sure to show your research impact. - The goal is to EXCITE people about your research. I always start with a walkthrough example (this works way better than diving straight into theory) - Guide viewer attention for EVERY. SINGLE. SENTENCE. (Use animations, strategic dimming, highlight what matters) - Time management is crucial: Aim for 40 mins + Q&A (And be prepared that talks often start late! Factor in technical issues and waiting for people and other things) - I add a progressive bar to help people track my talk. 6.2 Handling Q&A Like a Pro 🎯 - Drop mini Q&A slides after your first and second sections (if you want to increase interactions with audience, this works!) - Golden rule: Be concise + logical - Common questions to prep for: "How do you handle bias/safety issues in model learning? What about adversarial attacks?" "How do you create data in model learning?" "Would you say your work leans more towards ML theory or applications?" "I do not think it is the right way to get it work, what about xx" (I feel a lot of audience will be outside your area and when they try to connect to their direction, there will be far-out questions you never think about, or you will face challenges saying they do not believe black boxes or do not believe symbols, or do not believe some other things. It is totally okay! Do not panic! You should always be confident about your direction. No need to get irritated or defensive. No need to back down or bluntly disagree / turning it into a debate. Just treat it as a research discussion. Something like: I think it is an interesting angle. At the current research stage, I believe that my way is the most reliable and practical way of handling xx, however, later I would be happy to explore more on xx and it would be great if we could even collaborate together on this. ) 6.3 Surviving the Marathon: One-on-Ones 👥 I am super introverted, and not good at small talks, so it is more a guideline for introverted people haha. I feel these are not really about casual chats. Each one is a mini-presentation opportunity. - I heard that people are saying one-on-ones just try to see whether you are nice. I do not agree. People won't hire you because that you are nice, but more because of unique, exciting insights that you can bring, which can make you get high voting scores. - Do your homework on EACH professor: • Check their recent papers (Google Scholar, sort by time) • Know what made them famous (sort by citations) • Look up their grants and awards • Find personal connections (alma mater? city connections?) 6.4 Lunches & Dinners 🍽️ Again since I often worry about being too introverted, I like to prepare talking points in advance. I usually focus on my strengths, such as research, mentoring philosophy, and funding applications. If you happen to know something interesting about the city or the food, that is a great conversation starter and a bonus! (Job search season is here again! I have been receiving DMs about faculty interview advice, so I thought I'd share a few key insights that personally helped me navigate the process. If you have already seen the slides I shared earlier, this is essentially the same content. Just a heads-up to save your time!)

Virtual at #NeurIPS2025 👋 Exploring industry research/applied scientist roles CS PhD @UofIllinois -- I work on grounded instruction following in embodied dialogue, with an emphasis on spatial intelligence + low-data regimes DMs open if you're hiring. Details/links below! #NeurIPS #NLProc #AI @uiuc_nlp @siebelschool

We hope to expand LEGIT across diverse jurisdiction systems! If you are interested, feel free to contact me to push the frontiers of legal reasoning together. Huge thanks to my collaborators for their feedback and guidance: Kyoungwoon, @HanSineng, @armancohan, and Julia. (7/7)

Finally, we compare the effects of RAG and RL with rubrics-as-rewards in the LEGIT dataset. RAG improves both correctness and coverage. However, RL pushes models to focus on “safe” issues while discarding uncertain ones, boosting correctness but at the cost of coverage. (6/7)

🧑‍⚖️Rubrics proven to be powerful in legal NLP, too! Introducing LEGIT (LEGal Issue Trees): a novel legal reasoning dataset with rubrics for evaluating the reasoning traces. arxiv.org/pdf/2512.01020 (1/7)