Vincent Arel-Bundock

Whoa—my book is up for pre-order! Model to meaning: How to interpret Statistical & ML Models in #RStats and #PyData The book presents an ultra-simple and powerful workflow to make sense of ± any model you fit Note: The web version stays free forever. tinyurl.com/4fk56fc8

We have a new version of this paper out. The headline results are the same—political science must filter results heavily for statistical significance—but we've added many extensions and rewritten much of it in response to feedback (thank you!). A quick thread on updates 👇

@dylanarmbruste3 have had this in my bookmarks since forever. Maybe it's time. Thanks for the reminder!

@VincentAB I feel like this has the closest thing to it: raps-with-r.dev Using Quarto basically.

Do you know a good framework to document data analysis choices? Ex: dropping obs, recoding a var, tuning pars, model + test statistics, etc. Ideally human-readable so people don't have to read my entire codebase. Low-maintenance+ machine-readable for bonus points! #RStats #PyData

@adamaltmejd @haozhu233 Exactly! I'm putting a detailed spec together and starting to think about tooling. Maybe I'll ping you for feedback if you're interested.

@VincentAB @haozhu233 Ah yes that's interesting. Perhaps a kind of agent-driven testing scheme where each "decision" gets its own test where an agent is tasked to verify in code that the thing is actually being done. Not deterministic of course, but could work pretty well as verification.

@adamaltmejd @haozhu233 Ideally, these things should be reported in the paper, but Many-Analyst Projects showthis is far from the case. I think that if we can have a LLM assisted workflow to create "candidate decisions" and have humans approve them, that could be really useful. LLM-powered Lab notebook

@adamaltmejd @haozhu233 I mean more: "We report clustered standard errors at the level of random assignment: villages", with some justification and reference to literature, and link to the code line where this is done.

@adamaltmejd @haozhu233 Yeah, right. But an important desideratum is to distinguish "state" from "decision". I want a record of "analysis decisions" that could then be compared to code state by an agent to check for faithfulness. The LLM seeds "candidate decisions" that need explicit human approval.

@VincentAB @haozhu233 Curious if you find anything! On a specificity/determinism scale between AI summary and actual code, I'm not sure you can get much better than a well-prompted summary in a .md file without it being code.

@haozhu233 Yes, of course. But I was more thinking about something along the lines of a Architecture Decision Record, that the user "approves." Include both choice & justification. The AI can refer to it when implementing and to verify faithfulness. And humans can eval high level decisions.

@VincentAB AI generated code summary.

@__gsteck__ Looks fantastic!

We refactored Bambi's interpret module, extending its features to align more with @VincentAB R's marginaleffects.

This is infuriating. #NullEffectsMatter

@KyleLHandley @bechhof You can treat maintainers with respect and get the ***free*** stuff you want. Win-win.

@KyleLHandley @bechhof Instead of throwing insulting complaints up on social media ("anyone who picks up a textbook"), perhaps you could ask the maintainer (me) it we could convert the error into a warning. I am always happy to engage with users, listen to feedback, and fix (potentially) bad decisions.

So just to complain about open source R a bit. Somebody decided the "marginaleffects" package should not work with PPML models because of FE uncertainty that definitely apply in some model prediction (but NOT my application). So they just broke the whole thing instead. Why can't users just make their own mistakes? github.com/vincentarelbun… "car" package will still do what I want anyway, but not I have rewrite my teaching code, my slides, and my solutions for students. The problem of fixed effects being concentrated out of PPML and conditional logit models is well know. STATA will give you a warning on this and anyone that can be bothered to pick up a textbook can know this. Weird unilateral decisions by package managers probably breaking tons of code for past 7 months.

Say I wanted ~9k USD to buy out the teaching time of a prof (a brilliant coauthor of mine) so he can work on creating a validated ground truth dataset for a challenging data extraction and labelling task that current public LLMs (e.g. GPT5) have not yet saturated. Who do I pitch?

Discover how rare it can be to uncover meaningful effects in political science research in this week’s episode of Not Another Politics Podcast. Listen here: har.rs/493JVn3

@robma1982 routledge.com/Model-to-Meani…

@VincentAB the link seems broken

Whoa—my book is up for pre-order! Model to meaning: How to interpret Statistical & ML Models in #RStats and #PyData The book presents an ultra-simple and powerful workflow to make sense of ± any model you fit Note: The web version stays free forever. tinyurl.com/4fk56fc8

@xuyiqing I've been playing with your ideas for Step 2 and found a super cute 1-liner. Using the `transform` argument of the `comparisons` function, we can spline dydx immediately, instead of using cumbersome bins. Not sure this is actually useful, but it's too cool not to post!

It works for simple cases, but both steps may have some drawbacks, e.g., binning works not as well at boundaries. Thx a lot for digging into it!

@VincentAB Thought a bit more. CME estimation with cont. X typically involves: (1) Get signals for partial effect/CATE given covariates. (2) Reduce to 1D along X (e.g., binning, kernel, spline). Your implementation uses GAM numerically for (1) and binning for (2).

@xuyiqing Oh yeah, doubly-robust is the good stuff! I also really like the part of your paper about dimensionality and regularization bias. A very clear and concise treatment. Good work!

Thx, Vincent. Agreed! I didn’t know about this fn in marginaleffects -- super useful! Our point on the critique’s *implementation* of GAM stands. Moreover, accommodating additional Z and regularization bias remain challenging. Doubly-robust estimators generally perform better.

@xuyiqing Interesting paper! Thanks for posting. IIUC, your main concern with the GAM approach is that it targets the wrong estimand. If so, I feel that your criticism of the approach is kind of unfair, given that it's easy to target CME w/ GAM. See this notebook: arelbundock.com/hmx_simonsohn.…

1/ Recently, Professor Uri Simonsohn critiqued Hainmueller, Mummolo & Xu (2019), arguing that the proposed methods fail to recover the conditional marginal effect (CME): datacolada.org/121 We appreciate the critique and offer this response: arxiv.org/pdf/2502.05717 🧵

@LeoBaccini That's all fine but, fundamentally, I'm not convinced that the president is actually looking for policy concessions. And I worry that spending more CAD in these areas will embolden further extortion attempts.

Paradoxically, a trade war between bordering countries is at risk of worsening security border issues or to create problems that do not currently exist. And I leave the economic costs out in this thread. 11/11

The US declared to impose 25% tariffs on imports from Canada and Mexico (effective from Tuesday). Canada has retaliated with 25% tariffs on US imports. A trade war between these two major trading partners brings no benefits to their economies and their citizens. A thread: 1/11