Samir Khan

Large language model validity via enhanced conformal prediction methods Excited to share my work with Isaac Gibbs (@iswgibbs) and Emmanuel Candès arxiv.org/abs/2406.09714. Question: how can we provide practical and rigorous validity guarantees for LLMs? (1/12)

See the arXiv draft for more theoretical results, simulations showing how our bounds can be used to inform experimental designs, and comparison of complete randomization vs. Bernoulli randomization! arxiv.org/abs/2405.07979

More broadly, these results suggest that rather than asking “what is a good estimator for a Bernoulli design” or “what is a good design for an IPW estimator” we should be asking “what are good estimator/design pairs”

Excited to share new work with @jugander, Matt Eichhorn, and Christina Lee Yu on combining experimental design and outcome modeling for causal inference under network interference arxiv.org/abs/2405.07979

[Please RT] I’m recruiting PhD students to work with me at @UW! I’m looking for students passionate about using computational methods to study how social platforms can be reimagined to enable better conversations, bridge political divides, and reduce the spread of misinfo. >>>

New working paper with a fantastic team of coauthors: we study when and how to design cluster experiments with network spillovers. arxiv.org/pdf/2310.14983… with @lihua_lei_stat , @guido_imbens, Brian Karrer, Okke Schrijvers, Liang Shi

@kylefbutts You can get something similar to this using theme_economist from the ggthemes package yutannihilation.github.io/allYourFigureA…

I'm on the job market and can't let myself get distracted by dumb open source ideas, but it would be really cool to have an easy way in ggplot to put the legend like this and the y-axis label like this. NBER graphs always are 👨🏼‍🍳🤌🏼

@Apoorva__Lal yes! everything reinvented everywhere all the time

@stats_samir Prognostic score adjustment! It is a good idea. twitter.com/UnibusPluram/s…

A thread about CUPAC/CUPED - the tl;dr is that "CUPAC" is nearly equivalent to G-computation/Oaxaca-Blinder/imputation methods with a somewhat unusual choice of imputation model doordash.engineering/2020/06/08/imp… 1/10

@VincentAB very neat, thanks!

@stats_samir A trivial tip: The Lin strategy does not require centering; it only facilitates interpretation. Taking the slope of the uncentered model is equivalent. In this example, it's basically a wash, but the 2nd approach may be more convenient when there are lots of covariates:

So CUPAC is intimately connected to certain other methods for ATE estimation in experiments, and these connections also suggest possible extensions of/improvements to CUPAC drawing on the literature around those other methods, 10/10

Similarly, it models Y_i(1) by taking the non-parametric model for Y_i(0), and learning Y_i(1) as a linear transformation of Y_i(0) on the experimental data - note that this relies on Y_i(1) being an approximately linear function of Y_i(0) 9/10