Charlie Marx

Denied parole by an ML model? The next best model might have decided otherwise In our #ICML20 paper w @berkustun @FlavioCalmon, we study the ability for an ML problem to admit competing models with conflicting predictions, which we call "predictive multiplicity" THREAD ⬇️

I think it’s pretty clear that simulation is the next frontier for AI. The most impressive feats of AI to date are when we have a clear environment + reward, whether it be beating Le Sedol at Go, winning an IMO gold medal, or writing entire apps from scratch. In these cases, the RL algorithm can try different actions, and observe the well-defined consequences in the safety of a docker container. But what about messy real-world situations involving people? The rewards are unclear, the stakes are high, and you can’t experiment in the real world. But these situations are precisely where the next big opportunity in AI is. To crack this, we need to *simulate* society (“put society into a docker container”). Concretely, this means building a model that can predict what will happen in any given situation (real or hypothetical). If we can do this, we are only limited by our imagination: predict the future, optimize for better outcomes, answer hypothetical (“what if”) questions. Ultimately, this goes beyond making better decisions, but it’s about giving us a better understanding of ourselves and the world. Simulation is the whole enchilada. And this is exactly the research that @simile_ai is working on. Read more here: simile.ai/blog/simulatio…

Tired of chasing references across dozens of papers? This monograph distills it all: the principles, intuition, and math behind diffusion models. Thrilled to share!

I'm hiring! job-boards.greenhouse.io/deepmind/jobs/…

@HaileyJoren Congrats Hailey!!

PhD in Computer Science, University of California San Diego 🎓 My research focused on uncertainty and safety in AI systems, including 🤷‍♀️letting models say "I don't know" under uncertainty 🔎understanding and reducing hallucinations 🔁 methods for answering "how much will providing data X improve performance on Y?" at inference time Many thanks to my advisor @berkustun, to my incredible research collaborators, and to my wonderful friends, husband and family. Getting a PhD while becoming a first-time parent is definitely a recipe for growth!

@rose_e_wang @stanfordnlp Congrats Rose!! 🥳🥳

I defended my PhD from Stanford CS @stanfordnlp 🌲 w/ Stanford CS first all-female committee!! My dissertation focused on AI methods, evaluations & interventions to improve Education. So much gratitude for the support & love - and SO excited for the next chapter!!!! 🥳

I have been talking for years in multiple LinkedIn about significant benefits of calibration. It is interesting to see a new paper “Online Calibrated and Conformal Prediction Improves Bayesian Optimization” from Cornell University and Stanford University by @ShachiDeshpande, @CharlieTMarx and @volokuleshov looking at calibration in the Bayesian optimisation domain. Very interesting results: 1) same as in classifier calibration relying on unrealistic assumptions such as normality can result in inaccurate and overconfident probabilistic models which slows down optimisation and may result in incorrect local optima. The paper provides formal analysis on the benefits of calibration in optimisation setting. 2) conformal prediction based method that can be added to any Bayesian optimisation technique to improve optimisation via faster quality convergence. These findings are in line with papers I have shared previously, see Amazon Science paper “Optimizing Hyperparameters with Conformal Quantile Regression” Conformal prediction can significantly improve optimisation methods and is an interesting research domain. #conformalprediction #optimisation

📢New research on mechanistic architecture design and scaling laws. - We perform the largest scaling laws analysis (500+ models, up to 7B) of beyond Transformer architectures to date - For the first time, we show that architecture performance on a set of isolated token manipulation tasks is correlated with metrics of interest at scale, such as compute-optimal loss. Say hello to fast architecture improvement! - Striped architectures consistently outperform homogeneous architectures, as they benefit from specialization of each layer type to particular subtasks An avalanche of other findings in the paper: 📝Paper: arxiv.org/abs/2403.17844 🖥️Repo: github.com/athms/mad-lab

Announcing Score Entropy Discrete Diffusion (SEDD) w/ @chenlin_meng @StefanoErmon. SEDD challenges the autoregressive language paradigm, beating GPT-2 on perplexity and quality! Arxiv: arxiv.org/abs/2310.16834 Code: github.com/louaaron/Score… Blog: aaronlou.com/blog/discrete-… 🧵1/n

Personalized models should let users consent to the use of their personal information! In our latest, we describe how to build models that let users consent to the use of group attributes like sex, age, race, HIV status Spotlight Poster @NeurIPSConf: Tues 10:45-12:45 PM Link: nips.cc/virtual/2023/p… Thread 👇

@willieneis @USC Congrats Willie! 🎉

Excited to share that I will join @USC as an Asst. Professor of Computer Science in Jan 2024—and I’m recruiting students for my new lab! 📣 Come work at the intersection of machine learning, decision making, generative AI, and AI-for-science. More info: willieneis.github.io/lab

📢 Diffusion models (DM) generate samples from noise distribution, but for tasks such as image-to-image translation, one side is no longer noise. We present Denoising Diffusion Bridge Models, a simple and scalable extension to DMs suitable for distribution translation problems.

@stats_stephen Congrats Stephen!

Excited to share that I've joined MIT as an assistant professor in EECS! I'm thrilled to join many thoughtful, inspiring colleagues. Looking ahead, I'm working to develop statistical principles for AI models so that we can use them for science and reliable automated systems.

A founder of FAT/ML, Sorelle Friedler (@kdphd) led key OSTP efforts that placed equity, rights, and innovation TOGETHER at the center of tech policy, and illuminated the potential of automated science. Thank you for your incredible service! haverford.edu/college-commun… #AIBillofRights

@unsorsodicorda Hi Andrea! More than happy to chat -- still figuring out how Rocket.Chat works so also feel free to DM me or shoot me an email to find a time. :)

.@CharlieTMarx I saw your Spotlight presentation about Modular Conformal Prediction. Very interesting! Would you be available to discuss about it on Rocket.Chat in the next days? (Can't discuss in person since I'm attending virtually). Thanks! PS Have fun at ICML!

Thrilled to share that our paper "Comparing Distributions by Measuring Differences that Affect Decision Making" wins #ICLR2022 Outstanding Paper Award🎉blog.iclr.cc/2022/04/20/ann… Congratulations to my awesome students @shengjia_zhao @a7b2_3 @electronickale Aidan @baaadas👏

This caught my attention, as someone who is interested in decision theory, online learning, and statistical learning. arxiv.org/abs/2112.13487

@kdphd @WHOSTP @AlondraNelson46 @geomblog This is so exciting!! Congrats!

@KLdivergence Congratulations!!

So, yeah. Very excited to.... join the flock, take a couple pulls at the front of the V from time to time, and overuse bird puns and analogies to my heart's content.

I've always been drawn to thorny problems that require quantitative sophistication coupled with a compassionate understanding of complex social issues. And, oh my, is tech full of those kinds of problems! So, 🌟 big announcement 🌟.... 🥁🥁🥁 It's true! I'm joining Twitter!

The secret heart of academia is... Wikipedia. In an experiment, this paper found that a single quality Wikipedia article written by chemistry experts influenced the content of 250 published peer-reviewed academic papers! Articles referenced in Wikipedia also become more cited.

It's time for a transition. 1. Moving to @WHOSTP to work with @AlondraNelson46 on bias/fairness. 2. Moving to @BrownUniversity @BrownCSDept @Brown_DSI to work with @senykam on a new center on computing for the people. 14.5 years @UtahSoC - wow! blog.geomblog.org/2021/05/transi…