Peter Schulam

@WallaceUcsf Check out work by @david_sontag @ShalitUri in this area. Also @pschulam @suchisaria who use Gaussian processes instead of neural nets.

@MaxALittle @filippie509 Gotta love that sweet sweet irony

In fact, ML can be especially impactful in situations like this. The heuristics make excellent features for a linear model. The result is often good enough (or is a strong baseline). Keeping this in mind gives me a nice “playbook” for kicking off work on a new project.

For some classification problems, a first analysis usually uncovers several heuristics that would work ~50-75% of the time. My gut reaction to this is often: “Do we really need to use machine learning here?” After all, I don’t want to be the fool with a hammer looking for nails.

@pschulam How bout that Twitter algorithm

@IAmSamFin 😂

The majority of ML case studies floating around the internet are, unfortunately, fast food. I think this is a problem because we can’t share, learn from, and discuss our “recipes” as practitioners.

I think this might follow from results reported by Jonathan Byrd and @zacharylipton in this great paper: arxiv.org/abs/1812.03372

When we talk about covariate shift, the support of the train and test distributions may be the same but the frequency of seeing a given input may have changed. This is important when we use low-capacity models, but maybe less so with the richer classes we use today.

There are lots of recent papers in the ML literature that look at how to detect when we can’t make reliable predictions. I often see this described as detecting “out of distribution” samples. This is unusual to me, though. The same value can come from two different distributions.

@IAmSamFin Great write-up on this (unfortunately?) evergreen debate; thanks for sharing! I liked your point about avoiding the “who owns regression” question. The same tool can be used to accomplish different things.

@pschulam Well put, and I think it makes a lot of sense in historical context of the fields. ML is about building computers that do stuff, stats is about understanding things (and sometimes the things are even computers that do stuff) sgfin.github.io/2020/01/31/Com…

Very interesting paper from @laurence_ai: openreview.net/pdf?id=Rd138pW… Not the intended message, I think, but brought one thing into focus for me:

@TimRadtke Thanks for the link! This looks interesting; I’ll check it out.

@pschulam KDD's Applied Data Science track comes to mind: #ads-papers" target="_blank" rel="nofollow noopener">kdd.org/kdd2020/accept…

Stats journals often have a separate “applications” track. Does something like this exist for machine learning? I’m looking for good write ups of the nitty gritty details behind successful ML applications.

This reminds me of what @lawrennd calls “decomposition” in his three D’s of ML system design. inverseprobability.com/talks/notes/th…

The thread below is the kind of thing that we need more of in “applied ML literature”. In this case, they didn’t really need a model, but I would love to read more about this kind of clever detective work.

We've heard lots about #MachineLearning in healthcare, but usually with few specifics. A new paper in @AnnalsofIM, featuring our very own @suchisaria & @pschulam, cuts through the buzz and discusses the real applications and real benefits of clinical #AI: acpjournals.org/doi/10.7326/M1…