Depth First Learning

I LOVE this talk by @ShriramKMurthi on how to design a curriculum to teach programming. youtube.com/watch?v=5c0BvO… It's tempting to jump to solution X. He pulls back to focus on *constraints*, just as you'd do in an eng design problem. 🧵

"I would love somebody to take away from this paper that datasets are situated. It's not just the perspectives of the creators but also the socio-technical processes like search engines and the time+place particulars that filter through in the act of creation."

"There's a history of making these data sets. Well, what are the things that people bring to the table when they do that? If we can understand that, then we can see where the deficiencies are that could lead to things going forward that are just better approaches."

Today, we have @cephaloponderer and @cinjoncin talking about Emily's paper from July 2020 --> Bringing The People Back In: Contesting Benchmark Machine Learning Datasets. Check it out here: depthfirstlearning.com/2021/Emily-Den…

"... we invested a lot in monitoring and this was a pretty large scale effort. We're monitoring on the order of 10000 pieces of information a second. This was critical to our success because there's so many moving pieces and so many places where the optimization can go wrong."

"One difficulty in this work is just getting a better sense of what's going. People often monitor certain attributes in their models. But now there's not one model being trained. There's huge numbers of models being trained in very different settings. So throughout this work ..."

This week, we have @Luke_Metz and @cinjoncin talking about Luke's 3 (now 4) year journey on learning optimizers. Find the audio and transcript at depthfirstlearning.com/2021/Luke-Metz…. Some choice quotes in the thread 👇

"... just want the model to be compact. We may want it to be interpretable or robust or fair. The message of this paper is that by optimizing for one property, we may be compromising others."

"I grew up in Africa and am interested in making AI accessible in resource constrained environments similar to what I grew up in, but it's also important to understand that we are often making tradeoffs when we talk about properties that we want a model to have. We might not ..."

Debuting another new audio release! Listen to @sarahookr and @cinjoncin talk about Sara's paper - Characterising Bias in Compressed Models. Find the audio and transcript at depthfirstlearning.com/2021/Sara-Hook…. Some choice quotes in the thread 👇