Mike

@jeremyphoward Well now, someone's co-founder is sleeping in the spare room tonight

Yay Flash 2.5 knows of FastHTML! :) It's not yet very good at it without providing llms.txt context, but it's a start. (Also, who is Mike Innes?...)

@HeblingVieira Yeah, next I need to try making ChatGPT the player!

@MikeJInnes It should be an interesting experience for DM'ing too I guess.

Did not expect to be role-playing with a GPU this evening.

@CharlesHSimpson Thanks! I blogged about it here: mikeinnes.github.io/2020/12/17/cov…

@oxinabox_frames @apaszke @shoyer @dancherp On this view you get dual numbers by implementing an instruction tape tracer + transform, and then optimising it by fusing those steps together, without changing the semantics or PC issues.

@oxinabox_frames @apaszke @shoyer @dancherp I tend to think of the tracing and AD transform steps as orthogonal. AD transform looks similar everywhere (adjusted for Fortran AST, Jaxprs or SSA). Tracing systems are the source of PC errors, not the transform step.

Can someone (@dancherp ?) talk to me (or point me to a writeup) about the historical decision to move the Swift AD work from having distinct co-tangent and tangent types, to just having common tangent type used for both forward and reverse? I want to understand pros and cons.

Excited to share our @UofT & @VectorInst work "Emergent Road Rules in Multi-Agent Driving Environments" w/ @PhilionJonah, Andrew Liao & @FidlerSanja. Paper: arxiv.org/abs/2011.10753 Project Page: fidler-lab.github.io/social-driving/ Code: github.com/fidler-lab/soc… [1/5]

"The police and the national guard, who ringed every campus, keeping the students from creeping across to society like so many black rats swarming out of a leaky ship.”

“He knew that at one time she had been illegally married to a student commune leader, and that for one year she had lived in the rabbit warrens of Columbia University, along with all the smelly, bearded students kept subsurface lifelong by the pols and the nats. ...

“I can’t live two hours without my ID, he said to himself. I don’t even dare walk out of the lobby of this rundown hotel and onto the public sidewalk. They’ll assume I’m a student or teacher escaped from one of the campuses.” – Philip K. Dick

@vflorelo Yup, this is the source!

@wetlabsucks @MikeJInnes I really don't think so

I'm So Meta, Even This Acronym. Credit @MikeJInnes

The Imperial College epidemic simulation code that I helped a little on is now public: github.com/mrc-ide/covid-… I am a strong proponent of public code for models that may influence policy, and while this is a "release" rather than a "live" depot, it is a Good Thing.

@skornblith @StefanKarpinski The SK / Singapore approach depends on pretty extreme measures (location tracking, loose rules on gov access to medical records etc) that will be a lot harder to implement in the US (politically and logistically) than lockdown

@StefanKarpinski After 2 months of lockdown, hopefully the disease will not be as widespread as it is now. At that point, we could test all close contacts of anyone who tests positive, which might be enough. This seems to have worked in South Korea; not yet clear if it will work in the U.S.

This seems like by far the best way to start to get the economy back on track: get antibody tests to as many people as possible and allow people who have immunity to start get out and start doing things, helping out

@StefanKarpinski A lot depends on how long immunity lasts for; for some coronaviruses it's around three months, which would make things significantly harder. Hopefully more data coming out on that soon.

Imperial College paper that has informed UK pandemic policy. Well-explained exploration of some of the tradeoffs involved imperial.ac.uk/media/imperial…

@ChadScherrer @theshawwn @rbhar90 That's not the only option; e.g. you can just make up a derivative that's good enough for gradient descent. And there are probably other mathematically reasonable schemes that you could choose.

@ChadScherrer @theshawwn @rbhar90 All programs have a bunch of non-differentiable operations like mod. An AD will just assume the derivative is always 0 for that function.

@Inoryy @ianadwilliamson @shoyer Julia's Python interop is actually really solid, and in some ways more convenient (no PyObject casting, more auto conversions). More experimental (but basically working) is differentiation through PyCall: #slide=id.g5ba58bae6c_0_115" target="_blank" rel="nofollow noopener">docs.google.com/presentation/d…

@ianadwilliamson @shoyer To be clear, I know that Python interop exists to some extent in Julia s well via PyCall, but from what I've gathered it's more of an experimental feature developed by a third-party. Would be happy to learn more if I'm mistaken.

Have you ever wondered what will be the ML frameworks of the '20s? In this essay, I examine the directions AI research might take and the requirements they impose, concluding with an overview of what I believe to be two strong candidates: JAX and S4TF. inoryy.com/post/next-gen-…

@ChadScherrer @oxinabox_frames A big challenge is getting efficient source transform AD and higher-order derivatives; if done naively, nesting gradients produces exponential amounts of code. You'll notice this if you try to do third-order in Zygote

@oxinabox_frames What are the current big challenges in AD?

Seems like a lot of ML/Deep Learning folk think that automatic differentiation is a solved problem. This is very similar to NLP or Computer Vision people thinking that ML is a solved problem.