Remoty
1.1K posts

Remoty
@Remoty_AI
Boost your productivity team with us! AI-powered insights, privacy, daily summaries, no micromanaging. 🚀 Try free: https://t.co/LbV5cYyy5Y #RemoteWork #Productivity









Absolutely spot on from @simonw I am managing and flexing and guiding UI design, systems architecture, basic code hygeine and smells, product feature development, database integrations, external API quota usage etc etc It flexes *everything*. I would be so f'ing hopeless with this as a junior engineer. I feel I can maybe get a lot out of it now that I have 20 years experience in engineering and product across all kinds of company stages.





@frontierism Im gonna fix it for ya







Apparently today is the 4th year anniversary of GPT-3! arxiv.org/abs/2005.14165 Which I am accidentally celebrating by re-training the smallest model in the miniseries right now :). HellaSwag 33.7 (Appendix H) almost reached this a few steps ago (though this is only 45% of the training done). I remember when the GPT-3 paper came out quite clearly because I had to interrupt work and go out for a walk. The realization hit me that an important property of the field flipped. In ~2011, progress in AI felt constrained primarily by algorithms. We needed better ideas, better modeling, better approaches to make further progress. If you offered me a 10X bigger computer, I'm not sure what I would have even used it for. GPT-3 paper showed that there was this thing that would just become better on a large variety of practical tasks, if you only trained a bigger one. Better algorithms become a bonus, not a necessity for progress in AGI. Possibly not forever and going forward, but at least locally and for the time being, in a very practical sense. Today, if you gave me a 10X bigger computer I would know exactly what to do with it, and then I'd ask for more. It's this property of AI that also gets to the heart of why NVIDIA is a 2.8T company today. I'm not sure how others experienced it, but the realization convincingly clicked for me with GPT-3, 4 years ago.













