Laura Hanu

Excited to release code and models for detoxify 🙊, a simple python library built @UnitaryAI that aims to detect hate speech and toxic comments online! Includes training scripts built with @PyTorchLightnin ⚡ and transformers checkpoints @huggingface 🤗! github.com/unitaryai/deto…

pretty good high level intro to the current state of llms youtube.com/watch?v=xCRvOU…

main takeaways from the @dwarkeshpodcast @karpathy interview: *RL limitations*: hard to get past reward sparsity problem in RL when it comes to real world tasks, one promising direction could be more sample efficient learning by reflecting on mistakes *reduced model size by getting rid of redundant memorisation*: soon the cognitive engine might look like a 1 billion parameter model that just knows how to think without the need to memorise lots of data - it should be able to recognise what it doesn't know and look it up *gradual loss of control and understanding*: even if we still have humans delegating tasks to autonomous entities, it will get increasingly harder to fully control them, let alone understand what they're doing; similar thoughts in this great albeit on the pessimistic side paper arxiv.org/abs/2501.16946 *role of AI education post-AGI*: pre-AGI AI education is useful, post-AGI education is fun, could be seen as a way to train mentally just how you do physically

pantheon is now up there with some of my favourite sci fi of all time like asimov’s the end of eternity, the last question or story of your life with some elements of snow crash or dark the tv show but upping the scale even more, dare i say something akin to what @DavidDeutschOxf was describing in the beginning of infinity

a nice straight forward summary on some of the grpo limitations beyond just not being great for multi-turn e.g. * if you have multiple reward signals -> the model won't know which one it is being rewarded for since they're usually all collapsed into one * only the scalar reward is used for policy update when a more detailed textual feedback could be used (what gepa kinda does with their reflective prompt evolution)

nice watch for a reading group on verifiers, like a jigsaw falling into place🤭

It's been interesting to see RL having a comeback lately. Guess in retrospect it's not surprising that RLHF is not enough since it naturally hits a ceiling limited by the quality of human feedback. What's particularly exciting though is that this shift is happening just as AI is becoming more agentic and able to interact with the digital world. Combine this with some grounded reward signals (e.g. profit, likes, citations) and we should see AI not just replicate what humans do but create new knowledge through their own experiences. The next few years should be a fun and wild ride 🎢 For another interesting paper on the topic: incompleteideas.net/papers/TheEraO… or this podcast on it: youtu.be/zzXyPGEtseI?si…

Why does RL work so well to learn preferred completions in llm post training and why can't we just use supervised finetuning? This paper has an interesting information-theoretic explanation: arxiv.org/pdf/2503.01067 TLDR: The literature shows that the 2 stage RL approach (1. learn a good reward model that can score llm completions similar to how a human would 2. use that to learn good policies with RL) used today in the sota models outperforms using only supervised finetuning. The authors posit that this happens when the verification of an output is simpler than generating it. This is because, in the RL case, the search space is constrained to the subset of policies that are optimal for the learnt reward model. When they reduce the verification-generation gap empirically the difference in results diminishes too.

the fact that this could be the prequel to Adolescence 😳

note to self, don’t start addictive puzzles late at night, but also level 4 lets go?🕵️‍♀️ fairly consistent strategy so far 👀

great discussion! we need more public discourse between top ai leaders and economists 🤝 would’ve liked to see them engage with hinton’s concern that the benefits ai brings will deepen the economic divide in a capitalist system further creating fertile ground for fascism 🌶️

at @NeurIPSConf this week 👋

Introducing Cambrian-1, a fully open project from our group at NYU. The world doesn't need another MLLM to rival GPT-4V. Cambrian is unique as a vision-centric exploration & here's why I think it's time to shift focus from scaling LLMs to enhancing visual representations.🧵[1/n]

If you haven't seen it, part 1 dives into how to set up a HPC cluster! unitary.ai/articles/intro…

The 2nd part of the Intro to multi-node machine learning is out! 🥳 This all about how to use Slurm to scale up your ML applications! 🚀 unitary.ai/articles/intro…

Excited to kick off a new deep-dive blog series on how to build and set up the infrastructure for distributed training from scratch. Spoiler alert – setting up a cloud-based cluster that can scale to hundreds of nodes isn't as daunting as it sounds! 👀🚀 unitary.ai/articles/intro…

Hello from the ICCV magazine 👋 🙋‍♀️ #ICCV2023 @ICCVConference Paper: arxiv.org/abs/2309.10783

If you are at ICCV in Paris next week, come to talk to us at our poster at the "What is Next in Multimodal Foundation Models?" workshop! The title is: "Language as the Medium: Multimodal Video Classification through text only" Paper link: arxiv.org/pdf/2309.10783…

Can we use LLMs like GPT3.5/Claude/Llama2 to directly classify multimodal content like videos in-context with no training? 👀 Excited to share our findings at the "What is Next in Multimodal Foundation Models?" workshop at #ICCV2023 @anitaveroe @jdthewlis arxiv.org/abs/2309.10783

So fun to see so many people interested in LLMs, MLOps and generative AI in London!🐝

Our most popular detoxify model on @huggingface has been downloaded more than 7M times 🥳 huggingface.co/unitary/toxic-…