Lucio Dery Jnr Mwinm

New paper alert: "Latent Space Communication via K-V Cache Alignment" arxiv.org/abs/2601.06123 We propose a method for Large Language Models to communicate directly via their internal states, bypassing the need for discrete text generation 🧵

The DiLoCo team at Google DeepMind and Google Research is proud to release Decoupled DiLoCo, the next frontier for resilient AI pre-training. Decoupled DiLoCo enables training with datacenters across the world, using heterogeneous hardware, and never halting the system despite hardware failures.

The distillation debate is looking pretty one-sided around here! I've been thinking about this for quite a while (along with Yash Savani), so let me add some variety. Regardless of where you stand on the IP issues: Distillation attacks make the AI ecosystem LESS original, less safe, and -- believe it or not -- also less open. We expand on these points in a blog post linked in the replies. Our favorite realization is that distillation attacks lead to AI monoculture, and this monoculture could create unanticipated, systemic security risks for both individuals and companies.

In the past 6 months we’ve seen a divergence between the game-changing experience of coding w new models and tiny SWE-bench Verified gains. llm-stats.com/benchmarks/swe… New analysis finds most remaining unsolved problems have unfair tests, and many models are heavily contaminated.

In summary, K-V Cache Alignment offers a robust protocol for dense, latent-space communication in multi-agent systems. Read the full paper here: arxiv.org/abs/2601.06123

We hope this work opens new avenues for modular AI systems. By decoupling the communication method from text generation, we enable the construction of pools of specialized models that can collaborate efficiently, without the latency of de-tokenization and with higher bandwidth.

New paper alert: "Latent Space Communication via K-V Cache Alignment" arxiv.org/abs/2601.06123 We propose a method for Large Language Models to communicate directly via their internal states, bypassing the need for discrete text generation 🧵

@pazunre @ven1925143 @ghnewssummary @KhayaAI Similar to @pazunre — was born and raised in Ghana (and schooled there till university ! ) . Though I might work in the US, I’m still Ghanaian through and through

@ven1925143 @ghnewssummary @derylucio @KhayaAI Errm @KhayaAI is a Ghanaian limited liability company. As in it is BASED IN GHANA. CAPE COAST to be specific. I was born in Ghana with a single passport - a Ghanaian one. You are mad that I earned a National Interest Waiver in the US through hard work? Be clear

If they still don't believe you, tell them Dery Lucio(@derylucio ), a Ghanaian, works at Google DeepMind as full time employee. Remind them, @pazunre (MIT PhD), another Ghanaian, is building @KhayaAI , AI for African languages. Indeed, great things can come from small places!

I reverse engineered a phase change in GPT's training data... with the seahorse emoji 🌊🐴 My forensic investigation reveals why non-thinking models have started "thinking out loud" & what it reveals about how frontier labs train their latest models pratyushmaini.substack.com/p/reverse-engi…🧵

I'm starting a new project. Working on what I consider to be the most important problem: building thinking machines that adapt and continuously learn. We have incredibly talent dense founding team + are hiring for engineering, ops, design. Join us: adaptionlabs.ai

🥹🥹 All vague-posting aside, super happy this model is finally out there & proud of everyone for making this happen 💖 let us know what you think!

1/ With @BenDLaufer and Jon Kleinberg, we constructed the largest dataset of its kind to date: 1.86M Hugging Face models. In a new paper, we mapped how the open-source AI ecosystem evolves by tracing fine-tunes, merges, and more. Here's what we found 🧵

Come to our workshop !!! It’ll be fun — I promise !!

30+ accepted papers 6 oral papers 6 guest speakers join us at @iclr_conf on the 27th Hall 4 #3 for a full day of workshop on Modularity for Collaborative, Decentralized, and Continual Learning sites.google.com/corp/view/mcdc… @derylucio, Fengyuan Liu, and myself will be organizing that day in person