Daniel D'souza 

“Best Paper Award” @ ACL 2024 🪄What an incredible culmination of perseverance to connect and represent languages around the 🗺️! 🪄 🤗 Huge thanks to the @aclmeeting committee for recognizing the massive effort behind Project Aya @CohereForAI 💙 #ACL2024

We’re excited to announce our next #AI Seminar! Marzieh Fadaee @mziizm (Head of @Cohere_Labs) will join us virtually for a talk on "Data Strategies for Language Models: Insights from Multilingual Research:" 📅 Thursday, April 9 | 1:30 pm ET 🔗cse.engin.umich.edu/event/data-str…

So exciting to see our work 𝑭𝒖𝒔𝒊𝒐𝑵 : Making, not Taking the Best-of-N in action @OpenRouter 🔥 I will be presenting this work from @Cohere_Labs at @iclr_conf 🇧🇷 in 2 weeks, come and find us if you are there !

🚨There is always exciting work that comes from The Expedition at @Cohere_Labs that goes on to become highly-technical blogspots or even full-blown research papers! 🪄 Sign up for this years edition where we work to build BIG ideas using 🤏 "Tiny" Aya 🌎🌍🌏

Tiny Aya reimplementation From Scratch! Have been reading through the technical reports of the recent wave of open-weight LLM releases (more on that soon). Tiny Aya (2 days ago) was a bit under the radar. Looks like a nice, small 3.35B model with strongest multilingual support of that size class. Great for on-device translation tasks. Just did a from-scratch implementation here: github.com/rasbt/LLMs-fro… Architecture-wise, Tiny Aya is a classic decoder-style transformer with a few noteworthy modifications (besides the obvious ones like SwiGLU and Grouped Query Attention): 1. Parallel transformer blocks. A parallel transformer block computes attention and MLP from the same normalized input, then adds both to the residual in one step. I assume this is to reduce serial dependencies inside a layer to improve computational throughput. 2. Sliding window attention. Specifically, it uses a 3:1 local:global ratio similar to Arcee Trinity and Olmo 3. The window size is also 4096. Also, similar to Arcee, the sliding window layers use RoPE whereas the full attention layers use NoPE. 3. LayerNorm. Most architectures moved to RMSNorm as it's computationally a bit cheaper and performs well. Tiny Aya is keeping it more classic with a modified version of LayerNorm (the implementation here is like standard LayerNorm but without shift, i.e., bias, parameter).

@sarahookr I’m loving this journey of discovery. It’s been a long time coming :)

The most consumed biscuit is the entire world. G stands for genius.

🤏

@sarahookr @Cohere_Labs 💗

Huge congrats to the @Cohere_Labs team and community! Super special to see tiny Aya in the open. Congrats to everyone involved, and the large amount of care required for this type of launch.

Extremely proud to present ✨Tiny Aya ✨ Tiny 🤏 but mighty 💪3.35B parameter models, massively multilingual from the ground up 🌎🌍🌏, built with immense care w.r.t language representation🤗 We had a blast building this! 💗 Have at it! 🎆

@sarahookr @adaptionlabs Congratulations @sarahookr ! 🚀

Beginnings are very special. Today is an important day for @adaptionlabs. Today a handful of one-size-fits-all-models are optimized for the average use case. Averages erase the exceptional. Everything intelligent adapts. So should AI.

Proud to share this work led by @oliverjbolton 🎉 We introduce SimMerge, a practical way to make model merging more reliable at scale. Paper and results in the thread 👇

🧩In modern LLM development, we often end up with many specialized checkpoints. Merging into one model is attractive, but the results depend a lot on the selected merge method & order.  Our paper introduces SimMerge: a simple way to choose the merge configuration automatically.

@sarahookr “without an agenda” is such a key part of this

I always ask a trusted group for feedback at the beginning of an endeavor. critical to find 1-2 calibrated skeptics without an agenda. Take their view over time seriously since they can be swayed w evidence. If their view remains the same, you are not making progress.

“You can just do things” energy in 2026 ✨

Sr Research Scientist, Julia Kreutzer: Treasure Hunt paper. 🗺️ This work introduces a method to improve model performance by adding markers to tokens of the pretraining data, enabling real-time targeting of the long tail using training-time markers. youtu.be/K3BUpKag_nA

@championswimmer Hafiz is the bomb ❤️ try their pomegranate turkish delight specifically 😍

Went to the famous Hafiz Mustafa 1864 baklava place in Istanbul and had the Humbara. By far one of the best desserts I've had in recent memory. The clotted cream is out of the world. 10/10 highly recommend. They have branches in London and Dubai too.

Careers are long. My first director gave me something that has guided my entire career: treat people with respect because you will meet them again and again. Interactions are rarely one off, and what matters in the long run is that you are fair and have integrity.

Doing More with Less: "Treasure Hunt" & The Future of Multilingual AI ft @mrdanieldsouza

Such a fun chat with @readsail at #NeurIPS2025 ! 🤗 Got to chat about our recent work "TreasureHunt" (arxiv.org/abs/2506.14702…) and play a game of Guess Who? 👀 Kudos to @natolambert for setting this in motion! 🔥

@rao2z @anilananth Airlines 💙

Chatting #AI with @anilananth over ravva dosa/poori/idlee/kapi at Tivoli garden in B'lore.. 😎

📢 Join the final Conversational AI Reading Group meeting of 2025! 📅 Thursday, Dec 18th | 11 AM - 12 EST 🎙 Speaker: Daniel D'souza (@mrdanieldsouza) - @cohere Labs 📖 Topic: "Data as Leverage: Improving Foundation Models Beyond Scaling." 🔗 Details: poonehmousavi.github.io/rg.html