Nicola Piovesan

Pre-2022 software and content is the low-background steel of the information era.

ARC-AGI-3 scores for GPT-5.4, Gemini 3.1 Pro and Opus 4.6 Gemini 3.1 Pro: 0.37% GPT-5.4: 0.26% Opus 4.6: 0.25% Grok 4.2: 0%

Three days ago I left autoresearch tuning nanochat for ~2 days on depth=12 model. It found ~20 changes that improved the validation loss. I tested these changes yesterday and all of them were additive and transferred to larger (depth=24) models. Stacking up all of these changes, today I measured that the leaderboard's "Time to GPT-2" drops from 2.02 hours to 1.80 hours (~11% improvement), this will be the new leaderboard entry. So yes, these are real improvements and they make an actual difference. I am mildly surprised that my very first naive attempt already worked this well on top of what I thought was already a fairly manually well-tuned project. This is a first for me because I am very used to doing the iterative optimization of neural network training manually. You come up with ideas, you implement them, you check if they work (better validation loss), you come up with new ideas based on that, you read some papers for inspiration, etc etc. This is the bread and butter of what I do daily for 2 decades. Seeing the agent do this entire workflow end-to-end and all by itself as it worked through approx. 700 changes autonomously is wild. It really looked at the sequence of results of experiments and used that to plan the next ones. It's not novel, ground-breaking "research" (yet), but all the adjustments are "real", I didn't find them manually previously, and they stack up and actually improved nanochat. Among the bigger things e.g.: - It noticed an oversight that my parameterless QKnorm didn't have a scaler multiplier attached, so my attention was too diffuse. The agent found multipliers to sharpen it, pointing to future work. - It found that the Value Embeddings really like regularization and I wasn't applying any (oops). - It found that my banded attention was too conservative (i forgot to tune it). - It found that AdamW betas were all messed up. - It tuned the weight decay schedule. - It tuned the network initialization. This is on top of all the tuning I've already done over a good amount of time. The exact commit is here, from this "round 1" of autoresearch. I am going to kick off "round 2", and in parallel I am looking at how multiple agents can collaborate to unlock parallelism. github.com/karpathy/nanoc… All LLM frontier labs will do this. It's the final boss battle. It's a lot more complex at scale of course - you don't just have a single train. py file to tune. But doing it is "just engineering" and it's going to work. You spin up a swarm of agents, you have them collaborate to tune smaller models, you promote the most promising ideas to increasingly larger scales, and humans (optionally) contribute on the edges. And more generally, *any* metric you care about that is reasonably efficient to evaluate (or that has more efficient proxy metrics such as training a smaller network) can be autoresearched by an agent swarm. It's worth thinking about whether your problem falls into this bucket too.

Italian is an intrinsically understandable language, by people and machines.

@willccbb That’s the graph equivalent of “how many r’s are in strawberry” 😆

which is larger, 52.8 or 69.1?

Reasoning Language Models for Root Cause Analysis in 5G Wireless Networks. arxiv.org/abs/2507.21974

GPT-OSS ranking 34th on SimpleBench

We released TeleMath! 📡🧮 This project began with a simple but important question we kept asking ourselves: how can we measure AI’s ability to solve real-world telecom mathematical problems, beyond benchmarks focused on knowledge?

I'm excited to announce that I'll be speaking at the @AIforGood Global Summit 8-11 July at Palexpo in Geneva, hosted by the @UN @ITU. This event is a key platform for sharing innovative #AI applications and fostering #collaborations. aiforgood.itu.int/summit25/ #AIforGood

Great to present at #IEEE #GLOBECOM 2024 in Cape Town, sharing our latest innovations in green networks and large language models for telecom. Exciting discussions on the future of sustainable and intelligent connectivity!

Yesterday at ongoing @IEEEorg GLOBECOM 2024 in Cape Town, South Africa, our Product Lead, @ucheedwin20, joined IEEE Humanitarian panel session with @ashutoshdutta , @itspiove, and @AfroDiva . The session focused on sustainable telecom infrastructure for Africa’s digital growth. Uche emphasized the need for operators to rethink connectivity, addressing challenges like high internet costs, energy demands, and regulatory hurdles. He proposed solutions such as renewable energy, edge computing, community-driven satellite coverage, and government partnerships for regulatory sandboxes. He also highlighted @dsn_ai_network's work as an example of localized solutions, calling for a shift in infrastructure delivery to bridge the digital divide. Thanks to the @IEEEorg Humanitarian Technologies Board for organizing this panel. @IEEEComSoc @vukosi #datasciencenigeria

A highlight of the talk was Hermes, our framework using chains of #LLM agents to enable self-optimizing network. Check the paper here: arxiv.org/abs/2411.06490

This week, I attended the #6G Summit in Abu Dhabi—an inspiring event on the future of #wireless technology. I presented a talk, “Data-Driven Network Modelling & Optimization for the 6G Era: Human Expertise Meets AI”, sharing our vision of autonomous, intent-driven networks.

Honored to have participated at @AIforGood in Geneva! 🌍✨ I tackled the question: can AI, ML, and big data help us build greener, more sustainable networks? Let's continue innovating for a better future! #AIforGood #Sustainability #TechForGood

I'm happy to announce that I will be speaking at the @AIforGood @ITU @UN #ITUaiSummit, the most anticipated #AI event happening on 30-31 May 2024. I look forward to seeing you there! aiforgood.itu.int/summit24/

Today, I'll be presenting our latest work, "FlexTrain: A Dynamic Training Framework for Heterogeneous Devices Environments," at the #NeurIPS Workshop on Advancing Neural Network Training, in New Orleans. arxiv.org/abs/2310.20457

📢 TeleQnA, the first benchmark dataset designed to evaluate the knowledge of LLMs in telecommunications 📶 The questions in the dataset cover topics like - Lexicon - Standards and specification - Research 🧑‍💻 github.com/netop-team/Tel… #LLM #GenAI #Telecom

Join us to explore how ML/AI models can help understand and reduce 5G’s energy footprint.

As large language models (LLMs) find their way into the telecommunications industry, evaluating their knowledge becomes a pressing issue. We introduce TeleQnA, the first benchmark dataset designed to evaluate the telecom knowledge of LLMs. arxiv.org/abs/2310.15051

The #5G boom has led to spiralling #energy consumption for #mobile base stations. With demands for #connectivity growing, @Huawei thinks there is a solution for sustainable 5G - using #AI 🔋 @itspiove | @AIforGood Read: techinformed.co/47Q79KD