ทวีตที่ปักหมุด
Alexander Doria
45.1K posts
Alexander Doria
@Dorialexander
building open ai infrastructure @pleiasfr
เข้าร่วม Nisan 2011
4K กำลังติดตาม22.4K ผู้ติดตาม
Can’t deny this is an unusually French week in my life.
Cody Blakeney@code_star
I can do this all day @Dorialexander Your move
English
And so, (continuous) synthetic pretraining getting hot.
Cursor@cursor_ai
We were able to significantly improve the model quality and cost to serve. These quality improvements come from our first continued pretraining run, providing a far stronger base to scale our reinforcement learning.
English
@MinistereCC Full dataset on @huggingface : huggingface.co/datasets/PleIA…
And early exploration on dedicated website: french-science-commons.pleias.dev
English
The project was led in cooperation the EU infrastructure OPERAS and the Chaire du Québec sur la découvrabilité des contenus scientifiques, under the support from @MinistereCC.
Français
And new data release: French-Science-Commons, the largest scientific corpus in French in open access including 1.25 million documents/42 million pages re-digitized with VLM (dots ocr).
English
Missed opportunity to go all in on floppy disk challenge.
Yuchen Jin@Yuchenj_UW
OpenAI just dropped a training challenge: Train a <16MB language model in 10 minutes on 8×H100s and minimize held-out loss on a fixed FineWeb dataset. Basically NanoGPT Speedrun. They’re sponsoring $1M in compute. I can summon my autoresearch army to win it… if I have time.
English
@siliconcarnesf Mouais mon expérience du terrain : c’est surtout copilot.
Français
Bravo aux medias et aux "intellectuels" français ! Allez encore un petit effort et bientôt on reviendra aux arc et aux flèches...
Catherine Cervoni@CathCervoni
Pourquoi l'usage de l'IA recule en entreprise pour la première fois depuis son déploiement Plus de la moitié des salariés n'ont pas utilisé un seul outil d'IA au cours de l'année 2025 — et seuls 7 % y ont recours quotidiennement, contre 14 % dans le monde itsocial.fr/?p=108065
Français
@omooretweets @a16z @Kantrowitz I’m not sure you’re really addressing the issue of SaaS high-margin evaporating in the context of transient/last mile use cases. Seems hardly scalable and under constant pressure from the next synth environment.
English
How can AI application startups compete with the big labs and incumbents?
I shared some of our thoughts on this @a16z with @Kantrowitz 👇
English
Meanwhile multiple people in the field do seem to be prompting/tuning decoders all day and, yeah, it must feel narrow.
English
While attending an NLP conference today, I realized you’re actually much less *into* LLMs while building LLMs. It’s regex in tokenizer, encoders to filter data at scale, knowledge graphs for synth pipelines, rule-based systems to backtranslate… There’s a whole village.
English
@leavittron I have really no idea (maybe except for actual ablations?), and I'm scouting every single specialized benchmark I can find to monitor next SYNTH iteration. Everything, everywhere, all at once.
English
@Dorialexander Strong agree. I'm curious whether there's ever a scenario in which you DON'T want to put end task-relevant data earlier into training (assuming you have unique tokens consituting at least ~0.5% of your pretraining budget)
English
"To get the most of domain data incorporate it as early in training as possible" — and I think the same hold true for structured/agentic data.
Rosinality@rosinality
Incorporating SFT data during pretraining is more effective for finetuning than the plain pretraining and finetuning scheme, even considering replay during finetuning. But the ratio of SFT data during pretraining should consider the token budget for pretraining. They built a scaling law for this.
English
When I told you theses things were not really existing.
Olivia Solon@oliviasolon
During last year's Paris AI Action Summit, Fluidstack announced plans for a €10 billion AI data center championed by president @EmmanuelMacron. A year on, and Fluidstack has pulled out of the project. Scoop from @BenoitBerthelot bloomberg.com/news/articles/…
English
@Dorialexander hmm, would there be a point (and a market) to train specialized enterprise SLMs from scratch then?
English
@indiana_duck Bouh non… Si près du centenaire pour Mickey.
Français
Ca pue cette histoire, j'espère pouvoir vous en parler bientot.
Adrien Schwyter@Adschwyter
🔴Info @Challenges Disney va mettre fin à ses contrats de licence pour Le Journal de Mickey et Picsou Magazine avec UHM. Potentiellement une trentaine de licenciements. « On ne s’y attendait pas du tout. On ne comprend pas. Les chiffres sont bons. » challenges.fr/entreprise/med…
Français
@llm_wizard @fujikanaeda You have to rename your account now.
English
@bijucyborg @pleiasfr @nvidia Nothing planned yet but Germany is not unlikely since the other half of Pleias is there.
English
@Dorialexander @pleiasfr @nvidia Any plans for German and Austrian?
For 🇩🇪 using Sinus Milieu might be a good method.
sinus-institut.de/en/sinus-milie…
English
@IanBaer Oh yes totally. I like to complain all the time, can’t deny.
English
@Dorialexander You are angry with Claude because you are angry with yourself. Free yourself from this, brother.
English
I feel it's very healthy to be regularly annoyed/angry at Claude. Otherwise you end up like Gary Tan.
English