Sabitlenmiş Tweet
And so important news: we're launching an early beta access to the synth pipelines that originally created SYNTH and have been further refined and enhanced through the last few months.
English
Alexander Doria
46.1K posts
Alexander Doria
@Dorialexander
building open ai infrastructure @pleiasfr
Katılım Nisan 2011
4K Takip Edilen23.2K Takipçiler
@advait_jayant Yes. To some extent they had surprisingly dated prior (also over scaling: highly sparsed MoE were not what they had in mind)
English
@Dorialexander funnily enough ai-2027 got itself backwards on china. it had the ccp nationalizing everything into a single megalab next to a nuclear plant.
the opposite happened. deepseek, qwen, kimi, minimax, and as you pointed out even meituan and xiaomi are all shipping their own models!
English
After months of delay, the successor post to "The model is the product": the AI decoupling. vintagedata.org/blog/posts/the…
English
@Dorialexander Très intéressant. Merci ! (il doit cependant manquer un bout à cette phrase : "By 2023, OpenAI and Anthropic had an early lead in model development but nothing that would prevent...")
Français
@ed_brz9 @jackson_stokes Thought briefly about that but disagree now. Memory is actually needed for many agentic processes (basically model needs to understand what is being asked, and how to look for things)
English
@Dorialexander @jackson_stokes For now I just do synth data and fine tune so I’m not at all qualified to talk about model architecture, but do LLM need to know much to be useful? I feel like more weight dedicated to attention could benefit agentic capabilities. Near zero knowledge models that can use tools
English
There seems to be a ~3B lower limit for useful LLMs. below that, instruction following and ICL drop off a cliff? Is there some fundamental reason for this?
English
and here is anthropic soft power: blessed be circuit transformers (and the data that feed it).
English
@Noahpinion Just creating conditions for proper open-ended research (the kind OpenAI has just started to emulate) and selecting for people willing to do that.
English
I've wondered about this. If demand for intelligence were so infinite, why do we have so.many of our smartest people sitting around doing esoteric fun stuff for smallish salaries in academia?
Chris Anderson@chr1sa
In all of human history, has there ever been a commodity with infinite demand, as there appears to be for intelligence? I can't think of one. Even compute, energy or just silicon/sand are just downstream of intelligence, which is the main demand driver. In economics, rather than modeling the usual price/demand curve to reach an equilibrium, perhaps you'd have to model price/*rate of demand growth* (ie, the derivative of demand, or some other indicator of velocity) Interestingly, ChatGPT (below) prefers the framework of "recursive expansion of demand" as increasing intelligence opens new applications/markets. But the end result is the same -- the demand curve keeps moving to the right, maybe forever. Which I think is unprecedented.
English
@JulienBlanchon @bastiengares (les documents vraiment corpos dans certains domaines c'est totalement galère à localiser en webcralw et les quantités sont pas là : je comprends la logique de racheter des données de boîte)
Français
@JulienBlanchon @bastiengares Moi c'est un peu les échos que j'ai eu côté Anthropic (et de certains fournisseurs RL) : migration d'environnements contraints avec pas mal de connaissance métiers vers des workflow plus ouverts de la donnée en gros, moins structuré.
Français
Pourquoi Anthropic est si mauvais en maths et les sujets de recherche par rapport à openAI et Google
Dr Singularity@Dr_Singularity
this is pretty amazing news Google DeepMind's AI agent autonomously solved 9 of 353 open Erdos problems. Cost - only few hundred dollars per problem. This is a major signal that AI is moving from helpful math assistant to actual autonomous research engine. LLM agents connected to Lean are now solving real open mathematical problems, not just textbook exercises. The strongest system resolved 9 open Erdős problems and proved 44 OEIS conjectures, with some breakthroughs costing only a few hundred dollars each. Mathematics is one of the hardest domains because you cannot bluff your way through it. A proof either checks or it does not. And now AI systems are starting to generate proofs that survive formal verification. 1000x acceleration of progress is coming.
Français
@JulienBlanchon @bastiengares Je pense qu'il achètent surtout la donnée de seeding mais ensuite c'est multi-usage…
Français
@Dorialexander @bastiengares Je dirais plus l'achat d'env RL (en code/droit/corpo effectivement)
Français
Exactly why the frontier should be distributed.
will depue@willdepue
academics are unprepared for the coming world where much scientific progress is majorly a function of inference compute. whether OpenAI points the Eye of Stargate at your particular field will decide its acceleration. talent will leach away into the labs. it's already begun
English
@code_star @ariaurelium In this budget will need to wait for a raise anyway.
English
The upside: frontier labs have until now been suspiciously advancing in directions where they had deep internal expertise at the core. As we move toward open-endedness, this might be the actual bottleneck.
English
@menhguin Meanwhile I’m still waiting for 100B compute investments in France…
English
@Dorialexander still thinking about that the time the br*tish said they'd deploy a billion to make BritGPT and then absolutely nothing happened
x.com/menhguin/statu…
Minh Nhat Nguyen@menhguin
so does anyone remember the British Government wanted to spend a billion pounds on training BritGPT and then absolutely nothing came of it after 1.5 years, or am I going crazy here
English
it's wild that
1. Chinese frontier AI labs have roughly 1/20th the funding and revenue of American frontier labs
2. American frontier labs consider Chinese labs only 6-12 months behind
English
@thkostolansky The kind of fresh take we have seen with OpenAI Erdös could get interesting at an industrial scale.
English
@thkostolansky many current blocks are conceptual rather than experimental and general feel of standstill for the last decades.
English
To be honest things will really get interesting when we’ll start tackling open ended problems in physics.
English