
Follow along with researcher @xariusrke in these detailed field notes as he hunts down MoE training bottlenecks nousresearch.com/moe-scaling-fi…
xr-5 🐀
1.8K posts

@xariusrke
distributed training infra and pretraining @nousresearch | ex-nanotron at @huggingface | dms open

Follow along with researcher @xariusrke in these detailed field notes as he hunts down MoE training bottlenecks nousresearch.com/moe-scaling-fi…




@justinskycak "Hell is the day you meet the man you could've been."





Today, at the @DARPA expMath kickoff, we launched 𝗢𝗽𝗲𝗻𝗚𝗮𝘂𝘀𝘀, an open source and state of the art autoformalization agent harness for developers and practitioners to accelerate progress at the frontier. It is stronger, faster, and more cost-efficient than off-the-shelf alternatives. On FormalQualBench, running with a 4-hour timeout, it beats @HarmonicMath's Aristotle agent with no time limit. Users of OpenGauss can interact with it as much or as little as they want, can easily manage many subagents working in parallel, and can extend / modify / introspect OpenGauss because it is permissively open-source. OpenGauss was developed in close collaboration with maintainers of leading open-source AI tooling for Lean. Read the report and try it out:


it's been a longstanding dream of mine build an ai system that can tell a compelling story. it's what got me started in the space in the beginning, and with Hermes Agent I finally pulled it off 100% written, typeset, etc. by Hermes Agent those at our gtc event got hard copies🤗


SpaceX Starship rocket garden


I’m joining @SpaceX and @xai with @JasonBud. X is the company realizing science fiction - reusable rockets, humanoid robots, data centers in space, and more. Almost 10 years ago, I joined SpaceX as an intern on Dragon 2 crew displays. This was in the era of the first rocket landings on barges, long before the Dragon 2 restored human spaceflight to America or Starlink delivered internet from space. Every day since then, I’ve thought about the next steps to land on the Moon - and to build a city on Mars, data centers in space, the brains behind robots, and beyond. There is no better place to build teams and products from the ground up with planetary scale resources. If you’re looking to work on the hardest problems that lay a foundation for humanity’s future to the Moon, Mars, and beyond - DM me.

