Max

476 posts

Max banner
Max

Max

@Maxmatical

llm training

Entrou em Ekim 2013
631 Seguindo56 Seguidores
Max
Max@Maxmatical·
👀👀👀
Max tweet media
QME
0
0
0
4
Nathan Lambert
Nathan Lambert@natolambert·
@thdxr olmo is far more open my guy (better licenses, 100% data released, etc), but we also love nemotron! Nemotrons progress to be more open is a huge win for the ecosystem.
English
7
0
300
8.7K
Max
Max@Maxmatical·
rip to the ai labs that are just qwen finetunes
English
0
0
1
35
elie
elie@eliebakouch·
From a 2021 paper by @LiamFedus, @barret_zoph, and Noam Shazeer. Funny how 3) is still true today in the open ecosystem (only Megatron seems to have good mfu for DSv3/K2 scale and sparsity) We also had to wait ~3 years between the publication of this paper and the fix for 1) and 2) (DeepSeek MoE, DBRX, Mixtral, ...).
elie tweet media
English
5
5
103
25.5K
Max
Max@Maxmatical·
i keep trying to use := in my code and it never works out well for me
English
0
0
0
21
Max
Max@Maxmatical·
another win for @ServiceNowRSRCH
Lewis Tunstall@_lewtun

This is the most impressive plot I've seen all year: - Scaling RL not only works, but can be predicted from experiments run with 1/2 the target compute - PipelineRL crushes conventional RL pipelines in terms of compute efficiency - Many small details matter for stability & scaling, notably CISPO for the loss, FP32 for the LM logits (generator <> trainer mismatch), and filtering zero-variance prompts Really big kudos to Meta and the authors for burning a bonfire of silicon to uncover these valuable scaling laws 🔥

English
0
0
0
72
Max
Max@Maxmatical·
@corbtt @winglian @jayendra_ram So youre comparing ood perf vs training on in distribution data and claiming frontier performance? Oof
English
0
0
0
18
Kyle Corbitt
Kyle Corbitt@corbtt·
@winglian @jayendra_ram Just talked to the researcher who built this. He held out 10 questions of the 100 and the final results we reported are just on those 10. The other comparison models in the first chart are also from running on those 10 questions only. We should have been much more explicit here.
English
1
0
2
128
Max
Max@Maxmatical·
@winglian Training on test remains undefeated
English
0
0
1
73
Wing Lian (caseus)
Wing Lian (caseus)@winglian·
The best benefit of the doubt I can give is they are reporting the 10% (10 rows) of the benchmark data they held out from training against? #L93-L98" target="_blank" rel="nofollow noopener">github.com/OpenPipe/open_…
English
1
0
15
5.2K
Max
Max@Maxmatical·
@xeophon Remember to give reka $$$ if you want more models kthx
English
1
0
0
118
Max retweetou
Reka
Reka@RekaAILabs·
Reka Research is our AI agent that scours the web to answer your toughest questions. Ready to unlock its full potential? Learn directly from the team who built it!
English
2
12
45
7.1K
Max
Max@Maxmatical·
weekly reminder that you should be trying reka flash and reka research
English
0
0
1
38
Max retweetou
Sharath Raparthy
Sharath Raparthy@sharathraparthy·
looking for an agent which excels at questions that require dozens of sources and delivers accurate responses with reasoning traces in a few minutes? Reka Research is here for you. start building today: docs.reka.ai/quick-start
English
0
5
18
1.4K
Max
Max@Maxmatical·
@teknium how so? this just shows the training loss, which could be easily explained by the lr schedule. you've already seen this type of loss in deepseek v1 with their lr schedule
English
0
0
8
1.1K
Max
Max@Maxmatical·
Your daily reminder to try reka flash 3.1 and reka research
English
0
0
0
43
Max retweetou
𝚐𝔪𝟾𝚡𝚡𝟾
𝚐𝔪𝟾𝚡𝚡𝟾@gm8xx8·
Reka Flash 3.1: an upgraded coding-optimized 21B LLM with strong agent finetuning potential - +10pt on LiveCodeBench v5 (Full) vs Flash 3
→ from improved RL training with verifiable rewards (RLOO) - Uses a REINFORCE variant with token-level loss, on-policy updates, DAPO-style long-sample handling, and dynamic sampling - Converts multi-choice math questions to fill-in-the-blank to avoid reward hacking - RL filters low-quality math examples; code rollouts execute trajectories on the fly - Post-trained on public + synthetic SFT data, then RL-aligned on math/code (Numina-1.5 + executable test cases) - Competitive with Qwen3-32B, o3-mini, Gemini 2.5 Flash on code tasks - Powers Reka Research, an agentic system for document/web QA - Llama-format + 3.5-bit quantized version released for local use - Primarily English; limited multilingual ability
𝚐𝔪𝟾𝚡𝚡𝟾 tweet media
English
1
13
56
3K
Max
Max@Maxmatical·
can finally share what i've been working on, pushing the sota frontier is pretty exciting 🙂, plus actual transparent reasoning traces is also good. pretty much been using this to replace google the past while, so give it a try! ps. more open source stuff coming soon 🤫
Reka@RekaAILabs

🚀 Meet Reka Research––agentic AI that 🤔 thinks → 🔎 searches → ✏️ cites across the open web and private docs to answer your questions. 🥇 State-of-the-art performance, available now via our API and Playground!

English
0
0
2
56
finbarr
finbarr@finbarrtimbers·
@natolambert I actually think there’s a pretty big business here
English
1
0
5
541
Nathan Lambert
Nathan Lambert@natolambert·
A lot of people vastly underestimate the amount of companies that cannot use Qwen and DeepSeek open models because they come from China. It's a common thing and makes the adoption of open models significantly slowed across enterprises (and mostly to Llama/Gemma).
English
40
12
346
43.3K

Descobrir

@natolambert @thdxr @eliebakouch @LiamFedus @barret_zoph @ServiceNowRSRCH @corbtt @winglian