Sabitlenmiş Tweet

🚀 A new way to refine LLM training data is here!
Scaling LLMs is no longer just about collecting more data — it’s about making every token count.
Introducing UltraX, our framework for large-scale pre-training data refinement. Instead of rewriting data end-to-end, UltraX uses structured editing functions to precisely improve training data quality.
What’s inside? 👇
✅ Function-calling refinement
Fine-grained editing with insertion, deletion, and modification.
✅ Reliable data transformation
LAM + DCR pipelines generate high-quality supervision for scalable refinement.
✅ Better data efficiency
Achieves the best average performance across five corpora in 1B-model pre-training experiments.
No gatekeeping — explore how better data can help build better models.
📄Paper: arxiv.org/abs/2607.08646
💻 GitHub: github.com/openbmb/UltraX
🤗 Hugging Face: huggingface.co/datasets/openb…
Modelscope: modelscope.cn/datasets/OpenB…
modelscope.cn/models/OpenBMB…


English























