Jon Tow

Reinforcement Learning in the era of LLMs requires scalable, distributed systems to push the boundaries of reasoning and alignment. Today - we release Atropos - our RL environments framework. github.com/NousResearch/A… Atropos is a rollout framework for reinforcement learning with foundation models that supports complex and diverse environments for advancing the capabilities of foundation models. In Greek mythology, Atropos was the eldest of the three Fates. While her sisters spun and measured the threads of mortal lives, Atropos alone held the shears that would cut these threads, determining the final destiny of each soul. Just as Atropos guided souls to their ultimate fate, this system guides language models toward their optimal potential through reinforcement learning. The work on Atropos was led by @dmayhem93 and built alongside @teknium, @rogershijin, @max_paperclips, @nullvaluetensor, @JSupa15, @artemsya and @karan4d

Glad to see async LLM RL getting more recognition! Our Towards System 2 paper highlights its importance under infrastructure: arxiv.org/abs/2501.04682. Also, NeoX already supports this (github.com/EleutherAI/gpt…) with shared memory, allowing you to save vram by using the same memory for training and inference, making in-flight weight updates instant

@aman_gif spaces.ac.cn/archives/9577

why qwen use bias in qkv?

6ND matches reality until n_ctx ~= 32K . Brought to you by "flex-attention breaks FlopCounterMode."

@shxf0072 @torchcompiled @danielhanchen earlier #diff-1ec36c18d77be0fa276598d233f250404a8c9a80fd2d5956988d5edabe6332edR34" target="_blank" rel="nofollow noopener">github.com/Dao-AILab/flas…

@torchcompiled @danielhanchen found it earlier, unsloth.ai/blog/gemma

god bless torch.testing._internal.common_distributed.spawn_threads_and_init_comms

@teknium github.com/WecoAI/aideml

What is AIDE

Generative Reward Models Presents GenRM, an iterative algorithm that trains an LLM on self-generated reasoning traces, leading to synthetic preference labels matching human preference judgments arxiv.org/abs/2410.12832

My favorite bit in this paper: I and @bbrabbasi wrote an appendix formalizing what is done evaluating models with loglikelihood multiple choice and perplexity evals. afaik, none of this has been written up in one place in most papers and just been tacitly assumed before!

@abacaj No. Pure NL. The way Terry Winograd would like it.

@jonbtow no math, code?

@TeraflopAI @Geronimo_AI @dmayhem93 @StabilityAI Yup! We use the FLAN sources from huggingface.co/datasets/DataP… collected by TeraflopAI's very own @EnricoShippole. It's mentioned in our 1.6B technical report and tagged in the metadata of all Stable LM 2 base model cards.

We are releasing trillions of high-quality, copyright-free, permissively licensed tokens and multimodal data. Be sure to follow our releases @TeraflopAI.

Pre-train mix in arxiv.org/abs/2402.17834 if you want to replay some for continued pre-training. Open data mix >>>

Stable LM 2 12B is a pair of powerful 12 billion parameter language models trained on multilingual data in English, Spanish, German, Italian, French, Portuguese, and Dutch, featuring a base and instruction-tuned model. You can now try the model here: huggingface.co/stabilityai/st… (1/3)

How to define Diversity in the context of CodeLMs and Programming Languages ? 1. Diversity is positively correlated with Performance in solving a problem. 2. Shortcomings of diversity in small codeLMs. 3. Code Embedding models don't capture semantics. reshinthadithyan.github.io/blog/2023/code…

@TeraflopAI is excited to help support the @caselawaccess and @HarvardLIL, in the release of over 6.6 million state and federal court decisions published throughout U.S. history 🥳

Releasing Yarn-Llama-2-13b-128k, a Llama-2 model, trained for 128k context length using YaRN scaling. The model was trained in collaboration with u/bloc97 and @theemozilla of @NousResearch and @Void13950782 of @AiEleuther.

Releasing a new PaLM 2.1b model trained at a context length of 8k on C4. This model release is a continuation of the previously released 150m, 410m, and 1b models. twitter.com/EnricoShippole…

Introducing three new open-source PaLM models trained at a context length of 8k on C4. Open-sourcing LLMs is a necessity for the fair and equitable democratization of AI. The models of sizes 150m, 410m, and 1b are available to download and use here: github.com/conceptofmind/…

#ML #AI #NLProc Researchers, especially academics: I’m writing a paper about accessible finetuning of transformer models. If you’re looking to finetune a LLM, where on this list of compute access do you fall? If you don’t fall on this list, what resources are accessible to you?