Daniel Nichols

LongCoT is adding two new leaderboards! Due to the interest in agents (particularly RLMs), we’re adding a “Restricted Harness” and an “Open Harness” leaderboard. GPT 5.2 RLM from our paper is SOTA on “Open Harness” at 25.12%. We expect tool-use SOTA to exceed this very soon! On “Open Harness”, we allow all tool-use and code execution. On “Restricted Harness”, models may manage context, call subagents, etc, but may not write specific solver code (e.g. writing a BlocksWorld or Sudoku solver). We’re particularly excited about this leaderboard, as it allows agents to do their own context management, while sticking to LongCoT’s goal of testing models’ intrinsic reasoning capabilities.

We’re releasing LongCoT, an incredibly hard benchmark to measure long-horizon reasoning capabilities over tens to hundreds of thousands of tokens. LongCoT consists of 2.5K questions across chemistry, math, chess, logic, and computer science. Frontier models score less than 10%🧵

Meet our 2025 Fernbach Fellow, Daniel Nichols! His research explores how AI can be leveraged to maximize the efficiency & impact of HPC resources. computing.llnl.gov/about/people-h…

🚨 New paper alert 🚨 Can Large Language Models Write Parallel Code? @DanielNichols10 and @jhdavis_josh have conducted a comprehensive study of several state-of-the-art open-source code and GPT models for many parallel libraries/runtimes. #HPC #AI #DL arxiv.org/abs/2401.12554

2nd New Paper Alert! PSSG student @DanielNichols10 is tackling the problem of performance variability. Joint work w/ Aniruddha Marathe, Kathleen Shoga, & @tgamblin of @Livermore_Comp to be presented @IPDPS 2022. pssg.cs.umd.edu/assets/papers/… Interesting details about the paper 👇🧵

@TheCollectionNC Beautiful song!

**Blue day is out on all streaming platforms!** you’re not gonna believe me when I tell you it was written before the whole world got sick and worse. Anyway, be kind and unwind today!

Here’s my statement on today’s violence at the Capitol.

Thank you @thecollectionnc for spending 5,103 minutes with me this year on @Spotify. You are my #1. #2020Wrapped open.spotify.com/artist/3sva1Uj…

This Admiral Schofield poster had everybody out of their seat 😨 twitter.com/BleacherReport…

how to get funding: keep saying blockchain really fast until people in suits get confused and throw you money

Hi

Hey