Papers with Datasets

🪐 Introducing Galactica. A large language model for science. Can summarize academic literature, solve math problems, generate Wiki articles, write scientific code, annotate molecules and proteins, and more. Explore and get weights: galactica.org

🦿QDax: a Quality-Diversity benchmark suite for Deep Neuroevolution in Reinforcement Learning domain for robot control. It specifies different benchmarks based on the complexity of both the task and the agent controlled by a deep neural network. paperswithcode.com/dataset/qualit…

🏞ImageNet-X: a set of annotations of factors like pose, background, or lighting of the ImageNet validation set and 12K training images for studying the types of mistakes as a function of a model's architecture, learning paradigm, and training procedures. paperswithcode.com/dataset/imagen…

For more details about the dataset and full evaluation of current MT metrics. Do check out the link below! Paper, Code & Results: paperswithcode.com/paper/aces-tra… 4/4

Results analysis suggests three potential improvements: 1) Combine metrics of various strengths 2) Develop metrics that put more weight to the source instead of surface-level overlap with the reference 3) Include strategies to explicitly model more language-specific info 3/4

🎯ACES: a Translation Accuracy Challenge Set, consisting of 36,476 examples covering 146 language pairs and representing challenges from 68 phenomena, designed to evaluate a wide range of machine translation metrics. More details below… Dataset: paperswithcode.com/dataset/aces 1/4

We've explored the latest progress and ideas in the world of math world problem solving. We’ll continue to keep track of new and interesting developments in ML for math and the MATH benchmark. For more threads like this, be sure to follow @paperswithcode and @paperswithdata! 7/7

Minerva’s results have shown that the key in improving ML models’ problem solving ability does not solely depend on scaling the model but also novel ideas like majority voting. This encourages new algorithm alteration ideas to improve performance. 6/7

🤔How good are language models at solving math problems? Let’s review the progress of LMs on the MATH benchmark in this thread. 1/7

➗Lila: a mathematical reasoning benchmark consisting 23 tasks covering mathematical abilities, language format, language diversity and external knowledge. It's an extension of 20 datasets by collecting Python programs’ task instructions and solutions. paperswithcode.com/dataset/lila

{}Code Syntax: a dataset of programs annotated with the syntactic relationships in their syntax trees designed for code syntax understanding tasks. It contains 30K+ code samples annotated with 2M+ relation edges in 43 relation types for Python and Java. paperswithcode.com/dataset/codesy…

💻The Stack: a dataset for pre-training Code LLMs. It contains 3TB of permissively-licensed code in 30 programming languages. It’s created as part of the BigCode Project, an open scientific collaboration working on responsible development of Code LLMs. paperswithcode.com/dataset/the-st…

🏺Breaking Bad: a large-scale dataset of fractured objects. It has 10k meshes, each with 100 fractures mode, totalling 1M breakdown patterns. It serves as a benchmark for fractured object reassembly and new challenges for geometric shape understanding. paperswithcode.com/dataset/breaki…

🌄Diffusion DB: a large text-to-image prompt dataset containing 2M images generated by Stable Diffusion using user’s prompts and hyperparameters. It opens up research in prompt engineering, deepfakes detection, and designing public AI tools. paperswithcode.com/dataset/diffus…

👀Perception Test: a benchmark for evaluating the perception and reasoning skills of multimodal models. It uses real-world videos to define tasks that require understanding of memory, patterns, physics, and semantics across visual, audio, and text. paperswithcode.com/dataset/percep…