Dynabench รีทวีตแล้ว
Progress in AI continues to outpace benchmarks.
Check out this new plot, inspired by @DynabenchAI, that shows just how quickly it's happening.
Read more about it here: contextual.ai/plotting-progr…
English
Dynabench
90 posts
Big congrats to longterm Dynabenchers @douwekiela and @apsdehal on their exciting launch! Looking forward to seeing what you build! 🎉
Douwe Kiela@douwekiela
Super excited to announce that @apsdehal and I have launched a new company: @ContextualAI! Why did we start it? Because LLMs are going to radically change the way enterprises operate, and we see a huge need for LLMs that actually work for enterprise use cases. 1/5
English
@DADCworkshop @MLCommons 📣Big shout out to @23rd_Conspiracy @ciropython 📣 for bringing all these exciting updates to the platform! We’re excited to share the future of Dynabench with you!
7/7
English
@DADCworkshop @MLCommons We've also streamlined the process for leaderboard users! Our new, easy-to-use templates that can support many kinds of challenges, and any python-based machine learning framework (tensorflow, pytorch, sklearn etc). Log in to learn more about it here: dynabench.org/tasks/hs/uploa…
6/7
English
At Dynabench, we're gearing up for the AI race, and we embrace the rapid pace of change! We're excited to announce some big updates that make our new-and-improved platform faster, better, and easier to use, for leaderboard users, dataset creators, and benchmark owners! 1/7
GIF
English
The first round of Dataperf 2023 challenges are open for participation through May 26th, 2023! Challenge winners will have the opportunity to present their work @icmlconf #ICML2023! Visit dataperf.org to learn more and get involved!
English
We are excited to welcome the #DataPerf community to the Dynabench platform! DataPerf consists of 5 data-centric challenges spanning data selection, cleaning, generation and valuation across speech, text and vision modalities. Head over to dynabench.org/dataperf to explore them!
English
Dynabench รีทวีตแล้ว
Today, Meta researchers together with @MLCommons working group, are launching DataPerf, the first platform for building data & data-centric AI algorithm leaderboards.
We're excited for how DataPerf will help to push the data-centric AI field forward ⬇️
English
Dynabench รีทวีตแล้ว
Announcing the BabyLM 👶 Challenge,
the shared task at @conll_conf and CMCL'23!
We’re calling on researchers to pre-train language models on (relatively) small datasets inspired by the input given to children learning language.
babylm.github.io
arxiv.org/abs/2301.11796
English
@rebeccatqian The Dynabench project welcome new ideas for metrics! If you have a cool idea for better measuring fairness, robustness, throughput, memory, performance, or anything else, we're listening! Feel free to reach out, or submit a PR to have your metric included on the platform! 2/2
English
Exciting new work on fairness at #NeurIPS2022: @rebeccatqian will present Perturbation Augmentation for Fairer NLP tomorrow at the RobustSeq WS (robustseq2022.github.io). We're excited to apply this work to improve fairness measurement on Dynabench! You won't want to miss it! 1/2
English
Dynabench รีทวีตแล้ว
Great talk today by @max_nlp from @ucl_nlp @CohereAI on Dynamic Adversarial Data Collection at the @kclinformatics Distributed AI group!
English
Dynabench รีทวีตแล้ว
Ever wanted to know more about generalisation in NLP but overwhelmed with the number of papers on ArXiv? Fear not! We read 400+ papers, 600+ experiments, and designed a taxonomy 📝 to categorise the research for you! (1/n) 🧵
arxiv.org/abs/2210.03050
English
Dynabench รีทวีตแล้ว
How can we improve benchmarking? The @DynabenchAI experiment aims to make faster progress with dynamic data collection, and today, we are pleased to introduce our next stage: @MetaAI has funded 5 exciting research proposals on the theme of "Rethinking Benchmarking”! Congrats to:
English
We are very excited to be teaming up with @MLCommons! Stay tuned for our upcoming new datasets and other updates!
MLCommons@MLCommons
Today @MLCommons has formed a Working Group to develop the open-source Dynabench Platform to support benchmarking of datasets, data-centric algorithms & models. Join us in helping to build the data-centric AI community! Visit mlcommons.org/en/groups/rese…
English
Dynabench รีทวีตแล้ว
"DataPerf: Benchmarks for Data-Centric AI Development"
What if instead of holding the data constant and benchmarking different models, we held the model constant and benchmarked different data pipelines? [1/7]
English