Bruce Sun

OpenResearcher Unleashing AI for Accelerated Scientific Research discuss: huggingface.co/papers/2408.06… The rapid growth of scientific literature imposes significant challenges for researchers endeavoring to stay updated with the latest advancements in their fields and delve into new areas. We introduce OpenResearcher, an innovative platform that leverages Artificial Intelligence (AI) techniques to accelerate the research process by answering diverse questions from researchers. OpenResearcher is built based on Retrieval-Augmented Generation (RAG) to integrate Large Language Models (LLMs) with up-to-date, domain-specific knowledge. Moreover, we develop various tools for OpenResearcher to understand researchers' queries, search from the scientific literature, filter retrieved information, provide accurate and comprehensive answers, and self-refine these answers. OpenResearcher can flexibly use these tools to balance efficiency and effectiveness. As a result, OpenResearcher enables researchers to save time and increase their potential to discover new insights and drive scientific breakthroughs.

🥳🥳

🥳 Work done with my excellent collaborators and advisors: @lockonlvange, @WeizheY, @stefan_fee. (7/7)

Could I trust LLM critique? 🔥 We are the pioneers in prioritizing critique evaluation and introducing the critique of critique, termed MetaCritique. Repo: github.com/GAIR-NLP/MetaC… Paper: arxiv.org/abs/2401.04518 (1/7)

🤔 MetaCritique ranks critique models. AUTO-J is the best in Meta-R and Meta-F1. Human and GPT-3.5 achieve Meta-P exceeding 80%, surpassing all open-source critique models. So the research of open-source critique models should pay more attention to factuality issues. (6/7)

🏆 The superior critique chosen by our MetaCritique enhances refinement significantly compared to its counterparts. (5/7)

🏆 Meta-evaluation experiments (including pairwise comparison and correlation coefficients) show that our MetaCritique beat its counterparts. Moreover, Meta-P and Meta-R scores are mutually supportive. (4/7)

🏆 Through human evaluation and extensive experiments, we demonstrate that GPT-4 achieves near-human performance, confirming the feasibility of prompting GPT-4 to power our MetaCritique. (3/7)

🚀MetaCritique establishes criteria : Meta-P: precision score evaluates factuality. Meta-R: recall score evaluates comprehensiveness. Meta-F1: harmonic mean of Meta-P and Meta-R. 👊 MetaCritique is more interpretable and transparent due to our proposed AIUs. (2/7)

🔥Introducing Auto-J: A 13B generative judge for LLM alignment evaluation on myriad scenarios with detailed explanations. * Far superior to ChatGPT, critique better than GPT-4 * Out-of-the-box, reference-free, support multiple evaluation protocols gair-nlp.github.io/auto-j/ 1/n