Lapis Labs

Announcing the first fully AI-generated scientific discovery to pass the highest level of peer review – the main track of an A* conference (ACL 2025). Several groups have shown AI-generated work at workshops, but main conference acceptance is a far higher bar. While workshops often accept more than 50% of submissions, top-tier venues like ACL accept ~20%.

@soren_dunn_ @IntologyAI @haider1 @Dr_Singularity @LiorOnAI @LinusEkenstam @AiBreakfast @teknium thoughts?

@IntologyAI @haider1 @Dr_Singularity @LiorOnAI @LinusEkenstam @AiBreakfast @teknium curious what you think?

The 1st fully AI-generated scientific discovery to pass the highest level of peer review – the main track of an A* conference (ACL 2025). Zochi, the 1st PhD-level agent. Beta open.

📰 Zochi @ ICLR 2025! After productive discussions with the workshop organizers that accepted Zochi's work, we are pleased to announce that all work produced by Zochi has been approved for presentation at ICLR 2025. Intology reps will present on Zochi’s behalf. As requested by the organizers, papers/posters will note that authors don’t claim sole credit. The exact wording requested can be found in our blog below 🧵👇

🤖🔬Today we are debuting Zochi, the world’s first Artificial Scientist with state-of-the-art contributions accepted in ICLR 2025 workshops. Unlike existing systems, Zochi autonomously tackles some of the most challenging problems in AI, producing novel contributions in days—from idea to finalized publication. With a standardized automated reviewer, Zochi’s papers score an average of 7.67 compared to other publicly available papers generated by AI systems that score between 3 and 4. 🧵👇

🎉 Oral, Spotlight, NeurIPS, oh my! 🎉 We're thrilled to announce that two of our papers at #NeurIPS2024 have received special recognition: ❗Oral❗ presentation in the Datasets and Benchmarks track AND ❗Spotlight❗ 👏 Massive congratulations to our student researchers, Nikhil Khandekar and Soren Dunn, for their groundbreaking work on MEDCALC-BENCH, which earned them the Oral @ NeurIPS 2024 recognition, alongside our amazing collaborators at the National Institutes of Health. Catch us on stage! 🫡 📄 Paper: arxiv.org/abs/2406.12036 👏 And a big shoutout to @andyz245 for the fantastic work on RPO, which secured a Spotlight @ NeurIPS 2024! 📄 Paper: arxiv.org/abs/2401.17263 We’re incredibly grateful to the @NCSAatIllinois and National Science Foundation (NSF) for their invaluable computational support that helped make this research possible. If you’re attending NeurIPS this year, don’t hesitate to stop by and say hello! We look forward to seeing everyone in Canada 🇨🇦

Excited to announce three papers were accepted to #NeurIPS2024! 🤖 Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attacks (Spotlight) arxiv.org/abs/2401.17263 - Jailbreaking defense based on adversarial training Jailbreaking Large Language Models Against Moderation Guardrails via Cipher Characters arxiv.org/abs/2405.20413 - A cipher-based attack on guardrail models like LlamaGuard RedCode: Risky Code Execution and Generation Benchmark for Code Agents - A benchmark for evaluating unsafe code execution and generation Check out our findings! Looking forward to visiting Vancouver 🇨🇦

Seems like Kira has thoughts on Tamper-Resistant Safeguards... On alphaXiv: alphaxiv.org/abs/2408.00761…

@cais @rishub_t @8hrugu Discussion is open on @askalphaxiv: alphaxiv.org/abs/2408.00761

🎉 We are very excited to be announcing our work done with CAIS (@cais)! Congratulations to our student researchers @rishub_t and @8hrugu for their amazing work along with the rest of the authors on this project. Read @rishub_t's tweet below to learn more 🫡

@cais @rishub_t @8hrugu Our work was also featured in Wired: wired.com/story/center-f…

@cais @rishub_t @8hrugu arXiv: arxiv.org/abs/2408.00761

Happy to announce our collaboration with @askalphaxiv 🫡 "We are incredibly excited to support this effort by helping develop their reviewer program with our student researchers at Lapis Labs, along with utilizing this service to promote direct discussion with our work and authors." Read more: lapisrocks.substack.com/p/working-with…

“This was a highly interdisciplinary project, where we had individuals not only from computational backgrounds, but also had MD students from Yale, UIC, UChicago, and Rosalind Franklin University who helped us ensure that the patient notes in the dataset were appropriate for each calculation task.”¹ - Nikhil Khandekar¹, student researcher

We are excited to have collaborated on MedCalc-Bench, with the @NIH MedCalc-Bench is the first computation-based benchmark for evaluating LLMs in clinical settings. Paper link: arxiv.org/pdf/2406.12036 Dataset link: huggingface.co/datasets/ncbi/… Shout out to our students Nikhil Khandekar Soren Dunn! Read more about the work: lapisrocks.substack.com/p/medcalc-benc…

We are excited to introduce FIRST! Our novel LLM reranking approach boosts efficiency by 50% while maintaining performance. Paper link: arxiv.org/pdf/2406.15657 Big thanks to @gangi_official and @hengjinlp, along with our collaborators @IBMResearch (@aviaviavi__ and Arafat Sultan)