
Michaela Hinks
3.2K posts

Michaela Hinks
@MichaelaThinks
automating myself at @EdisonSci | PhD with @BintuLab and @WJGreenleaf at Stanford | views/opinions my own




















Today, we’re pushing a major update to Edison Analysis, our data analysis agent, which is tuned for scientific research and SOTA across data analysis benchmarks. In contrast to Kosmos, which runs for 6-12 hours and produces tens of thousands of lines of code, Edison Analysis runs for seconds to minutes and is best for specific, well-defined computational tasks. It is available both on our platform under the Analysis tab, and via API, and costs only one credit per run, so it is available to users on both free and paid tiers. Edison Analysis is a modified version of the data analysis agent Kosmos uses in its trajectories. Try it out! One of the most important improvements over our previous data analysis agents has been the addition of a specialized data retrieval tool. Edison Analysis can either use this tool to access data, or can pull data down directly via API. To evaluate this tool, we ranked the most commonly used public data repositories across recent papers from BioRxiv, and created a new benchmark that measures the ability of a language agent system to retrieve raw data from those sources. Edison Analysis gets 71% on this benchmark, and we’ll be working to increase this over time. You can read more about our benchmarks in the our blog post, link below. Some features worth highlighting: 1. Edison Analysis produces a report on the analysis it runs, along with a Jupyter notebook that you can download to reproduce the analysis yourself. Every figure it produces is linked back to the specific lines of code used to produce the figure, to make it easy to reproduce. 2. It works well with both Python and R. 3. One of the best uses for Edison Analysis is to use it to retrieve datasets that you can then analyze with Kosmos. We have a bunch of major improvements to Edison Analysis coming in the next few months that we’re excited to share. In the meantime, congratulations to the team, especially @ludomitch, @jonmlaurent, @cvi94 , @alexjandonian, and many more.




Kosmos is unlike any other agent we have at Edison, both in terms of outputs and infrastructure. Running at scale requires our platform to support order-of-magnitude swings in resource requirements, all unknown at submit time. Each run sees between 0 and 120 sandbox environments with up to 4 TB of memory, between 125 and 3,200 documents parsed, and timing between 1 hour and 32 minutes and 14 hours and 12 minutes. It's like a bad interview question!


The people have spoken! We're doubling the number of free credits for edu users to 1300, enough for six Kosmos runs + a bunch of agent runs. These expire after a month, so use them before they run out! Also, we are announcing a satisfaction guarantee. If your Kosmos run doesn't produce good results, we'll give you the credits back. This applies to all users. You'll see the new credits in your account shortly. If you want to claim credits under the satisfaction guarantee, contact us via our website. (We'll make some UI changes to make that easier soon.) Have fun, and share your results as you get them. x.com/SGRodriques/st…

Have been playing around with Kosmos for the last couple of months, and wanted to share a line of analysis I thought was really cool (at least for me!) where Kosmos tried to use protein structure searching.


