Lamin

Hi friends, I wrote a guest post for Lamin on using the open source LaminR package in an R workflow with the PBMC 3k dataset. Focus: provenance — tracking code, environment & execution order so analyses are reproducible when you (or someone else) comes back to them.

Read the full post: blog.lamin.ai/sparse-measure…

Existing data infrastructure can't make sparse measurements across millions of features queryable. Warehouses are too rigid, data lakes can't be queried, tabular lakehouses don't understand the formats. Biology needs a data lakehouse with support for bio-formats and registries.

We partnered with @jejomath to help us explain the relation between biology’s sparse measurements and the data lakehouse concept.

Two years ago we partnered with Mark Keller from Nils Gehlenborg’s Lab at Harvard to make Vitessce work seamlessly with LaminDB for interactive visualization of multimodal + spatial datasets. The integration has found much use across academia, biotech, and pharma — so we wrote up on design principles & use cases. This was a team effort involving Altana, Richard & Sunny in addition to Mark. Read the post: blog.lamin.ai/vitessce

What should the shared memory layer for agents and humans look like? Will it live in embeddings or in records? A high-level note.

Nice, detailed benchmark of backends that allow for batched training on a large scRNA-seq corpus - efficiently dealing with the specifics of a scenario can be a big engineering challenge, lowering this barrier will enable cool computational biology down the road!

What's a good way of organizing scRNA-seq data for training foundation models? Say you run 1k experiments and each measures counts for 1M cells with varying metadata and orthogonal data. Storing these data in one gigantic array isn’t exactly easy. We wondered whether it’s necessary to train foundation models and found 3 setups that made sense to us. lamin.ai/blog/arrayload…

Organizing scRNA-seq data with LaminDB - nxn.se/valent/2024/1/…

Thank you for the awesome collaboration, @marenbuettner! With Pytometry, we'd like to share readfcs: A package to load data and metadata from FCS files to AnnData. pip install readfcs

New tool: nbproject helps manage Jupyter notebooks! A lightweight open-source ELN for the drylab. pip install nbproject