LanceDB

1.1K posts

LanceDB banner
LanceDB

LanceDB

@lancedb

The multimodal lakehouse for AI, accelerating large-scale data curation and feature engineering so teams can build better models faster.

San Francisco, CA Katılım Nisan 2023
65 Takip Edilen4.4K Takipçiler
LanceDB
LanceDB@lancedb·
@Ceph has never had native nearest neighbor search, so RAG meant bolting on a vector database and running two systems. 🖥️ July 16, @seagate's Christine Bassani and @ibm's Kyle Bader are joined by Prashanth Rao @tech_optimist from LanceDB to show embedding search built directly into Ceph, through the S3 Vectors API. Billion scale, multi tenant, same storage stack you already run. 🖇️ Register: na2.hubs.ly/H06F5HW0
LanceDB tweet media
English
0
0
7
330
LanceDB
LanceDB@lancedb·
Embodied AI runs on complete real-robot trajectories covering long-horizon deformable manipulation, dual-arm coordination, and recovery from failure. The bottleneck is rarely the model. It's the data. Lion Rock rebuilt their foundation on Lance. One copy in S3, read and written in place. Long videos cut into GOP blobs so you keep video-level compression and still get frame-level random access. Actions, states, video, and annotations land in a single table row. The result is 1.7-6.0x faster random reads than their previous system and roughly 42% storage savings, with no high-frequency action data lost. Read how they did it: na2.hubs.ly/H06Cvb90
LanceDB tweet media
English
0
0
1
264
LanceDB
LanceDB@lancedb·
🌟 June's highlight: Lance averaged 140ms commit latency across a 10,000-commit S3 benchmark, versus 534ms for Delta Lake and 457ms for Iceberg. lancedb.com/blog/a-metadat… 1/6
LanceDB tweet media
English
1
1
8
630
LanceDB
LanceDB@lancedb·
And that's a wrap on World's Fair in SF 🎉 Thanks to everyone who stopped by the booth to say hi and dive into LanceDB internals. Some of the best conversations happen at a conference booth and this year was no exception. It was great meeting everyone who came out to SPIN SF with @theoryvc & @ollama and to our USA World Cup watch party with @ExaAILabs, @ExtendHQ, & @fastinoAI last week. Both were packed, both were a good time, and we'd absolutely do them again. See you at the next one 👋
LanceDB tweet media
English
3
2
15
4.9K
LanceDB
LanceDB@lancedb·
How do you trace one number in a 200 page ESG report back to the exact page it came from? We dug into that with the @llama_index team behind LiteParse. We tested five ways to retrieve evidence across six sustainability reports and 50 labeled questions. Fusing pages, chunks, and figures across separate LanceDB tables, joined by one page_id, got 82% any-page-hit@5 at 4.7ms. An agent answering from that got 74% right. Full pipeline and benchmarks: lancedb.com/blog/from-mess…
English
1
3
4
1.6K
LanceDB
LanceDB@lancedb·
LibriSpeech in Lance format on @huggingface. One table holds FLAC audio, transcripts, speaker metadata, and transcript embeddings, with search and FTS indices pre-built. Semantic search over transcripts never touches the audio column, even at full scale. huggingface.co/datasets/lance…
English
0
4
17
781
LanceDB
LanceDB@lancedb·
Qwen2.5-VL-3B QLoRA fine-tuning on TextVQA: the vision tower is frozen during training but most pipelines still run it on every epoch. Compute the embeddings once, add them as a new column on the same Lance table as the raw images, no table rewrite, then drop the vision tower from the training loop entirely. lancedb.com/blog/faster-vl…
English
1
3
10
737
LanceDB
LanceDB@lancedb·
1/3 At @aiDotEngineer World's Fair this week. Booth B1, come say hi, grab swag, or get your hardest data stack questions answered by the people who built it. Building training pipelines, curation workflows, or feature engineering on multimodal data at scale? Come find us!
LanceDB tweet media
English
1
0
4
372
LanceDB
LanceDB@lancedb·
2/ How it works: → Each Lance fragment maps 1:1 to a Presto split, reads fan out across workers automatically → SQL filters compile down to the native Rust scanner, pruning rows on disk before JVM memory load → Arrow RecordBatches stream across JNI with no serialization overhead
English
1
0
2
255
LanceDB
LanceDB@lancedb·
1/ Lance is now a native @prestodb connector. Run distributed SQL across Lance tables and join them against Hive, Iceberg, or Postgres in a single query.
LanceDB tweet media
English
1
2
14
781
LanceDB
LanceDB@lancedb·
Updating a label on a row that contains a 50MB video shouldn't require Spark to read that video. Lance Blob V2 in Lance Spark keeps blob columns as descriptors through the plan. Bytes materialize only on write. A blob column stays a column. The SQL stays the same. What changes is what Spark actually carries through the plan. lancedb.com/blog/lance-blo…
English
0
1
13
637