Daft

.@SourcetableApp CTO @andrewgrosser shares his recommended tech stack for serious startups - a "wicked combination" that includes: - S3 + Cassandra for data - Daft for processing - Python, WASM, Ray Learn how they built the first AI-powered spreadsheet: daft.ai/blog/how-sourc…

Breaking: @huggingface and @CommonCrawl partnered to democratize access to the largest dataset for AI You can now load Common Crawl in one LoC from ANYWHERE and for FREE thx to pre-warmed CDN in multi-region/multi-cloud, no data movement fees, and to @daftengine @everettkleven

So excited to finally announce this collaboration with @huggingface @daftengine + @CommonCrawl is the best way to work with petabytes of internet data. Stay tuned, we’re just getting started💪🦾

Processing images, audio, and video alongside structured data in one pipeline is painful. Daft (@daftengine) is an open-source data engine built for multimodal AI workloads, with GPU/CPU co-scheduling and 5x lower memory than alternatives. More specs in the next post. #DevTools

Who's coming to #AUTONOMOUS? Our 50 curated speakers helping shape the future of robotics & physical AI 👇 1. Paul Mikesell @carbon_robotics 2. Sankaet Pathak @foundation_robo 3. James Kuffner @SymboticTweet 4. John Ha @bearrobotics 5. Tessa Lau @DustyRobotics 6. AJ Meyer @pickle_robot 7. Bilal Zuberi @redglassvc 8. Xiaodi Hou @BotAutoAV 9. Peter Wilczynski @vantortech 10. Shubham Shrivastava @KodiakRobotics 11. Juraj Kabzan @SkydioHQ 12. Vibhor Sood @burro_ai 13. Ziv Binyamini @ForetellixHQ 14. Tim Bucher @Agtonomy 15. David Lin Abundance 16. Adarsh Kulkarni @FoundryRobotics 17. Danny Bernstein @reservoirfarms 18. Aadeel Akhtar @PSYONICinc 19. Lukas Pankau Industrial Next 20. Kevin Peterson @BedrockRobotics 21. Rajesh Radhakrishnan @ServeRobotics 22. Ed Mehr @MachinaLabs_ 23. Kevin A. Damoa @GlidTech 24. Amos Miller @Glidance_io 25. Chris Chen @FaradayFuture 26. Samir Menon @DexterityInc 27. James Hardiman @DCVC 28. Tyler Niday @bonsairobotics 29. Grace Brown Andromeda 30. Peter Vaughan Schmidt @torc_robotics 31. Andrew Culhane @torc_robotics 32. Jari Safi @simberobotics 33. Leonardo Carvalho @solinftec 34. Kanu Gulati @khoslaventures 35. Noah Ready-Campbell @BuiltRobotics 36. Jamie Shotton @wayve_ai 37. Reed Ginsberg @ShinkeiSystems 38. Akash Gupta @GoGreyOrange 39. Brett McMickell Kubota USA 40. Andrew Wooten @RhodaAI 41. AIIvan Poupyrev @PhysicalAI 42. Sammy Sidhu @daftengine 43. Shayegan Omidshafiei @fieldai_ 44. Deepak Pathak @SkildAI 45. Rocket Drew @theinformation 46. Rya Jetha @BusinessInsider 47. Rishabh Aggarwal @Raise_Robotics 48.Harry McCracken @FastCompany 49. Russ Tedrake Toyota Research Institute 50. Kishor Veerashekar Plug & Play Ventures

🚢 Daft v0.7.16 has shipped. 🦾 ROBOTICS DATA PIPELINES (and we're just getting started) > daft.datasets.droid 76k robot manipulation demos, camera feeds, and language annotations. Load them as DataFrames, transform with expressions, feed into PyTorch with .to_torch_dataloader() 22 contributors, 44 changes. daft.ai/blog/daft-v0-7…

When contributors roll through ... 🔥🔥🔥 🦾 @_BabTuna_ has been crushing PRs lately on daft so we invited him out to the office for lunch. Come back any time!

We trained this robot 🤖 to dance Bhangra 🕺🕺🕺through @UFBots Looks sooooooooo cool! #UltimateBotsStudio

As execution gets cheaper, thought leadership becomes even more important.

Pretraining Physical AI models is materially more complex due to the multimodality. Video and sensor data inputs must feed action outputs. Feeding GPUs at line rate with performant video decoding is no small task, making Physical AI one of the hardest domains to maintain high GPU utilization and MFUs.

There's a difference between leveraging AI for everything and leveraging AI to help you automate your work. Daniel walks us through the difference.

After breaking his wrist in a scootering accident in SF, running to a Tinder date, Daniel realized he wanted to be able to contribute to open source, but also do it from Bali.

🚢 Daft v0.7.15 just shipped. try_cast() converts types without crashing your pipeline — invalid values become null instead of throwing a runtime error. Also in this release: LZ4 flight shuffle compression, UUIDv7 partition transforms, PostgreSQL source. daft.ai/blog/daft-v0-7…

More details in the blog! daft.ai/blog/disk-is-t…

On a TPC-H repartition across 32 workers: at 10 TB the object-store shuffle runs the head node out of memory past 1,000 partitions, while Flight Shuffle completes every partition count and runs 3.6 to 4.7x faster where both finish.

If a distributed query has to materialize more than a few terabytes of data, there's one operation that will dominate: the shuffle. Shuffling data at scale has been a real bottleneck for Daft users, so we took the time to fix the root cause and rebuild the shuffle from scratch.