SeaweedFS

Refactor complete. Swapped out MinIO for SeaweedFS. Added an encrypted blob export and local offline decrypt option too via the arkfile-client Go CLI util. github.com/arkfile/Arkfile Much of this would not be possible as a 1 man team w/o @PPQdotAI / @cline / @AnthropicAI Opus 4.6

@derekdfulton Welcome to create github issues for any performance problems. For optimization, "weed mount" has direct access to storage rather than limited by S3 APIs. Recently it added per-mount POSIX locks and writeback caching.

Right!? That was my first thought as well. I've been experimenting with rclone and JuiceFS recently. How does Seaweed compare to these for edge cases such as large concurrent write (two users on same bucket/object) or repeated read/writes? rclone is really good as it uses a cache for this kind of situation. I feel like JuiceFS + Seaweed could be a serious combo!

Nice! SeaweedFS already does this! Use FUSE mount to access the files in s3 buckets.

A really big S3 announcement - #S3 Files!! If you've ever had to copy data out of S3 just so a file-based tool could use it, you know the frustration. S3 Files lets you mount S3 buckets as file systems directly, no data duplication, no sync pipelines, no workarounds. Your data stays in S3 while apps access it natively. This looks really useful and a nice improvement over previous approaches like S3 Mount Point. AI agents, ML pipelines, and traditional file-based applications can all read and write to S3 as a file system without code changes. #aws #S3Files lckhd.eu/yt6Tz5

SeaweedFS on NVidia doc: docs.nvidia.com/tao/tao-toolki…

@RisingWaveLabs Nice! Does it cache S3 listing requests?

S3 is cheap… until you start reading from it. Modern data systems (like RisingWave) are S3-native. Great for storage, but frequent reads can add latency and cost. So what’s the fix? A smarter caching strategy. This architecture shows how it works: Memory cache (fast, but limited) Elastic disk cache (NVMe/EBS, bigger, still fast) S3 (cheap, durable, but slower) In RisingWave, the elastic disk cache uses Foyer, a hybrid cache engine that keeps hot data closer to the compute. Foyer, built by Yao Meng (@CroxxMr), is also used in: Chroma (vector DB for LLMs) SlateDB (embedded object storage engine) Why it matters: Fewer S3 reads → lower costs Lower latency → faster queries Better scaling → less pressure on object storage In production, this approach has shown: ~10× fewer S3 operations ~90% cost reduction Much more predictable performance The takeaway: Don’t just store data in S3, cache it intelligently.

@Sirupsen Cloud object stores all hosted mixed data together to increase storage efficiency. Latency is not a priority. Why not use your own object store?

Between S3, GCS and Azure, there is no undisputed winner on small object GET latency. It depends heavily on the size of object and percentile (128 threads sampling 512 objects, no overlap)

That left Garage and SeaweedFS. After trying to set up both on my low-end Oracle VPS, SeaweedFS won me over! It’s incredibly lightweight, simple to get started with, and fits perfectly on my small AMD server. (5/6) #SelfHosted #Developer

@kapibara19 Let us know your experience!

seaweedfsで遊んだ

@changhiskhan Which object store this is using? Want to compare it with SeaweedFS?

Object store is sooooo slow for search

TIL: SeaweedFS supports Amazon S3 Tables github.com/seaweedfs/seaw…

@DoDontoF_Srv "can barely do anything" what are you trying to do and can not do?

minioからの脱出を試みているけど、 seaweedfs: 管理UIがAlpha版でほぼ何もできない。コマンド複雑すぎ。 rustfs: ライフサイクルポリシーとバケットのセキュリティポリシーが未実装。パスワードが平文保存。 と、どちらも置き換えるにはびみょい。

@lcheylus "There are structural limitations to resilience, notably the inability to write during volume backups, which is a deal-breaker for critical 24/7 operations." Not true! Writes would be assigned to a different volume. This is a unique feature to achieve HA when servers are down.

Suite au passage en maintenance mode du projet MinIO, comparaison des solutions alternatives pour le stockage S3 : SeaweedFS, RustFS, Garage, CubeFS et Ceph - Avantages, inconvénients et gouvernance de chaque projet #S3 #OSS blog.cockpitio.com/cloud/minio-ma…

SeaweedFS supported S3 APIs comparing with Minio. github.com/seaweedfs/seaw… This is generated by AI and serve as a TODO list.

@NicolasKoehl @peaklabs_dev Try to use "weed mini" for instant setup.

@peaklabs_dev They really had a great UI. Seaweedfs is a good fallback but hard to setup

And the circle is complete. MinIO is now archived. 1️⃣ MinIO deprecated the UI. 2️⃣ They stopped providing images and pre-built binaries, going source code only. 3️⃣ The documentation “disappeared”, available only to enterprise users. 4️⃣ MinIO started to only receive security updates and bug fixes - no new features. 5️⃣ MinIO gets archived and the CE version is no longer maintained.

@derekdfulton @illumina @Minio If any issue, create a github issue and we will fix it.

Does anybody know if it's possible to connect an @illumina machine to a non-AWS S3-compatible bucket, like @SeaweedFS or @Minio?

@simonw @HamelHusain @JnBrymn Interesting! Thanks!

@HamelHusain @JnBrymn It's working really well for me so far! I used it to experiment with SeaweedFS in Claude Code for web this morning github.com/simonw/researc…

AI coding: I feel like I’m a glorified UX tester lately 🤣

SeaweedFS adds Table Buckets and Iceberg Catalog. You can use tools such as Spark, Trino, etc, to read or write Iceberg tables, with full SigV4 authentication. github.com/seaweedfs/seaw… github.com/seaweedfs/seaw…

@eatonphil can I join? :)

If you've run seaweedfs, versity, juicefs, beegfs, or rustfs—permissively-licensed s3-compatible storage systems— and are open to chatting about your experience, I'd like to chat with you for The Consensus.

@valyala @valyala Nice talk! I am thinking to push down the queries to the storage layer. It seems SeaweedFS can store the data of each column and return query results directly. Let me know if you are interested.

Slides from my talk at Go Kraków Meetup "Writing a TSDB from scratch in Go. Performance optimizations" docs.google.com/presentation/d…