Wauplin

Available right now at github.com/huggingface/hf… and soon in any Spaces or Jobs ran on HF infra!

Storage Buckets gave the Hub a mutable storage layer. HF-MOUNT makes that storage feel local. Mount models, datasets, and Buckets as a filesystem and work with them through normal paths. Buckets are read-write; models and datasets are read-only.

@sudoingX Link to GH PR for more explanations: github.com/huggingface/xe… TL;DL: upgrade your dependencies, should work smoothly :)

@sudoingX My previous comment is still valid, but you were right there was a bug in hf-xet that could lead to deadlocks while streaming large files from the Hub. It is now resolved and shipped: x.com/Wauplin/status…

hey if you're downloading large GGUFs from huggingface and the download keeps crashing with "File Reconstruction Error: Background writer channel closed" it's not your connection. it's HuggingFace's new Xet storage backend. it can't reliably reconstruct files over ~15GB during download. known bug, still open. the disable flag doesn't work either. HF_HUB_DISABLE_XET=1 is supposed to force legacy downloads but it doesn't. confirmed broken (github issue #3266). the actual fix: pip uninstall hf-xet removes the Xet client entirely. huggingface cli falls back to legacy HTTP. downloads complete. no crashes. found this trying to download a 48GB model on a production. burned 45 minutes before tracing it. save yourself the time.

@huggingface Full release notes: github.com/huggingface/hu…

If you transfer large files from the @huggingface Hub, upgrade now huggingface_hub v1.7.0 / hf-xet 1.4.2 fixes: - deadlocks/stalls on large file downloads - faster uploads by cutting redundant SHA-256 computation - extended timeout for shard uploads (was silently failing)

@realmrfakename How did it go? 😀

About to try to upload 100TB to try out HF Buckets... 👀

get PRO on @huggingface and instantly 10x your storage to 1 TB private + 10 TB public ...for $9 a month 😮 a deal this good should be illegal

It's 2026. Build. For. Agents. We just launched Hugging Face Storage Buckets, a faster storage layer for AI workflows, 3x cheaper than S3 with no Git overhead. Buckets are basically built for workflows that generate artifacts: checkpoints, logs, traces, and other intermediate outputs produced by training jobs and pipelines. They come with a CLI designed with coding agents in mind. Agents can easily explore commands, examples, and documentation via --𝚑𝚎𝚕𝚙 (there’s a Skill for that as well), then sync directories, inspect bucket contents, move files around, and script storage operations. The CLI interface makes Buckets easy for agents to use: they can chain commands, automate storage tasks, and integrate Buckets directly into scripts, pipelines, and agent workflows.

@sudoingX I'd encourage you to try this out. We've made a lot of patches in the past weeks to ensure downloads are fast and reliable, and even more for large downloads. If you encounter any new issue let me know!

@sudoingX Hey @sudoingX just as a heads-up: - HF_HUB_DISABLE_XET=1 does work. You just have to set it before launching the script (as explained in the issue you've linked) - huggingface-cli is pretty outdated. Much better to update to latest version and use `hf` CLI #install-the-hugging-face-cli" target="_blank" rel="nofollow noopener">huggingface.co/docs/huggingfa…

@Chain_AlphaX Yes!

@Wauplin Hmm, so like… storage for AI stuff? Interesting.

1/6 Today we’re introducing Storage Buckets on the Hugging Face Hub. They’re built for mutable, non-versioned ML artifacts: checkpoints, optimizer states, processed shards, logs, traces, eval outputs, and agent-generated files.

6/6 Mental model: Buckets = mutable working data Repos = finalized, versioned deliverables That gives ML teams a cleaner separation between storage and publication, while staying in one Hub-native workflow. huggingface.co/blog/storage-b…

5/6 Typical use cases: - continuously writing checkpoints during training - storing intermediate outputs in data pipelines - keeping agent traces, memory, and shared artifacts remote - separating working data from published assets

NEW in huggingface_hub v1.6.0 2 new datasets commands in the hf CLI: 📁 `parquet` - list & fetch parquet files for any dataset 🔍 `sql` - run SQL queries on datasets with DuckDB Let your agent query 900k+ datasets from the CLI

More from huggingface_hub v1.5.0 (beyond Buckets 🪣): Spaces now supports hot-reload (live edit of Gradio files). And the CLI got an extension system à la `gh extension`, plus native support for AI agents (@opencode, @cursor_ai ). Full notes: github.com/huggingface/hu…