Blosc

Unsure about the optimal compression parameters for Blosc2? 😖 Introducing Btune. It predicts the best parameters based on your preferences 🌟 Select the best compression ratio, best speed, or a tradeoff between them, and let Btune decide for you😄 👉btune.blosc.org

We recently implemented read-ahead capabilities in Blosc2-Python to interleave computation and I/O as much as possible. Stay tuned for our forthcoming release of Python-Blosc2 3.0. We are working super-hard into it. Make compression better 😀

@leo_ayala01 Ooops. Deleted the original post and published a new one. Thanks!

📢 Learn how using Blosc2 and Btune is improving the compression ratio of data (both lossless and lossy) coming from photon sciences. We were able also to reach compression speeds exceeding 23 GB/s. Report 👉blosc.org/docs/LEAPS-INN… Make compression better 😀

Caterva2 uses #Python-Blosc2 for achieving high performance compression and out-of-core computation

New JPEG2000 codecs, HDF5 integration, deep learning via Btune, sparse data handling and remote Blosc2/HDF5 access and visualization. Check how we have accomplished all these tasks in the framework of the LEAPS-INNOV program: blosc.org/docs/Blosc2-HD… We make compression better

ANN: C-Blosc2 2.14.4 has been released! 🥳🥳 This is a patch release where the SOVERSION was bumped since the API changed. For more info see : github.com/Blosc/c-blosc2… Enjoy!😄

ANN: Btune 1.2.0 is out! 🥳🥳🎇 This is a minor release where new support for tuning compression parameters taking into account not only the compression ratio and speed, but also the quality was added. To get an idea of how this works, see: #btune-quality-mode" target="_blank" rel="nofollow noopener">github.com/Blosc/Blosc2-B…. Enjoy!

ANN: blosc2-grok 0.3.1 has been released! This is a patch release so that now aarch64/Linux wheels are available. You can have a look at the release notes in: github.com/Blosc/blosc2_g… Enjoy!😁

ANN: blosc2-grok 0.3.0 is out! 🥳This is a minor release where a simpler way to set the `rates` value through the `codec_meta` from blosc2 cparams was added. For more info, you can have a look at the release notes in: github.com/Blosc/blosc2_g… Enjoy!😄

🚀 Discover the latest insights in data analysis, compression and visualization with this article from SciPy 2023: conference.scipy.org/proceedings/sc… Delve into practical techniques and strategies for enhancing performance in Python-based data processing. #DataScience #Python #SciPy2023

ANN: blosc2-grok 0.2.3 is out! 😁This is a patch release which fixes a memory leak when decompressing. For more info, you can have a look at the release notes in: github.com/Blosc/blosc2_g… Enjoy!🥳🥳

ANN: python-blosc2 2.5.1 has been released! 🎉. There, we have updated to the latest C-Blosc 2.13.x, which implemented a new filter for truncating the precision of integers, allowing for lossy compression. More info: github.com/Blosc/python-b… Enjoy!

ANN: blosc2-grok 0.2.2 is out! 🥳This is a minor release, where the grok library is automatically initialized when needed instead of having to import the package. For more info, you can have a look at the release notes in: github.com/Blosc/blosc2_g… Enjoy!

ANN: C-Blosc2 2.13.1 is out!🥳 This is a patch release for fixing a bug regarding the included files in public headers. For more info, see the release notes: github.com/Blosc/c-blosc2… Enjoy data!📷

ANN: C-Blosc2 2.13.0 is out!🎇🎆 This is a minor release with several improvements like the optimization of the zstd codec or the creation of a new filter for integers truncation. For more info, see the release notes: github.com/Blosc/c-blosc2… Enjoy data!📷😄

blosc2-grok 0.2.1 is out! 🥳This is a minor release, where the default parameters are automatically changed to the defaults in blosc2-grok without having to call `set_params_defaults`. For more info, you can have a look at the release notes in: github.com/Blosc/blosc2_g… Enjoy! 😁

ANN: C-Blosc2 2.12.0 is out!🥳🥳 This is a minor release with several improvements like the registration of the dynamically loaded codec blosc2_grok. For more info, see the release notes: github.com/Blosc/c-blosc2… Enjoy data!😄

Btune 1.1.0 is out!🥳🥳 This is a minor release where added some performance optimizations when creating many arrays instead of one array with many chunks (see README.md). For more info see the release notes: github.com/Blosc/blosc2_b… Enjoy!😄

Recent backport of the upstream #bitshuffle filter, with support for #AVX512, into Blosc2 is providing good vibes on expected performance: up to 20% faster on Zen4 architecture 🚀 More info 👉 github.com/Blosc/c-blosc2…

@FangQ @MATLAB @GnuOctave @FrancescAlted Great to hear! BTW, fortunately Blosc2 is not the work of only @FrancescAlted, but all the @Blosc2 team (for a list of the persons involved, see e.g. blosc.org/docs/Exploring…) 😀

Excited to announce #ZMat 1.0 RC1 - a fast in-memory data compression toolbox for @MATLAB & @GNUOctave A big update is the support of high-speed @Blosc2 metacompressor (by @FrancescAlted) and #ZStd Support zlib/gzip/lz4/lzma/blosc2/zstd with a 600kb mex github.com/NeuroJSON/zmat…