collapse

A few updates: (1) new fastverse domain at fastverse.org (2) the collapse (kit) repos moved to github.com/fastverse/coll… (/kit) and site to fastverse.org/collapse (/kit) (3) A group of maintainers has been given access (4) collapse has a DeepWiki deepwiki.com/fastverse/coll…

Recording of my talk on {collapse} and the {fastverse} at the Bank of Portugal‘s workshop on „Speeding up Empirical Research: Tools and Techniques for Fast Computing“ in December is now online: youtu.be/qO5dHIPsfK8?si… #Rstats #DataScience

Updated windows benchmarks for in-memory database-like operations by Adrian Antico show that {collapse} still leads on lagging and casting benchmarks (not covered in DuckDB benchmarks) and remains overall very competitive: #benmark-results" target="_blank" rel="nofollow noopener">github.com/AdrianAntico/B… #Rstats #DataScience

I've released a new package {flownet} for efficient transport modeling and graph manipulation: sebkrantz.github.io/flownet/. It builds on 7 {fastverse} libraries, most notably {collapse}, from which it imports 60 functions. Thus, another great learning resource for developers #rstats

I'm excited to share the release and rOpenSci publication of dfms 1.0 (docs.ropensci.org/dfms), a high-performance, {collapse} and {RcppArmadillo}-based, and feature-rich, implementation of Dynamics Factor Models in R. More in the release blog post at: sebkrantz.github.io/Rblog/2026/01/…

{collapse} 2.1.0 is out! It introduces a new fslice() function (sebkrantz.github.io/collapse/refer…), a new theory-consistent weighted quantile algorithm (sebkrantz.github.io/collapse/refer…) with excellent properties. And some convenience features such as join requirements: #rstats #DataScience

@JosiahParry It’s a never ending hackathlon, unfortunately. But still good to have rigorous C/C++ checks done for free on many platforms. Creates more robust software.

The {collapse} arXiv paper has just been updated - following extensive revision: arxiv.org/abs/2403.05038. I believe it is a great resource for anyone doing scientific computing with #rstats.

There is now a #fastverse benchmark wiki (github.com/fastverse/fast…) where users can freely contribute benchmarks. If you have benchmarks involving {fastverse} packages ({collapse}, {data.table}, etc., including extensions) please contribute them (takes 1 min) #rstats #DataScience

I just improved the vignette a bit further, adding some detailed benchmarks and a section on Global Options. I needed to correct myself: it is not true that {collapse} global options should never be invoked in packages - they just need to be reversed like #rstats global options.

It's nice to see an increasing number of #rstats packages using {collapse}. A developer focused vignette was long planned and now it is here - with modest advice on writing efficient R package code in general and using {collapse} in particular: sebkrantz.github.io/collapse/artic…

{collapse} is now also on BlueSky (bsky.app/profile/rcolla…) and I am also there (bsky.app/profile/sebkra…) [and on Mastodon: @sebkrantz" target="_blank" rel="nofollow noopener">fosstodon.org/@sebkrantz]. I will repost {collapse} posts but also share about research/data. This X account will remain active. #rstats

@MislavSagovac @r_data_table Of course not. But {collapse} can do many things, particularly complex statistical things, that cannot be done with {data.table} or are more difficult to do or do efficiently with {data.table} - as shown in this post and further in the docs/resources: sebkrantz.github.io/collapse/artic…

@r_data_table Do we need collapse if already use data.table for data wrangling?

Check out the latest package to be granted the Seal of Approval: {collapse} by Sebastian Krantz! {collapse} is a partner package, that implements various data transformation and statistical analysis tasks using ultra fast C/C++ implementations. rdatatable-community.github.io/The-Raft/posts…

{collapse} v2.0.15, with fast aggregation pivots, has just reached CRAN. A minor but neat feature worth pointing out in this release is enhanced join verbosity. In addition to the join success rates, the join relationship is now determined and reported - at no extra cost #rstats

@statquant @JosiahParry Agreed, it’s not more apples to apples, but equally valid. DuckDB benchmarks max out all frameworks on a large linux cloud server. This one compares performance on a local windows system through IDE‘s and is thus closer to many users. For the 100M data they are quite similar…

@collapse_R @JosiahParry Why is it more apple to apple than other benchmarks? Always surprised by benchmarks, it’s more difficult to write benchmarks than write the library! In this family I recommend the kdb/shakti bench where they say they are 100x faster than polars: shakti.com (about)

New independent benchmark by Adrian Antico: github.com/AdrianAntico/B… Setup: - large local Windows machine - real data - broad range of tasks - scripts executed inside Rstudio and VScode -> shows that {collapse} is an absolute top performer in this setting #rstats #DataScience

@Yann_the3rd @JosiahParry Thanks, but I doubt that {collapse} is buggy. Many people are using it and most issues I get are feature requests. It simply does not support tidyselect syntax. across() inside fmutate() works fine. Please carefully read the docs and report any issues that you find on GitHub.

@JosiahParry No, not at the moment, but it uses pointers to access their contents.