
We just open sourced our ActNN library for memory efficient training. It reduces the training memory footprint by compressing the saved activations to 2 bits. It's only a few lines of code in PyTorch, try it!
github.com/ucbrise/actnn
arxiv.org/abs/2104.14129

English