Nir Shavit

@NaveenGRao @jefrankle @Muhtasham9 @SquashBionic @MosaicML @neuralmagic Great Naveen. Help people train on GPU to get sparsity/quant in model and then NM deep-sparse engine will deliver the 10x speedup on CPUs. You don't need to do anything that is not training, just effective sparsity in models which @jefrankle knows a thing or two about ...

@nir_shavit @jefrankle @Muhtasham9 @SquashBionic @MosaicML @neuralmagic Our aim from the start was to be platform independent. We use Nvidia GPUs simply because that is the most cost efficient way to train currently. Perf/$ or learning/$ is all we care about. Also, we are only looking at training atm, not inference yet

Would you rather invest $200m+ into a new computing arch and get maybe 2x perf, or <$5m and get 7x with better algos? We @MosaicML did just that! We released Mosaic ResNet and it achieves SOTA perf in just 27min on standard HW. You read that right👇mosaicml.com/blog/mosaic-re…

@jefrankle @Muhtasham9 @NaveenGRao @SquashBionic @MosaicML @neuralmagic Isn't the $ savings mosaicml.com can deliver in inf cost per $ training via your platform a great measure also? if you effectively deliver users a sparse-quantized model that is 10x faster isn't that a worthy measure to consider?

@nir_shavit @Muhtasham9 @NaveenGRao @SquashBionic @MosaicML @neuralmagic I'd be eager to find out which of our speedup methods work best on CPU and where we land in the perf/$ axes in mosaicml.com/explorer

@jefrankle @Muhtasham9 @NaveenGRao @SquashBionic @MosaicML @neuralmagic Jonathan, could be interesting to think about the target hardware (CPU) independently of the one used for training. If you guys work on training (on CPUs or GPUs) sparse quantized models we will deliver GPU latencies on CPUs with them...

@Muhtasham9 @NaveenGRao @SquashBionic @MosaicML @neuralmagic We will happily train on any hardware if (a) software support is there and (b) it provides greater efficiency.