SLIDE : In Defense of Smart Algorithms over Hardware Acceleration for Large-Scale Deep Learning Systems
A seminal paper that proposes SLIDE (Sub-LInear Deep learning Engine). It uniquely blends smart randomized algorithms, with multi-core parallelism and workload optimization. Using just a CPU, SLIDE drastically reduces the computations during both training and inference, outperforming an optimized implementation of Tensorflow on the best available GPU.
Published at MLSys 2020