One weird trick for parallelizing convolutional neural networks

A new way to parallelize the training of convolutional neural networks across multiple GPUs is presented, which scales significantly better than all alternatives when applied to modern convolutionAL neural networks.

Abstract

I present a new way to parallelize the training of convolutional neural networks across multiple GPUs. The method scales significantly better than all alternatives when applied to modern convolutional neural networks.