On some similarities and differences between deep neural networks and kernel learning machines

Abstract

This paper presents a thorough computational comparison of the predic- tive performances of deep neural networks and kernel learning machines. The work featured here successfully establishes that on both real-life datasets and artificially simulated ones, kernel learning machines tend to be just as good as deep neural net- works, and quite often outperform them predictively. It turns out from the findings of this paper that while deep neural networks might have worked well on tasks for which millions of observations are available, kernel learning machines just happen to be predictively better on a wide variety of tasks with the kind of sample size that one should realistically expect to have in practice.