Deep Learning is Not So Mysterious or Different

The paper argues that the phenomena observed in deep learning, such as benign overfitting and double descent, are not unique to neural networks and can be understood using existing generalization frameworks. It introduces the concept of soft inductive biases as a unifying principle, where a...