no code implementations • NeurIPS 2023 • Nikhil Vyas, Alexander Atanasov, Blake Bordelon, Depen Morwani, Sabarish Sainathan, Cengiz Pehlevan
We call this the bias of narrower width.
1 code implementation • 23 Dec 2022 • Alexander Atanasov, Blake Bordelon, Sabarish Sainathan, Cengiz Pehlevan
For small training set sizes $P$, the generalization error of wide neural networks is well-approximated by the error of an infinite width neural network (NN), either in the kernel or mean-field/feature-learning regime.