no code implementations • 24 May 2024 • William L. Tong, Cengiz Pehlevan
We further show that MLPs outperform Transformers on a subset of ICL tasks designed to test relational reasoning.
no code implementations • 1 Mar 2022 • Jacob A. Zavatone-Veth, William L. Tong, Cengiz Pehlevan
Moreover, we show that the leading-order correction to the kernel-limit learning curve cannot distinguish between random feature models and deep networks in which all layers are trained.