Search Results for author:
Found 9 papers, 4 papers with code
Transformers, parallel computation, and logarithmic depth
We show that a constant number of self-attention layers can efficiently simulate, and be simulated by, a constant number of communication rounds of Massively Parallel Computation.
Learning Single-Index Models with Shallow Neural Networks
Single-index models are a class of functions given by an unknown univariate ``link'' function applied to an unknown one-dimensional projection of the input.
On Scrambling Phenomena for Randomly Initialized Recurrent Networks
Recurrent Neural Networks (RNNs) frequently exhibit complicated dynamics, and their sensitivity to the initialization process often renders them notoriously hard to train.
Intrinsic dimensionality and generalization properties of the $\mathcal{R}$-norm inductive bias
We study the structural and statistical properties of $\mathcal{R}$-norm minimizing interpolants of datasets labeled by specific target functions.
Near-Optimal Statistical Query Lower Bounds for Agnostically Learning Intersections of Halfspaces with Gaussian Marginals
This lower bound is essentially best possible since an SQ algorithm of Klivans et al. (2008) agnostically learns this class to any constant excess error using $n^{O(\log k)}$ queries of tolerance $n^{-O(\log k)}$.
Expressivity of Neural Networks via Chaotic Itineraries beyond Sharkovsky's Theorem
Given a target function $f$, how large must a neural network be in order to approximate $f$?
Support vector machines and linear regression coincide with very high-dimensional features
The support vector machine (SVM) and minimum Euclidean norm least squares regression are two fundamentally different approaches to fitting linear models, but they have recently been connected in models for very high-dimensional data through a phenomenon of support vector proliferation, where every training example used to fit an SVM becomes a support vector.
On the Approximation Power of Two-Layer Networks of Random ReLUs
This paper considers the following question: how well can depth-two ReLU networks with randomly initialized bottom-level weights represent smooth functions?
Uniform Convergence Bounds for Codec Selection
We frame the problem of selecting an optimal audio encoding scheme as a supervised learning task.
