Search Results for author:
Found 19 papers, 1 papers with code
Probing Biological and Artificial Neural Networks with Task-dependent Neural Manifolds
Recently, growth in our understanding of the computations performed in both biological and artificial neural networks has largely been driven by either low-level mechanistic studies or global normative approaches.
Connecting NTK and NNGP: A Unified Theoretical Framework for Neural Network Learning Dynamics in the Kernel Regime
In this regime, two disparate theoretical frameworks have been used, in which the network's output is described using kernels: one framework is based on the Neural Tangent Kernel (NTK) which assumes linearized gradient descent dynamics, while the Neural Network Gaussian Process (NNGP) kernel assumes a Bayesian framework.
Globally Gated Deep Linear Networks
The rich and diverse behavior of the GGDLNs suggests that they are helpful analytically tractable models of learning single and multiple tasks, in finite-width nonlinear deep networks.
A theory of learning with constrained weight-distribution
Our theory predicts that the reduction in capacity due to the constrained weight-distribution is related to the Wasserstein distance between the imposed distribution and that of the standard normal distribution.
Temporal support vectors for spiking neuronal networks
However, for dynamical systems with event based outputs, such as spiking neural networks and other continuous time threshold crossing systems, this optimality criterion is inapplicable due to the strong temporal correlations in their input and output.
Optimal quadratic binding for relational reasoning in vector symbolic neural architectures
In these processes, two different modalities, such as location and objects, events and their contextual cues, and words and their roles, need to be bound together, but little is known about the underlying neural mechanisms.
Soft-margin classification of object manifolds
A neural population responding to multiple appearances of a single object defines a manifold in the neural response space.
Macroscopic Fluctuations Emerge in Balanced Networks with Incomplete Recurrent Alignment
Finally, we define the alignment matrix as the overlap between left and right-singular vectors of the structured connectivity, and show that the singular values of the alignment matrix determine the amplitude of macroscopic variability, while its singular vectors determine the structure.
Statistical Mechanics of Deep Linear Neural Networks: The Back-Propagating Kernel Renormalization
This procedure allows us to evaluate important network properties, such as its generalization error, the role of network width and depth, the impact of the size of the training set, and the effects of weight regularization and learning stochasticity.
Predicting the Outputs of Finite Networks Trained with Noisy Gradients
A recent line of works studied wide deep neural networks (DNNs) by approximating them as Gaussian Processes (GPs).
A new role for circuit expansion for learning in neural networks
We find that sparse expansion of the input of a student perceptron network both increases its capacity and improves the generalization performance of the network when learning a noisy rule from a teacher perceptron when these expansions are pruned after learning.
Predicting the outputs of finite deep neural networks trained with noisy gradients
A recent line of works studied wide deep neural networks (DNNs) by approximating them as Gaussian Processes (GPs).
Classification and Geometry of General Perceptual Manifolds
The effects of label sparsity on the classification capacity of manifolds are elucidated, revealing a scaling relation between label sparsity and manifold radius.
Learning Data Manifolds with a Cutting Plane Method
We consider the problem of classifying data manifolds where each manifold represents invariances that are parameterized by continuous degrees of freedom.
Balanced Excitation and Inhibition are Required for High-Capacity, Noise-Robust Neuronal Selectivity
To evaluate the impact of both input and output noise, we determine the robustness of single-neuron stimulus selective responses, as well as the robustness of attractor states of networks of neurons performing memory tasks.
Optimal Architectures in a Solvable Model of Deep Networks
Deep neural networks have received a considerable attention due to the success of their training for real world machine learning applications.
Linear Readout of Object Manifolds
Objects are represented in sensory systems by continuous manifolds due to sensitivity of neuronal responses to changes in physical features such as location, orientation, and intensity.
Inferring Stimulus Selectivity from the Spatial Structure of Neural Network Dynamics
How are the spatial patterns of spontaneous and evoked population responses related?
Short-term memory in neuronal networks through dynamical compressed sensing
Prior work, in the case of gaussian input sequences and linear neuronal networks, shows that the duration of memory traces in a network cannot exceed the number of neurons (in units of the neuronal time constant), and that no network can out-perform an equivalent feedforward network.
