Search Results for author:
Found 10 papers, 7 papers with code
Flexible inference in heterogeneous and attributed multilayer networks
In this paper, we develop a probabilistic generative model to perform inference in multilayer networks with arbitrary types of information.
The Local Interaction Basis: Identifying Computationally-Relevant and Sparsely Interacting Features in Neural Networks
We present a novel interpretability method that aims to overcome this limitation by transforming the activations of the network into a new basis - the Local Interaction Basis (LIB).
Using Degeneracy in the Loss Landscape for Mechanistic Interpretability
We propose that if we can represent a neural network in a way that is invariant to reparameterizations that exploit the degeneracies, then this representation is likely to be more interpretable, and we provide some evidence that such a representation is likely to have sparser interactions.
Black-Box Access is Insufficient for Rigorous AI Audits
External audits of AI systems are increasingly recognized as a key mechanism for AI governance.
Large Language Models can Strategically Deceive their Users when Put Under Pressure
We demonstrate a situation in which Large Language Models, trained to be helpful, harmless, and honest, can display misaligned behavior and strategically deceive their users about this behavior without being instructed to do so.
Will we run out of data? Limits of LLM scaling based on human-generated data
We investigate the potential constraints on LLM scaling posed by the availability of public human-generated text data.
Machine Learning Model Sizes and the Parameter Gap
From 1950 to 2018, model size in language models increased steadily by seven orders of magnitude.
Compute Trends Across Three Eras of Machine Learning
Since the advent of Deep Learning in the early 2010s, the scaling of training compute has accelerated, doubling approximately every 6 months.
Laplace Matching for fast Approximate Inference in Latent Gaussian Models
The method can be thought of as a pre-processing step which can be implemented in <5 lines of code and runs in less than a second.
Fast Predictive Uncertainty for Classification with Bayesian Deep Networks
We reconsider old work (Laplace Bridge) to construct a Dirichlet approximation of this softmax output distribution, which yields an analytic map between Gaussian distributions in logit space and Dirichlet distributions (the conjugate prior to the Categorical distribution) in the output space.
