no code implementations • 2 Mar 2024 • Shivchander Sudalairaj, Abhishek Bhandwaldar, Aldo Pareja, Kai Xu, David D. Cox, Akash Srivastava
This work introduces LAB (Large-scale Alignment for chatBots), a novel methodology designed to overcome the scalability challenges in the instruction-tuning phase of large language model (LLM) training.
1 code implementation • 29 Feb 2024 • Zhang-Wei Hong, Idan Shenfeld, Tsun-Hsuan Wang, Yung-Sung Chuang, Aldo Pareja, James Glass, Akash Srivastava, Pulkit Agrawal
To probe when an LLM generates unwanted content, the current paradigm is to recruit a \textit{red team} of human testers to design input prompts (i. e., test cases) that elicit undesirable responses from LLMs.
no code implementations • 28 Feb 2024 • Samuel J. K. Chin, Matthias Winkenbach, Akash Srivastava
In this paper, we present the Foundation Model for the Montreal Capacitated Vehicle Routing Problem (FM-MCVRP), a novel Deep Learning (DL) model that approximates high-quality solutions to a variant of the Capacitated Vehicle Routing Problem (CVRP) that characterizes many real-world applications.
no code implementations • 15 Oct 2023 • Haoyuan Sun, Navid Azizan, Akash Srivastava, Hao Wang
When machine learning models are trained on synthetic data and then deployed on real data, there is often a performance drop due to the distribution shift between synthetic and real data.
1 code implementation • NeurIPS 2023 • Zhang-Wei Hong, Aviral Kumar, Sathwik Karnik, Abhishek Bhandwaldar, Akash Srivastava, Joni Pajarinen, Romain Laroche, Abhishek Gupta, Pulkit Agrawal
We argue this is due to an assumption made by current offline RL algorithms of staying close to the trajectories in the dataset.
1 code implementation • NeurIPS 2023 • JiaQi Zhang, Chandler Squires, Kristjan Greenewald, Akash Srivastava, Karthikeyan Shanmugam, Caroline Uhler
Causal disentanglement aims to uncover a representation of data using latent variables that are interrelated through a causal model.
no code implementations • 27 Jun 2023 • Giorgio Giannone, Lyle Regenwetter, Akash Srivastava, Dan Gutfreund, Faez Ahmed
This is particularly problematic when the generated data must satisfy constraints, for example, to meet product specifications in engineering design or to adhere to the laws of physics in a natural scene.
no code implementations • 11 Jun 2023 • Lazar Valkov, Akash Srivastava, Swarat Chaudhuri, Charles Sutton
Comparing to a wide range of approaches, we show that PICLE is the first modular CL algorithm to achieve perceptual, few-shot and latent transfer while scaling well to large search spaces, outperforming previous state-of-the-art modular CL approaches on long problem sequences.
1 code implementation • 8 Jun 2023 • Ligong Han, Song Wen, Qi Chen, Zhixing Zhang, Kunpeng Song, Mengwei Ren, Ruijiang Gao, Anastasis Stathopoulos, Xiaoxiao He, Yuxiao Chen, Di Liu, Qilong Zhangli, Jindong Jiang, Zhaoyang Xia, Akash Srivastava, Dimitris Metaxas
Null-text inversion (NTI) optimizes null embeddings to align the reconstruction and inversion trajectories with larger CFG scales, enabling real image editing with cross-attention control.
no code implementations • 1 May 2023 • Akash Srivastava, Seungwook Han, Kai Xu, Benjamin Rhodes, Michael U. Gutmann
We show that if these auxiliary densities are constructed such that they overlap with $p$ and $q$, then a multi-class logistic regression allows for estimating $\log p/q$ on the domain of any of the $K+2$ distributions and resolves the distribution shift problems of the current state-of-the-art methods.
no code implementations • 2 Apr 2023 • Ligong Han, Seungwook Han, Shivchander Sudalairaj, Charlotte Loh, Rumen Dangovski, Fei Deng, Pulkit Agrawal, Dimitris Metaxas, Leonid Karlinsky, Tsui-Wei Weng, Akash Srivastava
Recently, several attempts have been made to replace such domain-specific, human-designed transformations with generated views that are learned.
1 code implementation • 4 Mar 2023 • Charlotte Loh, Seungwook Han, Shivchander Sudalairaj, Rumen Dangovski, Kai Xu, Florian Wenzel, Marin Soljacic, Akash Srivastava
In this work, we present Multi-Symmetry Ensembles (MSE), a framework for constructing diverse ensembles by capturing the multiplicity of hypotheses along symmetry axes, which explore the hypothesis space beyond stochastic perturbations of model weights and hyperparameters.
no code implementations • 6 Feb 2023 • Lyle Regenwetter, Akash Srivastava, Dan Gutfreund, Faez Ahmed
This paper doubles as a review and a practical guide to evaluation metrics for deep generative models (DGMs) in engineering design.
no code implementations • 30 Jan 2023 • Max Schrader, Navish Kumar, Nicolas Collignon, Esben Sørig, Soonmyeong Yoon, Akash Srivastava, Kai Xu, Maria Astefanoaei
Light goods vehicles (LGV) used extensively in the last mile of delivery are one of the leading polluters in cities.
no code implementations • 10 Oct 2022 • Charlotte Loh, Rumen Dangovski, Shivchander Sudalairaj, Seungwook Han, Ligong Han, Leonid Karlinsky, Marin Soljacic, Akash Srivastava
State-of-the-art (SOTA) semi-supervised learning (SSL) methods have been highly successful in leveraging a mix of labeled and unlabeled data by combining techniques of consistency regularization and pseudo-labeling.
1 code implementation • 30 Aug 2022 • Amin Heyrani Nobari, Akash Srivastava, Dan Gutfreund, Faez Ahmed
LINKS is made up of various components including 100 million mechanisms, the simulation data for each mechanism, normalized paths generated by each mechanism, a curated set of paths, the code used to generate the data and simulate mechanisms, and a live web demo for interactive design of linkage mechanisms.
no code implementations • NeurIPS 2021 • Kai Xu, Akash Srivastava, Dan Gutfreund, Felix Sosa, Tomer Ullman, Josh Tenenbaum, Charles Sutton
In this paper, we propose a Bayesian-symbolic framework (BSP) for physical reasoning and learning that is close to human-level sample-efficiency and accuracy.
2 code implementations • 28 Oct 2021 • Rumen Dangovski, Li Jing, Charlotte Loh, Seungwook Han, Akash Srivastava, Brian Cheung, Pulkit Agrawal, Marin Soljačić
In state-of-the-art self-supervised learning (SSL) pre-training produces semantically good representations by encouraging them to be invariant under meaningful transformations prescribed from human knowledge.
2 code implementations • NeurIPS 2021 • Cole Hurwitz, Akash Srivastava, Kai Xu, Justin Jude, Matthew G. Perich, Lee E. Miller, Matthias H. Hennig
These approaches, however, are limited in their ability to capture the underlying neural dynamics (e. g. linear) and in their ability to relate the learned dynamics back to the observed behaviour (e. g. no time lag).
no code implementations • 29 Sep 2021 • Akash Srivastava, Seungwook Han, Benjamin Rhodes, Kai Xu, Michael U. Gutmann
As such, estimating density ratios accurately using only samples from $p$ and $q$ is of high significance and has led to a flurry of recent work in this direction.
no code implementations • ICLR 2022 • Rumen Dangovski, Li Jing, Charlotte Loh, Seungwook Han, Akash Srivastava, Brian Cheung, Pulkit Agrawal, Marin Soljacic
In state-of-the-art self-supervised learning (SSL) pre-training produces semantically good representations by encouraging them to be invariant under meaningful transformations prescribed from human knowledge.
no code implementations • 1 Jan 2021 • Seungwook Han, Akash Srivastava, Cole Lincoln Hurwitz, Prasanna Sattigeri, David Daniel Cox
First, we generate images in low-frequency bands by training a sampler in the wavelet domain.
no code implementations • 1 Jan 2021 • Kai Xu, Akash Srivastava, Dan Gutfreund, Felix Sosa, Tomer Ullman, Joshua B. Tenenbaum, Charles Sutton
As such, learning the laws is then reduced to symbolic regression and Bayesian inference methods are used to obtain the distribution of unobserved properties.
no code implementations • 25 Oct 2020 • Akash Srivastava, Yamini Bansal, Yukun Ding, Cole Lincoln Hurwitz, Kai Xu, Bernhard Egger, Prasanna Sattigeri, Joshua B. Tenenbaum, Phuong Le, Arun Prakash R, Nengfeng Zhou, Joel Vaughan, Yaquan Wang, Anwesha Bhattacharyya, Kristjan Greenewald, David D. Cox, Dan Gutfreund
Current autoencoder-based disentangled representation learning methods achieve disentanglement by penalizing the (aggregate) posterior to encourage statistical independence of the latent factors.
no code implementations • 9 Sep 2020 • Seungwook Han, Akash Srivastava, Cole Hurwitz, Prasanna Sattigeri, David D. Cox
First, we generate images in low-frequency bands by training a sampler in the wavelet domain.
no code implementations • 29 Mar 2020 • Robin Hirt, Akash Srivastava, Carlos Berg, Niklas Kühl
As the number of data sets in business networks grows and not every neural net transfer is successful, indicators are needed for its impact on the target performance-its transferability.
no code implementations • ICLR 2020 • Akash Srivastava, Yamini Bansal, Yukun Ding, Bernhard Egger, Prasanna Sattigeri, Josh Tenenbaum, David D. Cox, Dan Gutfreund
In this work, we tackle a slightly more intricate scenario where the observations are generated from a conditional distribution of some known control variate and some latent noise variate.
no code implementations • 19 Nov 2019 • Akash Srivastava, Jessie Rosenberg, Dan Gutfreund, David D. Cox
Then an inference network (encoder)is trained to invert the decoder.
no code implementations • 1 Jun 2019 • Akash Srivastava, Kristjan Greenewald, Farzaneh Mirzazadeh
Well-definedness of f-divergences, however, requires the distributions of the data and model to overlap completely in every time step of training.
1 code implementation • NeurIPS 2019 • Cole L. Hurwitz, Kai Xu, Akash Srivastava, Alessio P. Buccino, Matthias H. Hennig
Determining the positions of neurons in an extracellular recording is useful for investigating functional properties of the underlying neural circuitry.
3 code implementations • ICML 2018 • Mohammad Emtiyaz Khan, Didrik Nielsen, Voot Tangkaratt, Wu Lin, Yarin Gal, Akash Srivastava
Uncertainty computation in deep learning is essential to design robust and reliable systems.
no code implementations • ICLR 2020 • Akash Srivastava, Kai Xu, Michael U. Gutmann, Charles Sutton
In this work, we take their insight of using kernels as fixed adversaries further and present a novel method for training deep generative models that does not involve saddlepoint optimization.
no code implementations • 21 Apr 2018 • Akash Srivastava, Charles Sutton
The Pachinko Allocation Machine (PAM) is a deep topic model that allows representing rich correlation structures among topics by a directed acyclic graph over topics.
2 code implementations • NeurIPS 2018 • Lazar Valkov, Dipak Chaudhari, Akash Srivastava, Charles Sutton, Swarat Chaudhuri
We present a neurosymbolic framework for the lifelong learning of algorithmic tasks that mix perception and procedural reasoning.
1 code implementation • NeurIPS 2017 • Akash Srivastava, Lazar Valkov, Chris Russell, Michael U. Gutmann, Charles Sutton
Deep generative models provide powerful tools for distributions over complicated manifolds, such as those of natural images.
6 code implementations • 4 Mar 2017 • Akash Srivastava, Charles Sutton
A promising approach to address this problem is autoencoding variational Bayes (AEVB), but it has proven diffi- cult to apply to topic models in practice.
Ranked #6 on Topic Models on 20NewsGroups
no code implementations • 19 Jun 2016 • Akash Srivastava, James Zou, Ryan P. Adams, Charles Sutton
A good clustering can help a data analyst to explore and understand a data set, but what constitutes a good clustering may depend on domain-specific and application-specific criteria.
no code implementations • 22 Feb 2016 • Akash Srivastava, James Zou, Charles Sutton
A good clustering can help a data analyst to explore and understand a data set, but what constitutes a good clustering may depend on domain-specific and application-specific criteria.