1 code implementation • 19 Dec 2023 • Yulai Cong, Sijia Li
Mixture models serve as one fundamental tool with versatile applications.
no code implementations • 8 Jul 2022 • Yulai Cong, Miaoyun Zhao
Recent advances in big/foundation models reveal a promising path for deep learning, where the roadmap steadily moves from big data to big models to (the newly-introduced) big learning.
no code implementations • 13 Jul 2020 • Miaoyun Zhao, Yulai Cong, Shuyang Dai, Lawrence Carin
Maximum likelihood (ML) and adversarial learning are two popular approaches for training generative models, and from many perspectives these techniques are complementary.
1 code implementation • 16 Jun 2020 • Yulai Cong, Miaoyun Zhao, Jianqiao Li, Junya Chen, Lawrence Carin
An unbiased low-variance gradient estimator, termed GO gradient, was proposed recently for expectation-based objectives $\mathbb{E}_{q_{\boldsymbol{\gamma}}(\boldsymbol{y})} [f(\boldsymbol{y})]$, where the random variable (RV) $\boldsymbol{y}$ may be drawn from a stochastic computation graph with continuous (non-reparameterizable) internal nodes and continuous/discrete leaves.
no code implementations • 15 Jun 2020 • Hao Zhang, Bo Chen, Yulai Cong, Dandan Guo, Hongwei Liu, Mingyuan Zhou
Given a posterior sample of the global parameters, in order to efficiently infer the local latent representations of a document under DATM across all stochastic layers, we propose a Weibull upward-downward variational encoder that deterministically propagates information upward via a deep neural network, followed by a Weibull distribution based stochastic downward generative model.
1 code implementation • NeurIPS 2020 • Yulai Cong, Miaoyun Zhao, Jianqiao Li, Sijia Wang, Lawrence Carin
As a fundamental issue in lifelong learning, catastrophic forgetting is directly caused by inaccessible historical data; accordingly, if the data (information) were memorized perfectly, no forgetting should be expected.
1 code implementation • ICML 2020 • Miaoyun Zhao, Yulai Cong, Lawrence Carin
Demonstrated by natural-image generation, we reveal that low-level filters (those close to observations) of both the generator and discriminator of pretrained GANs can be transferred to facilitate generation in a perceptually-distinct target domain with limited training data.
1 code implementation • ICLR 2019 • Yulai Cong, Miaoyun Zhao, Ke Bai, Lawrence Carin
Within many machine learning algorithms, a fundamental problem concerns efficient calculation of an unbiased gradient wrt parameters $\gammav$ for expectation-based objectives $\Ebb_{q_{\gammav} (\yv)} [f(\yv)]$.
no code implementations • ICML 2017 • Yulai Cong, Bo Chen, Hongwei Liu, Mingyuan Zhou
It is challenging to develop stochastic gradient based scalable inference for deep discrete latent variable models (LVMs), due to the difficulties in not only computing the gradients, but also adapting the step sizes to different latent factors and hidden layers.
no code implementations • 9 Dec 2015 • Mingyuan Zhou, Yulai Cong, Bo Chen
To infer multilayer deep representations of high-dimensional discrete and nonnegative real vectors, we propose an augmentable gamma belief network (GBN) that factorizes each of its hidden layers into the product of a sparse connection weight matrix and the nonnegative real hidden units of the next layer.
no code implementations • NeurIPS 2015 • Mingyuan Zhou, Yulai Cong, Bo Chen
Example results on text analysis illustrate interesting relationships between the width of the first layer and the inferred network structure, and demonstrate that the PGBN, whose hidden units are imposed with correlated gamma priors, can add more layers to increase its performance gains over Poisson factor analysis, given the same limit on the width of the first layer.