1 code implementation • 5 Feb 2024 • Yehui Tang, Fangcheng Liu, Yunsheng Ni, Yuchuan Tian, Zheyuan Bai, Yi-Qi Hu, Sichao Liu, Shangling Jui, Kai Han, Yunhe Wang
Several design formulas are empirically proved especially effective for tiny language models, including tokenizer compression, architecture tweaking, parameter inheritance and multiple-round training.
no code implementations • 23 Dec 2023 • Leo Maxime Brunswic, Yinchuan Li, Yushun Xu, Shangling Jui, Lizhuang Ma
GFlowNets is a novel flow-based method for learning a stochastic policy to generate objects via a sequence of actions and with probability proportional to a given positive reward.
no code implementations • 23 Dec 2023 • ZiCheng Zhang, HaoNing Wu, Zhongpeng Ji, Chunyi Li, Erli Zhang, Wei Sun, Xiaohong Liu, Xiongkuo Min, Fengyu Sun, Shangling Jui, Weisi Lin, Guangtao Zhai
Recent advancements in Multi-modality Large Language Models (MLLMs) have demonstrated remarkable capabilities in complex high-level vision tasks.
1 code implementation • 9 Dec 2023 • Zijian Chen, Wei Sun, HaoNing Wu, ZiCheng Zhang, Jun Jia, Zhongpeng Ji, Fengyu Sun, Shangling Jui, Xiongkuo Min, Guangtao Zhai, Wenjun Zhang
In this paper, we take the first step to benchmark and assess the visual naturalness of AI-generated images.
no code implementations • 1 Sep 2023 • Shiqi Yang, Yaxing Wang, Joost Van de Weijer, Luis Herranz, Shangling Jui, Jian Yang
We capture this intrinsic structure by defining local affinity of the target data, and encourage label consistency among data with high local affinity.
1 code implementation • 15 Aug 2023 • BoYu Chen, Hanxuan Chen, Jiao He, Fengyu Sun, Shangling Jui
We present a simple yet novel parameterized form of linear mapping to achieves remarkable network compression performance: a pseudo SVD called Ternary SVD (TSVD).
1 code implementation • 5 Mar 2023 • Alexander Detkov, Mohammad Salameh, Muhammad Fetrat Qharabagh, Jialin Zhang, Wei Lui, Shangling Jui, Di Niu
Reparameterization aims to improve the generalization of deep neural networks by transforming convolutional layers into equivalent multi-branched structures during training.
no code implementations • 21 Feb 2023 • Fred X. Han, Keith G. Mills, Fabian Chudak, Parsa Riahi, Mohammad Salameh, Jialin Zhang, Wei Lu, Shangling Jui, Di Niu
In this paper, we propose a general-purpose neural predictor for NAS that can transfer across search spaces, by representing any given candidate Convolutional Neural Network (CNN) with a Computation Graph (CG) that consists of primitive operators.
1 code implementation • 30 Nov 2022 • Keith G. Mills, Di Niu, Mohammad Salameh, Weichen Qiu, Fred X. Han, Puyuan Liu, Jialin Zhang, Wei Lu, Shangling Jui
Evaluating neural network performance is critical to deep neural network design but a costly procedure.
1 code implementation • 30 Nov 2022 • Keith G. Mills, Fred X. Han, Jialin Zhang, Fabian Chudak, Ali Safari Mamaghani, Mohammad Salameh, Wei Lu, Shangling Jui, Di Niu
In this paper, we propose GENNAPE, a Generalized Neural Architecture Performance Estimator, which is pretrained on open neural architecture benchmarks, and aims to generalize to completely unseen architectures through combined innovations in network representation, contrastive pretraining, and fuzzy clustering-based predictor ensemble.
1 code implementation • 4 Oct 2022 • Kai Wang, Chenshen Wu, Andy Bagdanov, Xialei Liu, Shiqi Yang, Shangling Jui, Joost Van de Weijer
Lifelong object re-identification incrementally learns from a stream of re-identification tasks.
1 code implementation • 7 Jun 2022 • Shiqi Yang, Yaxing Wang, Kai Wang, Shangling Jui, Joost Van de Weijer
In this paper, we investigate Source-free Open-partial Domain Adaptation (SF-OPDA), which addresses the situation where there exist both domain and category shifts between source and target domains.
1 code implementation • ICLR 2022 • Shengyao Lu, Bang Liu, Keith G. Mills, Shangling Jui, Di Niu
Systematicity, i. e., the ability to recombine known parts and rules to form new sequences while reasoning over relational data, is critical to machine intelligence.
1 code implementation • 9 May 2022 • Shiqi Yang, Yaxing Wang, Kai Wang, Shangling Jui, Joost Van de Weijer
Treating SFDA as an unsupervised clustering problem and following the intuition that local neighbors in feature space should have more similar predictions than other features, we propose to optimize an objective of prediction consistency.
1 code implementation • 9 Nov 2021 • Kai Wang, Xialei Liu, Andy Bagdanov, Luis Herranz, Shangling Jui, Joost Van de Weijer
We propose an approach to IML, which we call Episodic Replay Distillation (ERD), that mixes classes from the current task with class exemplars from previous tasks when sampling episodes for meta-learning.
no code implementations • NeurIPS 2021 • Ke Sun, Yafei Wang, Yi Liu, Yingnan Zhao, Bo Pan, Shangling Jui, Bei Jiang, Linglong Kong
Anderson mixing has been heuristically applied to reinforcement learning (RL) algorithms for accelerating convergence and improving the sampling efficiency of deep RL.
2 code implementations • NeurIPS 2021 • Shiqi Yang, Yaxing Wang, Joost Van de Weijer, Luis Herranz, Shangling Jui
In this paper, we address the challenging source-free domain adaptation (SFDA) problem, where the source pretrained model is adapted to the target domain in the absence of source data.
Ranked #3 on Source-Free Domain Adaptation on VisDA-2017
no code implementations • 29 Sep 2021 • Ke Sun, Yi Liu, Yingnan Zhao, Hengshuai Yao, Shangling Jui, Linglong Kong
In real scenarios, state observations that an agent observes may contain measurement errors or adversarial noises, misleading the agent to take suboptimal actions or even collapse while training.
Distributional Reinforcement Learning reinforcement-learning +1
no code implementations • 29 Sep 2021 • Fred X. Han, Fabian Chudak, Keith G Mills, Mohammad Salameh, Parsa Riahi, Jialin Zhang, Wei Lu, Shangling Jui, Di Niu
Understanding and modelling the performance of neural architectures is key to Neural Architecture Search (NAS).
no code implementations • ICLR 2022 • Yaxing Wang, Joost Van de Weijer, Lu Yu, Shangling Jui
Therefore, we investigate knowledge distillation to transfer knowledge from a high-quality unconditioned generative model (e. g., StyleGAN) to a conditioned synthetic image generation modules in a variety of systems.
no code implementations • 25 Sep 2021 • Keith G. Mills, Fred X. Han, Mohammad Salameh, SEYED SAEED CHANGIZ REZAEI, Linglong Kong, Wei Lu, Shuo Lian, Shangling Jui, Di Niu
In this paper, we propose L$^{2}$NAS, which learns to intelligently optimize and update architecture hyperparameters via an actor neural network based on the distribution of high-performing architectures in the search history.
1 code implementation • 25 Sep 2021 • Keith G. Mills, Fred X. Han, Jialin Zhang, SEYED SAEED CHANGIZ REZAEI, Fabian Chudak, Wei Lu, Shuo Lian, Shangling Jui, Di Niu
Neural architecture search automates neural network design and has achieved state-of-the-art results in many deep learning applications.
1 code implementation • 17 Sep 2021 • Ke Sun, Yingnan Zhao, Shangling Jui, Linglong Kong
In real scenarios, state observations that an agent observes may contain measurement errors or adversarial noises, misleading the agent to take suboptimal actions or even collapse while training.
1 code implementation • ICCV 2021 • Shiqi Yang, Yaxing Wang, Joost Van de Weijer, Luis Herranz, Shangling Jui
In this paper, we propose a new domain adaptation paradigm called Generalized Source-free Domain Adaptation (G-SFDA), where the learned model needs to perform well on both the target and source domains, with only access to current unlabeled target data during adaptation.
Ranked #2 on Source-Free Domain Adaptation on VisDA-2017
7 code implementations • CVPR 2021 • Yixing Xu, Yunhe Wang, Kai Han, Yehui Tang, Shangling Jui, Chunjing Xu, Chang Xu
An effective and efficient architecture performance evaluation scheme is essential for the success of Neural Architecture Search (NAS).
no code implementations • 19 May 2021 • SEYED SAEED CHANGIZ REZAEI, Fred X. Han, Di Niu, Mohammad Salameh, Keith Mills, Shuo Lian, Wei Lu, Shangling Jui
Despite the empirical success of neural architecture search (NAS) in deep learning applications, the optimality, reproducibility and cost of NAS schemes remain hard to assess.
1 code implementation • 28 Apr 2021 • Yaxing Wang, Abel Gonzalez-Garcia, Chenshen Wu, Luis Herranz, Fahad Shahbaz Khan, Shangling Jui, Joost Van de Weijer
Therefore, we propose a novel knowledge transfer method for generative models based on mining the knowledge that is most beneficial to a specific target domain, either from a single or multiple pretrained GANs.
no code implementations • 1 Jan 2021 • SEYED SAEED CHANGIZ REZAEI, Fred X. Han, Di Niu, Mohammad Salameh, Keith G Mills, Shangling Jui
Despite the empirical success of neural architecture search (NAS) algorithms in deep learning applications, the optimality, reproducibility and cost of NAS schemes remain hard to be assessed.
2 code implementations • 23 Oct 2020 • Shiqi Yang, Yaxing Wang, Joost Van de Weijer, Luis Herranz, Shangling Jui
When adapting to the target domain, the additional classifier initialized from source classifier is expected to find misclassified features.
Source-Free Domain Adaptation Unsupervised Domain Adaptation
no code implementations • 1 Sep 2020 • Tong Mo, Yakun Yu, Mohammad Salameh, Di Niu, Shangling Jui
Deep neural networks have recently become a popular solution to keyword spotting systems, which enable the control of smart devices via voice.
Ranked #1 on Keyword Spotting on Google Speech Commands (Google Speech Commands V1 6 metric)
1 code implementation • 20 Apr 2020 • Xialei Liu, Chenshen Wu, Mikel Menta, Luis Herranz, Bogdan Raducanu, Andrew D. Bagdanov, Shangling Jui, Joost Van de Weijer
To prevent forgetting, we combine generative feature replay in the classifier with feature distillation in the feature extractor.
2 code implementations • CVPR 2020 • Lu Yu, Bartłomiej Twardowski, Xialei Liu, Luis Herranz, Kai Wang, Yongmei Cheng, Shangling Jui, Joost Van de Weijer
The vast majority of methods have studied this scenario for classification networks, where for each new task the classification layer of the network must be augmented with additional weights to make room for the newly added classes.
4 code implementations • 30 Sep 2019 • Yixing Xu, Yunhe Wang, Kai Han, Yehui Tang, Shangling Jui, Chunjing Xu, Chang Xu
An effective and efficient architecture performance evaluation scheme is essential for the success of Neural Architecture Search (NAS).
no code implementations • 25 Sep 2019 • Chao GAO, Martin Mueller, Ryan Hayward, Hengshuai Yao, Shangling Jui
A three-head network architecture has been recently proposed that can learn a third action-value head on a fixed dataset the same as for two-head net.
no code implementations • 26 Mar 2019 • Ramchalam Kinattinkara Ramakrishnan, Shangling Jui, Vahid Patrovi Nia
We provide an exhaustive search of deep neural network architectures and obtain a pareto front of Color Peak Signal to Noise Ratio (CPSNR) as the performance criterion versus the number of parameters as the model complexity that beats the state-of-the-art.
no code implementations • 20 Mar 2019 • Nazmus Sakib, Hengshuai Yao, Hong Zhang, Shangling Jui
In this paper, we use reinforcement learning for safety driving in adversary settings.