1 code implementation • 17 Feb 2024 • Kaixiang Zheng, En-hui Yang
As a technique to bridge logit matching and probability distribution matching, temperature scaling plays a pivotal role in knowledge distillation (KD).
Ranked #12 on Knowledge Distillation on ImageNet
1 code implementation • 16 Jan 2024 • Linfeng Ye, Shayan Mohajer Hamidi, Renhao Tan, En-hui Yang
To improve this estimate for KD, in this paper we introduce the concept of conditional mutual information (CMI) into the estimation of BCPD and propose a novel estimator called the maximum CMI (MCMI) method.
no code implementations • 10 Jan 2024 • Shayan Mohajer Hamidi, En-hui Yang
AdaFed adaptively tunes this common direction based on the values of local gradients and loss functions.
no code implementations • 17 Sep 2023 • En-hui Yang, Shayan Mohajer Hamidi, Linfeng Ye, Renhao Tan, Beverly Yang
The concepts of conditional mutual information (CMI) and normalized conditional mutual information (NCMI) are introduced to measure the concentration and separation performance of a classification deep neural network (DNN) in the output probability distribution space of the DNN, where CMI and the ratio between CMI and NCMI represent the intra-class concentration and inter-class separation of the DNN, respectively.
no code implementations • 19 Feb 2023 • Hossam Amer, Sepideh Shaterian, En-hui Yang
Experimental results show that in comparison with JPEG at the same CR, Deep Selector-JPEG achieves better Rate-Accuracy performance over the ImageNet validation set for all tested DNN classifiers with gains in classification accuracy between 0. 2% and 1% at the same CRs while satisfying HV constraints.
no code implementations • 16 Apr 2021 • Hossam Amer, Ahmed H. Salamah, Ahmad Sajedi, En-hui Yang
Our offline selections yield CNN inference time savings up to 9% and CR up to 10x.
2 code implementations • ECCV 2020 • Jiawang Bai, Bin Chen, Yiming Li, Dongxian Wu, Weiwei Guo, Shu-Tao Xia, En-hui Yang
In this paper, we propose a novel method, dubbed deep hashing targeted attack (DHTA), to study the targeted attack on such retrieval.