no code implementations • 23 Jan 2021 • Shuhang Wang, Vivek Kumar Singh, Alex Benjamin, Mercy Asiedu, Elham Yousef Kalafi, Eugene Cheah, Viksit Kumar, Anthony Samir
The salient features of our algorithm include: 1)no need for original training data or generative networks, 2) knowledge transfer between different architectures, 3) ease of implementation for downstream tasks by using the downstream task dataset as the transferal dataset, 4) knowledge transfer of an ensemble of models, trained independently, into one student model.
no code implementations • 1 Jan 2021 • Shuhang Wang, Eugene Cheah, Elham Yousef Kalafi, Mercy Asiedu, Alex Benjamin, Vivek Kumar Singh, Ge Zhang, Viksit Kumar, Anthony Edward Samir
Transfer learning often employs all or part of the weights of a pre-trained net-work to the problem at hand; this limits the flexibility of new neural architectures.