no code implementations • 19 Apr 2023 • Jian He, Chenxi Yang, Zhaoyuan He, Ghufran Baig, Lili Qiu
Based on this observation, we first design a novel scheduling algorithm to exploit the batching benefits of all requests that run the same DNN.
Scheduling