1 code implementation • 20 Jan 2024 • Zeyu Liu, Gourav Datta, Anni Li, Peter Anthony Beerel
Moreover, we present a spiking version of this architecture, which introduces the benefit of states within the patch embedding and channel mixer modules while simultaneously reducing the computing complexity.
no code implementations • 12 Dec 2023 • Gourav Datta, Zeyu Liu, James Diffenderfer, Bhavya Kailkhura, Peter A. Beerel
However, advanced ANN-to-SNN conversion approaches demonstrate that for lossless conversion, the number of SNN time steps must equal the number of quantization steps in the ANN activation function.
no code implementations • 28 Nov 2023 • Gourav Datta, Zeyu Liu, Anni Li, Peter A. Beerel
Recently proposed SNN training algorithms have significantly reduced the number of time steps (down to 1) for improved latency and energy efficiency, however, they target only convolutional neural networks (CNN).
no code implementations • 18 Oct 2023 • Sreetama Sarkar, Xinan Ye, Gourav Datta, Peter A. Beerel
Efficient and effective on-line detection and correction of bad pixels can improve yield and increase the expected lifetime of image sensors.
no code implementations • 15 Sep 2023 • Yue Hu, Gourav Datta, Kira Beerel, Peter Beerel
This implies that ML pipelines might not need explicit correction for RS for many object detection applications, but mitigating RS effects in ISP-less ML pipelines that target fine-grained location of the objects may need additional research.
1 code implementation • 6 Aug 2023 • Yue Hu, Xinan Ye, Yifei Liu, Souvik Kundu, Gourav Datta, Srikar Mutnuri, Namo Asavisanu, Nora Ayanian, Konstantinos Psounis, Peter Beerel
This paper presents "FireFly", a synthetic dataset for ember detection created using Unreal Engine 4 (UE4), designed to overcome the current lack of ember-specific training resources.
no code implementations • 6 Apr 2023 • Md Abdullah-Al Kaiser, Gourav Datta, Sreetama Sarkar, Souvik Kundu, Zihan Yin, Manas Garg, Ajey P. Jacob, Peter A. Beerel, Akhilesh R. Jaiswal
The massive amounts of data generated by camera sensors motivate data processing inside pixel arrays, i. e., at the extreme-edge.
no code implementations • 17 Feb 2023 • Shashank Nag, Gourav Datta, Souvik Kundu, Nitin Chandrachoodan, Peter A. Beerel
Vision Transformer models, such as ViT, Swin Transformer, and Transformer-in-Transformer, have recently gained significant traction in computer vision tasks due to their ability to capture the global relation between features which leads to superior performance.
no code implementations • 22 Jan 2023 • Md Abdullah-Al Kaiser, Gourav Datta, Zixu Wang, Ajey P. Jacob, Peter A. Beerel, Akhilesh R. Jaiswal
Edge devices equipped with computer vision must deal with vast amounts of sensory data with limited computing resources.
no code implementations • 21 Dec 2022 • Gourav Datta, Zeyu Liu, Md Abdullah-Al Kaiser, Souvik Kundu, Joe Mathai, Zihan Yin, Ajey P. Jacob, Akhilesh R. Jaiswal, Peter A. Beerel
Although the overhead for the first layer MACs with direct encoding is negligible for deep SNNs and the CV processing is efficient using SNNs, the data transfer between the image sensors and the downstream processing costs significant bandwidth and may dominate the total energy.
no code implementations • 20 Dec 2022 • Gourav Datta, Zeyu Liu, Peter A. Beerel
Spiking Neural networks (SNN) have emerged as an attractive spatio-temporal computing paradigm for a wide range of low-power vision tasks.
no code implementations • 23 Oct 2022 • Gourav Datta, Haoqin Deng, Robert Aviles, Peter A. Beerel
We obtain test accuracy of 94. 75% with only 2 time steps with direct encoding on the GSC dataset with 4. 1x lower energy than an iso-architecture standard LSTM.
no code implementations • 11 Oct 2022 • Gourav Datta, Zeyu Liu, Zihan Yin, Linyu Sun, Akhilesh R. Jaiswal, Peter A. Beerel
However, direct inference on the raw images degrades the test accuracy due to the difference in covariance of the raw images captured by the image sensors compared to the ISP-processed images used for training.
no code implementations • 16 Sep 2022 • Fang Chen, Gourav Datta, Souvik Kundu, Peter Beerel
With the aggressive down-sampling of the activation maps in the initial layers (providing up to 22x reduction in memory consumption), our approach achieves 1. 43% higher test accuracy compared to SOTA techniques with iso-memory footprints.
no code implementations • 28 May 2022 • Gourav Datta, Souvik Kundu, Zihan Yin, Joe Mathai, Zeyu Liu, Zixu Wang, Mulin Tian, Shunlin Lu, Ravi T. Lakkireddy, Andrew Schmidt, Wael Abd-Almageed, Ajey P. Jacob, Akhilesh R. Jaiswal, Peter A. Beerel
The designs also reduce the sensor and total energy (obtained from in-house circuit simulations at Globalfoundries 22nm technology node) per frame by 5. 7x and 1. 14x, respectively.
no code implementations • 11 Mar 2022 • Gourav Datta, Zihan Yin, Ajey Jacob, Akhilesh R. Jaiswal, Peter A. Beerel
Hyperspectral cameras generate a large amount of data due to the presence of hundreds of spectral bands as opposed to only three channels (red, green, and blue) in traditional cameras.
no code implementations • 7 Mar 2022 • Gourav Datta, Souvik Kundu, Zihan Yin, Ravi Teja Lakkireddy, Joe Mathai, Ajey Jacob, Peter A. Beerel, Akhilesh R. Jaiswal
Visual data in such cameras are usually captured in the form of analog voltages by a sensor pixel array, and then converted to the digital domain for subsequent AI processing using analog-to-digital converters (ADC).
no code implementations • 22 Dec 2021 • Gourav Datta, Peter A. Beerel
SOTA training strategies for SNNs involve conversion from a non-spiking deep neural network (DNN).
no code implementations • 26 Jul 2021 • Gourav Datta, Souvik Kundu, Akhilesh R. Jaiswal, Peter A. Beerel
However, the accurate processing of the spectral and spatial correlation between the bands requires the use of energy-expensive 3-D Convolutional Neural Networks (CNNs).
Computational Efficiency Hyperspectral Image Classification +1
no code implementations • 26 Jul 2021 • Gourav Datta, Souvik Kundu, Peter A. Beerel
This paper presents a training framework for low-latency energy-efficient SNNs that uses a hybrid encoding scheme at the input layer in which the analog pixel values of an image are directly applied during the first timestep and a novel variant of spike temporal coding is used during subsequent timesteps.
no code implementations • 16 Jul 2021 • Souvik Kundu, Gourav Datta, Massoud Pedram, Peter A. Beerel
To evaluate the merits of our approach, we performed experiments with variants of VGG and ResNet, on both CIFAR-10 and CIFAR-100, and VGG16 on Tiny-ImageNet. The SNN models generated through the proposed technique yield SOTA compression ratios of up to 33. 4x with no significant drops in accuracy compared to baseline unpruned counterparts.