MULDE: Multiscale Log-Density Estimation via Denoising Score Matching for Video Anomaly Detection
We propose a novel approach to video anomaly detection: we treat feature vectors extracted from videos as realizations of a random variable with a fixed distribution and model this distribution with a neural network. This lets us estimate the likelihood of test videos and detect video anomalies by thresholding the likelihood estimates. We train our video anomaly detector using a modification of denoising score matching, a method that injects training data with noise to facilitate modeling its distribution. To eliminate hyperparameter selection, we model the distribution of noisy video features across a range of noise levels and introduce a regularizer that tends to align the models for different levels of noise. At test time, we combine anomaly indications at multiple noise scales with a Gaussian mixture model. Running our video anomaly detector induces minimal delays as inference requires merely extracting the features and forward-propagating them through a shallow neural network and a Gaussian mixture model. Our experiments on five popular video anomaly detection benchmarks demonstrate state-of-the-art performance, both in the object-centric and in the frame-centric setup.
PDF Abstract
ShanghaiTech
ShanghaiTech Campus
UCF-Crime
UCSD Ped2
UBnormal
| Task | Dataset | Model | Metric Name | Metric Value | Global Rank | Benchmark |
|---|---|---|---|---|---|---|
| Video Anomaly Detection | CHUK Avenue | MULDE-object-centric-micro | AUC | 94.3% | # 1 | |
| Anomaly Detection | CUHK Avenue | MULDE-object-centric-micro | AUC | 94.3% | # 1 | |
| Video Anomaly Detection | ShanghaiTech | MULDE-object-centric-micro | AUC | 86.7% | # 1 | |
| Anomaly Detection | ShanghaiTech | MULDE-frame-centric-micro | AUC | 81.3% | # 14 | |
| Anomaly Detection | ShanghaiTech | MULDE-object-centric-micro | AUC | 86.7% | # 2 | |
| Video Anomaly Detection | ShanghaiTech | MULDE-frame-centric-micro | AUC | 81.3% | # 2 | |
| Anomaly Detection | UBnormal | MULDE-frame-centric-micro-one-class-classification | AUC | 72.8% | # 2 | |
| Video Anomaly Detection | UBnormal | MULDE-frame-centric-micro-one-class-classification | AUC | 72.8% | # 1 | |
| Video Anomaly Detection | UCF-Crime | MULDE-frame-centric-micro-one-class-classification | AUC | 78.5% | # 1 | |
| Anomaly Detection | UCF-Crime | MULDE-frame-centric-micro-one-class-classification | AUC | 78.5% | # 1 | |
| Anomaly Detection In Surveillance Videos | UCF-Crime | MULDE-frame-centric-micro-one-class-classification | ROC AUC | 78.5% | # 12 | |
| Anomaly Detection | UCSD Ped2 | MULDE-object-centric-micro | AUC | 99.7% | # 1 | |
| Video Anomaly Detection | UCSD Ped2 | MULDE-object-centric-micro | AUC | 99.7% | # 1 |
